Quote:

NutsAboutAmiga wrote: @Dandy Quote: hanks for this information. NutsAboutAmiga might be interested in this. Well its possible that JIT compiler breaks some thing that should be atomic, but it should not, propper synchronistic should be provided by native Exec functions for Mutex/Semaphore . I say the design is broken if they depend or hope some 68k assembler opcode is atomic.

There 2 problems here. The first is that 68Ks had a few atomic instructions, but they were substantially forbidden by Commodore due to possible conflicts with the chipset. Of course you can use them on fast ram, but most Amiga had no fast ram. Instead, they had most crappy thing on the world: slow ram; a memory which has all the limits of the chip ram without having the benefits of it. Since the o.s. has to run on any machine, so even on machines with chip ram and/or slow ram only, they cannot use instructions like TAS, for example, to guarantee the atomicity of certain operations. So they used something like the classic ADDQ, which is NOT guaranteed to be atomic.

The second, and most important thing, is that they used such kind of instruction as a PUBLIC method to forbit/permit operations, offering MACRO to the coders. The net result is that you cannot patch APIs like Forbid & Permit using internally proper atomic operations, because many coders has used the public method to directly access the proper task counter on ExecBase, so patching the APIs has no effect.

That's why implementing SMP is a damn thing on an Amiga/-like o.s..

Anyway, it's part of the long list of crappy things and mistakes that Commodore engineers have made...

@NutsAboutAmiga

Quote:

Well its possible that JIT compiler breaks some thing that should be atomic, but it should not, propper synchronistic should be provided by native Exec functions for Mutex/Semaphore . I say the design is broken if they depend or hope some 68k assembler opcode is atomic.

This is nonsense.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@cdimauro

Quote:

So they used something like the classic ADDQ, which is NOT guaranteed to be atomic.

Uhm, in context of AmigaOS it is atomic. Other task can not modify value in target address while another task is executing ADDQ.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

Quote:

itix wrote: @cdimauro Quote: So they used something like the classic ADDQ, which is NOT guaranteed to be atomic. Uhm, in context of AmigaOS it is atomic. Other task can not modify value in target address while another task is executing ADDQ.

Unfortunately ADDQ (and many other instructions) isn't atomic even considering the Amiga o.s./-like context.

When a 68K executes an ADDQ, it:
- first makes one access to memory to read the location;
- does the calculation internally, using the ALU;
- makes another access to the memory to write the new value in the location.

Just after the first step another peripheral can take ownership of the bus and write a new value in the location.

A peripheral can be the chipset, if the location is in the chip ram.
It can be the logic of a cartridge, like the Action replay, for example.
Or it can be a PowerPC processor running in parallel on the expansion card (it has access to the main memory, if I remember correctly).

It's quite unlikely, of course, but from a technical point of view it can happen.

But projecting everything on an SMP system, well, it becomes quite likely, especially if the considered location(s) is one of most (ab)used...

@cdimauro

Quote:

Unfortunately ADDQ (and many other instructions) isn't atomic even considering the Amiga o.s./-like context. When a 68K executes an ADDQ, it: - first makes one access to memory to read the location; - does the calculation internally, using the ALU; - makes another access to the memory to write the new value in the location. Just after the first step another peripheral can take ownership of the bus and write a new value in the location. A peripheral can be the chipset, if the location is in the chip ram. It can be the logic of a cartridge, like the Action replay, for example. Or it can be a PowerPC processor running in parallel on the expansion card (it has access to the main memory, if I remember correctly).

This is of course possible but then you are also having cache coherence issue.

Therefore you have only two choices: 68k instructions are always atomic or you flush/invalidate caches and synchronize memory access with other peripherals.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@itix

Exactly, but the problem is that the real hardware works in a non-atomic way in the first case, and that the existing software cannot be changed for the second case.

Emulation is different and you have much more control, so you can decide that 68K instructions are executed atomically. But if you want to emulate another, concurrent processor (running in another core), it means that you have to pay a big price to do it in performance terms. And JITing 68K code brings other problems for the atomicity of the instructions.

It's not a trivial task trying to execute the existing code on multi-core system while keeping coherence accessing some critical data structures.

@cdimauro

Quote:

Emulation is different and you have much more control, so you can decide that 68K instructions are executed atomically. But if you want to emulate another, concurrent processor (running in another core), it means that you have to pay a big price to do it in performance terms. And JITing 68K code brings other problems for the atomicity of the instructions.

I am not sure if 68k instructions are executed atomically on OS4 or MorphOS. On MorphOS it is possible, at least in theory, to synchronize 68k instruction stream with PPC native code. But I dont know if it is implemented to work like that.

With multicore this is not possible. At least not without sacrificing performance.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@itix

I don't know how Petunia and Trance work, but I don't think that they have any sort of atomicity for 68K instructions: this will hurt A LOT the performance, and they are well known to be fast.

@cdimauro

Not necessarily. If atomicity is implemented using atomic instructions in PPC then it would be very very slow. Instead Exec scheduler could be programmed to complete 68k instruction before task switching. Since emulation is part of kernel it can be done.

But I have my doubts anyone cares so much of atomicity. So far 68k programs have been very robust and stable.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@itix

Quote:

Instead Exec scheduler could be programmed to complete 68k instruction before task switching.

Well theoretically the JIT compiler can be intelligent add protection, but even so, will it be atomic in case of Interrupt. Or will you just get a dead lock.

Sure forbid() and permit() can solve it and programs use it where they should its not a problem, but its not the best way to do it, Mutex/Semaphore are better for the job as they do not halt multitasking.

Quote:

But I have my doubts anyone cares so much of atomicity. So far 68k programs have been very robust and stable.

I will think thats not many programs that depend on atomicity, or at lest the dice is mostly rolled in the users favor.

Anyway my idea of atomicity being a plausible cause of the problem is just me making a wild guess at what I think is plausible explanation.

Anyway some people say they have IB2.5, I belive this is a inofficial version, and they say they have no problem with it. I don't know if its PPC or 68k, I have not asked.

As long as programs are in developement and the developers care to fix the problems there is workaround for any problem I belive.

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS

@itix

You're right: if you keep track of the instructions executed to emulate a 68K one, you can force its atomicity at a relatively cheap price (the burden is moved to the scheduler, but the overall execution is very fast).

However it has to be seen what happens with the PowerPC applications. If there's nothing that prevents them to access the o.s. structures without locks et similia, then it doesn't make sense to force the atomicity of execution for the 68K instructions.

@NutsAboutAmiga

Applications can avoid using Forbid() and Permit(), and directly access the nested task counter. This is primary problem regarding implementing SMP on an Amiga o.s./-like.

@cdimauro

Quote:

Applications can avoid using Forbid() and Permit(), and directly access the nested task counter. This is primary problem regarding implementing SMP on an Amiga o.s./-like.

But do they in practice? How many common / useful apps do this? Are apps coded at this low level likely to work any on modern systems? It would be a PITA if things like Wordworth or ImageFX stopped working but not much loss if some rarely used old 1.3 tool went.

I can't help feeling this is a theoretical consideration rather than a practical one.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@NutsAboutAmiga

Quote:

Quote: Instead Exec scheduler could be programmed to complete 68k instruction before task switching. Well theoretically the JIT compiler can be intelligent add protection, but even so, will it be atomic in case of Interrupt. Or will you just get a dead lock.

On MorphOS you dont have hard interrupts in the ABox execution environment. Interrupts are routed via Exec task scheduler.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@cdimauro

Quote:

However it has to be seen what happens with the PowerPC applications. If there's nothing that prevents them to access the o.s. structures without locks et similia, then it doesn't make sense to force the atomicity of execution for the 68K instructions.

If PowerPC applications are using atomic instructions it would work. There is only one core so it is executing either 68k code (in emulation) or native PPC code. You just have to ensure 68k instruction is completed before task switching (to PPC native code or another 68k emulated task) can occur.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

Why are we talking about 68k ? What about PPC (only)? Would it change anything ?

Latest version of OSX doesn't even include Rosetta PPC emulation. I wish MOS/OS4 were moving forward as well.. Always looking backwards and keeping compatibility even though it's not needed anymore doesn't encourage evolution :(

Last edited by Leo on 23-Aug-2014 at 11:44 AM.

_________________
http://www.warpdesign.fr/

@broadblues

Quote:

broadblues wrote: @cdimauro Quote: Applications can avoid using Forbid() and Permit(), and directly access the nested task counter. This is primary problem regarding implementing SMP on an Amiga o.s./-like. But do they in practice? How many common / useful apps do this? Are apps coded at this low level likely to work any on modern systems? It would be a PITA if things like Wordworth or ImageFX stopped working but not much loss if some rarely used old 1.3 tool went. I can't help feeling this is a theoretical consideration rather than a practical one.

I've no idea. There are macros defined for directly accessing IDNestCnt without the function call overhead, so it's likely that someone did it to save some clock cycles.

Running in single core system it's difficult to see if some application did it: there's no evident side-effect.

Maybe a modified version of UAE can do the job and keep track of the "offenders" (albeit they were legal apps).

@itix

Quote:

itix wrote: @cdimauro Quote: However it has to be seen what happens with the PowerPC applications. If there's nothing that prevents them to access the o.s. structures without locks et similia, then it doesn't make sense to force the atomicity of execution for the 68K instructions. If PowerPC applications are using atomic instructions it would work. There is only one core so it is executing either 68k code (in emulation) or native PPC code. You just have to ensure 68k instruction is completed before task switching (to PPC native code or another 68k emulated task) can occur.

Yes, but... is it sure the PowerPC applications don't do like the 68K ones? IF 68K apps do it, of course.

@Leo

Quote:

Leo wrote: Why are we talking about 68k ?

Because it's still lovely used.
Quote:

What about PPC (only)? Would it change anything ?

See above: it depends if the PowerPC applications are polite or not.
Quote:

Latest version of OSX doesn't even include Rosetta PPC emulation. I wish MOS/OS4 were moving forward as well.. Always looking backwards and keeping compatibility even though it's not needed anymore doesn't encourage evolution :(

Moving forward means running PowerPC-only apps?

@cdimauro

Quote:

Moving forward means running PowerPC-only apps?

Moving forward would mean leaving PPC and going ARM/x86 road. But since no one seems to be ready for that, a first step would be to forget about 68k, at least for the OS...

Quote:

See above: it depends if the PowerPC applications are polite or not.

Not polite apps should not be possible. I don't get why developers didn't use the PPC switch as an opportunity to prevent developers from writing bad apps by locking API, structures,...

Last edited by Leo on 23-Aug-2014 at 03:49 PM.

_________________
http://www.warpdesign.fr/

@cdimauro

Quote:

Quote: If PowerPC applications are using atomic instructions it would work. There is only one core so it is executing either 68k code (in emulation) or native PPC code. You just have to ensure 68k instruction is completed before task switching (to PPC native code or another 68k emulated task) can occur. Yes, but... is it sure the PowerPC applications don't do like the 68K ones? IF 68K apps do it, of course.

I dont get what you mean here. PPC applications can use OS structures without locks. It makes no difference is task PPC native or 68k native. If you modify OS structures it must be done atomically or use locking mechanism, Forbid() or semaphores.

This if you are running OS4 or MorphOS.

On OS3 PPC can not access OS structures at all because there are two separate CPUs running simultaneously. This is not case in OS4 and MorphOS and cache coherency is maintained.

(edit: looks like this forum software has limit for nested quotes)

Last edited by itix on 23-Aug-2014 at 07:03 PM.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@Leo

Quote:

Leo wrote: @cdimauro Quote: Moving forward means running PowerPC-only apps? Moving forward would mean leaving PPC and going ARM/x86 road. But since no one seems to be ready for that,

AROS already did it. It's OS4 and MorphOS that are stick with PowerPCs...
Quote:

a first step would be to forget about 68k, at least for the OS...

Many people still uses 68K applications. Also new FPGAs will refresh and revamp the 68K machines.

What do you think about it?
Quote:

Quote: See above: it depends if the PowerPC applications are polite or not. Not polite apps should not be possible. I don't get why developers didn't use the PPC switch as an opportunity to prevent developers from writing bad apps by locking API, structures,...

The problem is/was Amiga o.s., which let all this. Something which is not explicitly prohibited could be allowed.

But I don't know how are the official guidelines for the PowerPC o.ses. Since they run 68K apps as "first-citizens", maybe the same old guidelines apply for all applications. See below for a discussion about this point.

@itix

Quote:

itix wrote: @cdimauro Quote: Yes, but... is it sure the PowerPC applications don't do like the 68K ones? IF 68K apps do it, of course. I dont get what you mean here. PPC applications can use OS structures without locks. It makes no difference is task PPC native or 68k native.

OK, that's what I was asking for.
Quote:

If you modify OS structures it must be done atomically or use locking mechanism, Forbid() or semaphores. This if you are running OS4 or MorphOS.

Fine, but a question here to better clarify: are PowerPC applications allowed to directly access and modify IDNestCnt, for example, as 68K apps did?

I mean: ADDQ was/is not really atomic on 68K, as we discussed before.

PowerPCs has to execute 3 instructions to do the same: the load, the increment, and then the store. They are not atomic, and a scenario similar to 68K can happen (with external peripherals accessing the same location, for example). You talked about atomic instructions for PowerPCs, so I assume there's a way to "own" changing a memory location (which is a common feature for processors). So, the real question is: there are guidelines for PowerPC applications which force them to use such atomic instructions to change o.s. data like IDNestCnt?

If the answer is yes, I think that it isn't a big problem enabling SMP on OS4 or MorphOS.
Quote:

On OS3 PPC can not access OS structures at all because there are two separate CPUs running simultaneously.

Strange. I thought that the PowerPC had access to whole Amiga machine memory.
Quote:

This is not case in OS4 and MorphOS and cache coherency is maintained.

With a single processor running is easy...
Quote:

(edit: looks like this forum software has limit for nested quotes)

Unfortunately. I don't know if it can be removed, because it limits following the discussion.

P.S. One interesting thing of x86 & x64 processors is that you don't need to use atomic instructions (with the LOCK prefix) if the memory location accessed by a read-modify-write instruction is under certain conditions: a single byte at any address, or a 16/32/64 word aligned at a 16/32/64 bit address. It can let implementing some data structures without speed penalties.