Quote:

cdimauro wrote: So, you cannot "tunnel" / "wrap" / "virtualize" every data structure created in a 64-bit environment to make it also accessible by 32-bit code.

Couldn't new apps use a new memory allocation memory attribute like MEMF_64BIT to allocate 64 bit memory while old apps would continue to get allocated memory in the 32 bit addressing range? The OS and structures without changes (other than 64 bit memory allocation functions) would all be allocated in 32 bit address space for compatibility.

Quote:

Last but not least, PowerPC 64-bit applications are slow on average compared with the same 32-bit ones, due to the large pointers and (usually) integers used, which create more caches and TLB pressures...

Sure, but it didn't even stop cellphones from going 64 bit with the iPhone 5s A7 Arm CPU where the extra power consumption matters and there is only 1GB of memory to address. Thumb2 has much better code density than PowerPC so 64 bit is needed even less on a phone. Ok, Apple wanted the new ARMv8 ISA and it doesn't eat too much battery being 64 bit. It's a really great marketing gimmick too. I guess PowerPC embedded applications have mostly gone 64 bit which I didn't expect. The low end "cheap" embedded must have gone to ARM but doesn't provide enough processing power for high end embedded yet? PowerPC does need lots of memory and caches so 64 bit makes sense. It's still barely worth the time and trouble to upgrade the OS to 64 bit compared to the large benefits of SMP and 3D. It always makes me wonder if the 68k had been further developed and enhanced (improving code density) whether there would be a need for 64 bit. I guess some sales guy somewhere wants to be able to sell a processor that is up to twice as fast.

Disclosure: I own Apple shares so buy the new Apple iPhone 5s with desktop class 64 bit technology that is up to twice as fast! Don't be stuck with 32 bit apps when you can own the worlds first 64 bit smartphone.

@TrevorDick

Thanks a lot for that information. I think this is really important.

_________________
Sometimes we give people a lot of credit just because they’re writing nice sentences even if it isn’t adding up to much.

@CodeSmith

Quote:

CodeSmith wrote: @cdimauro Quote: Windows is totally different: it hasn't a single, public and fully shared, address space for 32-bit applications, 64-bit applications, and even the o.s., like happens with the Amiga o.s.. Also Windows doesn't "publish" the internal data structure which it uses, and offers a different set of APIs for the 64-bit subsystems. That's not entirely true.

I've clearly talked about 32 & 64 bit applications and the o.s. regarding Windows. And what I said was all true.
Quote:

Windows still supports the win16 API

Only 32-bits versions of it.
Quote:

that was the main API used in all DOS-based versions of Windows.

Yes, but if you refer to Windows '95 (9x/ME in general) too, it was different: it used the DOS to "boot", and maintained the DOS box for backward compatibility. But its main API was Win32.
Quote:

Since DOS doesn't natively support an MMU, win16 applications all shared the same address space and could pass pointers to each other much like AmigaOS programs can.

"Could", but they aren't forced to do it.
Quote:

Private OS data was also exposed, see for example the SetWindowLong function which is a glorified poke into the internal data structures used to manage controls (in Windows everything is a window: actual windows, buttons, sliders, etc. It's like the base class of all controls).

That's not true. SetWindowLong is a completely controlled API: it's the o.s. which decides what to do when it's called. It can filter or even manipulate the passed data, or can use it as is.

That's the way that the "Compatibility tab" works for Windows Vista+: it can change the SetWindowLong behavior, in order to "emulate" the one of the previous o.s.; but everything is under the o.s. control...
Quote:

The way MS still supports that programming model in 32 bit Windows is interesting, and I expect that Hyperion's answer to running 32 bit apps in 64 bit AmigaOS will use similar techniques (the 3 second answer is that they sandbox all win16 apps into the same address space - it's obviously more complex than that since win16 apps can still interact with 32 bit apps in separate address spaces, but that's the basic technique).

Yes, it's interesting because the o.s. didn't exposed its internals, and provided APIs as an opaque interface. This way the internal implementation can change, even completely, without hurting compatibility. It's the o.s. responsibility to handle the passed data structures, and give back other ones to the applications. It's part of well defined API.

But I don't think that the same approach can be used for the Amiga o.ses.-/like, because they were thought with a completely different philosophy, and the o.s. structures are public and directly accessible.

You can think about sandboxing the 32-bit environment (o.s. & applications), and provide a complete new one for 64 bits, which DON'T expose its internal data structures and hopefully uses message passing by copy to finally implement address space isolation. But this way you will loose compatibility with the old software, and require brand new software to be developed (because the new APIs will be different; obviously you don't need to change them all).
Quote:

Win16 apps can't run in 64 bit windows because x86 CPUs can't easily mix 16 bit and 64 bit instructions like they can 16 and 32 bit or 32 and 64 bit.

The problem is that x64 LongMode cannot use at all 16-bit selectors. That's why the Win16 subsystem was dropped on the x64 version of Windows: it cannot execute such code. But you can use a virtual machine to run it, because it provides a correct emulation of the 16-bit environment in its sandbox.
Quote:

Finally, the API used to program 64 bit Windows programs is exactly the same that one uses for 32 bit. There are some new specialized memory management functions for dealing with the larger address space, but if you're writing a "normal" program like a text editor or an MP3 player all you have to do is pass different switches to the compiler.

Yes. Consider, also, that the concrete Windows APIs (the ones known as NT*) are not publicly exposed itself. They are hidden and used internally by the several subsystems (Win32, Win64, POSIX... and even OS/2). This way, again, the o.s. implementation can be completely changed, while the subsystems continue to work because they, and only they, are the publicly exposed interface.

This mechanism is so good that Windows offers even a POSIX subsystem (only for Professional+ versions, if I remember correctly) which can let develop and execute POSIX-compliant applications. In fact, it has also the (in)famous GNU utilities.
And in the past provided also an OS/2 subsystem...

But you have to think about this mechanism from the beginning, otherwise it'll be a HUGE work to do it when you decide to follow this way. I don't think that it'll be a practical way of the Amiga o.ses.-/like, because of the limited resource they have. Compatibility, also, will be killed.

Quote:

NutsAboutAmiga wrote: @cdimauro This highly theorical, but any way, there are ways this can be done, how Hyperion decides to do it we do not know, its not even on the road map yet.

They announced some kind of 64-bit support. But I think that it'll be something which I've already explained before.
Quote:

As for now we are waiting for SMP and 3D,

SMP support was announced years ago and "be ready" in 2 years. As usual, 2 years are well passed away and nothing was ready. Last but not least, some time ago Mr. Solie talked about "multi-core support", and tried to change the (consolidated, in the real world) meaning of the SMP term. So IMO you don't have to expect SMP support, but a form of AMP; which is a more simple way to let use more than one core, due to the Amiga o.s. structure.

For 3D, I think they are still wasting time supporting two different things. IMO you have put resources to the most promising one (for a non-mainstream o.s.), Gallium. Then you can provide other APIs on top of it, if needed.
Quote:

what is going to happen after that we do not know, If I may guess, I think the natural progression will be to make a pure 64bit version for AmigaOS for the brave.

Yes, it's a natural and expected way, but it's not easy: see my previous message.
Quote:

How and if the there will be backwords compatability is another issue, but as I see it should be possible given time and resources, to support 32bit in a 64bit OS, as you point out there are many differences, but consider this many of data structures in Exec was never intended to be poked around in anyway, its of no programs business to poke in the memory pools, or in the OS structures.

It's no true. They are publicly accessible, and even reading them is enough to hurt compatibility.

Anyway, you can "poke" them. Commodore even provided some assembly macros to change the task counter for quickly Forbidding, just to give the most common (and dangerous) example.
Quote:

All the programs should care about getting memory when they ask for it, where the memory is how the OS structures are organized and how the IPC is working is some thing that the OS will need to handel, its not program task to handel that it should only get what it expect.

I don't think so. It's much better to provide APIs with "opaque" data structure passing. The o.s. structures should NEVER be accessible: use the APIs instead if you want to set or get something, in a totally managed way. And it's better to have a message-by-copy IPC model to isolate the address spaces.
Quote:

Clearly there will be many differences in 64bit OS vs a 32bit OS.

Absolutely. There'll be new APIs, also.

Quote:

matthey wrote: Quote: cdimauro wrote: So, you cannot "tunnel" / "wrap" / "virtualize" every data structure created in a 64-bit environment to make it also accessible by 32-bit code. Couldn't new apps use a new memory allocation memory attribute like MEMF_64BIT to allocate 64 bit memory while old apps would continue to get allocated memory in the 32 bit addressing range? The OS and structures without changes (other than 64 bit memory allocation functions) would all be allocated in 32 bit address space for compatibility.

There's no need for a new flag, and that's from Amiga o.s. version 1.0.

You simple don't have to use MEMF_PUBLIC when you ask for memory, so the o.s. knows that it's a private stuff, which HOPEFULLY the application will NEVER share / pass to other applications.
Quote:

Quote: Last but not least, PowerPC 64-bit applications are slow on average compared with the same 32-bit ones, due to the large pointers and (usually) integers used, which create more caches and TLB pressures... Sure, but it didn't even stop cellphones from going 64 bit with the iPhone 5s A7 Arm CPU where the extra power consumption matters and there is only 1GB of memory to address. Thumb2 has much better code density than PowerPC so 64 bit is needed even less on a phone. Ok, Apple wanted the new ARMv8 ISA and it doesn't eat too much battery being 64 bit. It's a really great marketing gimmick too.

It was a very good move, because ARMv8 brought many enhancement to the ISA. ARM doubled both the general purpose and SIMD registers file; it also changed the ISA, removing some legacy / complex instructions, and also adding some other useful stuff.

Yes, it's something which AMD made 10 years ago when it introduced AMD64 AKA x86-64 AKA x64.

That's why actually ARM64 didn't lost performance due to the increased pointers and long integers sizes: they balanced the loss of performance with the new ISA features, and even gaining speed. Exactly what happened with x64.

Last but not least, Apple's ARM64 ISA is different from ARM's one, because they changed the pointers "meaning" to introduce some very interesting and useful things. 64-bit pointers only give 8GB of virtual address (which NOW is quite "enough" for anypne), leaving the remaining bits to be used as garbage collector's objects reference counting, plus quick data type check, and some other stuff. This change DRAMATICALLY improved speed for such called "managed languages / environments", such as Objective-C.

So, it wasn't just "hey, we give you 64 bits now!". No, it was a very smart move.
Quote:

I guess PowerPC embedded applications have mostly gone 64 bit which I didn't expect. The low end "cheap" embedded must have gone to ARM but doesn't provide enough processing power for high end embedded yet?

Don't know. Embedded is not my field of interest.
Quote:

PowerPC does need lots of memory and caches so 64 bit makes sense.

But the performance will drop, as I stated before. PowerPC 64-bit ISA didn't provide huge benefits, like x64 and ARM64 gave, to balance the loss and even provide better performance...
Quote:

It's still barely worth the time and trouble to upgrade the OS to 64 bit compared to the large benefits of SMP and 3D.

I already talked about them in my previous message. Anyway, at least AMP can be tried and provided quickly, with a small set of new APIs to enable it. It's an Amiga o.ses.-/like "friendly" way to do it; Amiga o.s. supported AMP from the beginning...
Quote:

It always makes me wonder if the 68k had been further developed and enhanced (improving code density) whether there would be a need for 64 bit. I guess some sales guy somewhere wants to be able to sell a processor that is up to twice as fast.

I don't think it'll get such great huge speed improvement, whatever a 68K 64-bit ISA can offer.

But I think that its code density can be affected by such extension, due to a prefix usage (as we discussed in the past, I think that it's the only way to introduce a 64-bit ISA based on the existing 68K). A prefix should be at least 16-bit wide, which is a lot of space.

The primary fault was by Motorola, which used the 11 (3) value of the size field to insert new instructions. Yes, they achieved very good code densities, but in the end they totally crippled a future 64-bit extensions...
Quote:

Disclosure: I own Apple shares so buy the new Apple iPhone 5s with desktop class 64 bit technology that is up to twice as fast! Don't be stuck with 32 bit apps when you can own the worlds first 64 bit smartphone.

ROFL

@embedded & telecom

Embedded SoC chips for Telecom nowdays need 64bit cores.
(a few year old designs already needed 2GB just to say "tada", I expect modern designs to be far above 4GB of RAM. Main chips currently used are PowerPC and MIPS.)

_________________
- KimmoK
// For freedom, for honor, for AMIGA
//
// Thing that I should find more time for: CC64 - 64bit Community Computer?

@cdimauro

Quote:

You simple don't have to use MEMF_PUBLIC when you ask for memory, so the o.s. knows that it's a private stuff, which HOPEFULLY the application will NEVER share / pass to other applications.

You can not count on it. In AmigaOS use of MEMF_PUBLIC was never enforced and Commodore never published documentation when MEMF_PUBLIC should be used. It is also quite mess because sometimes so called "system structures" are only used inside the application (for example hooks, internal messaging) and it is not clear if you should pass MEMF_PUBLIC allocated buffer to dos.library/Read().

Introducing MEMF_64BIT flag would be equally as bad as MEMF_SWAP flag is. When used wrong it might work right on some setups but fail on some other. The operating system should not leave such level of memory management to applications.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@KimmoK

Quote:

KimmoK wrote: @embedded & telecom Embedded SoC chips for Telecom nowdays need 64bit cores. (a few year old designs already needed 2GB just to say "tada", I expect modern designs to be far above 4GB of RAM. Main chips currently used are PowerPC and MIPS.)

Strange. I would never bet a cent that such devices require so much memory.

But, as I stated, I'm not interested in the embedded field...

@cdimauro

There's routers which have more than 4gig ram. But most of those have i3/i5 on custom made motherboards and cards. Also worth of mentioning that price range of those is north of €5K for just base unit. From cheaper stuff they are MIPS and PPC seems to be rarer every year...

Source - Me actually working in telecom.

@itix

Quote:

itix wrote: @cdimauro Quote: You simple don't have to use MEMF_PUBLIC when you ask for memory, so the o.s. knows that it's a private stuff, which HOPEFULLY the application will NEVER share / pass to other applications. You can not count on it. In AmigaOS use of MEMF_PUBLIC was never enforced and Commodore never published documentation when MEMF_PUBLIC should be used.

That's what I've found:

Memory that must not be mapped, swapped, or otherwise made non-addressable. ALL MEMORY THAT IS REFERENCED VIA INTERRUPTS AND/OR BY OTHER TASKS MUST BE EITHER PUBLIC OR LOCKED INTO MEMORY! This includes both code and data.

So this flag should be set for sharing memory with other tasks, which is the most important thing, since it affects their "collaboration".
Quote:

It is also quite mess because sometimes so called "system structures" are only used inside the application (for example hooks, internal messaging) and it is not clear if you should pass MEMF_PUBLIC allocated buffer to dos.library/Read().

An Amiga o.s. exposes its services through the APIs of our beloved libraries. The question is here to define how they should be used.

Since dos.library is part of the o.s., IMO it should be able to access any kind of memory.

For non-system libraries the question is different. I think that in principle they should require public memory. But its just my opinion.
Quote:

Introducing MEMF_64BIT flag would be equally as bad as MEMF_SWAP flag is. When used wrong it might work right on some setups but fail on some other. The operating system should not leave such level of memory management to applications.

I absolutely agree.

May be it's better to define the MEMF_PUBLIC usage. Or, otherwise, define a new ad hoc flag.

Quote:

Rose wrote: @cdimauro There's routers which have more than 4gig ram. But most of those have i3/i5 on custom made motherboards and cards.

It's incredible for me: I never thought about using x86 processor for such stuff.

Good to know.
Quote:

Also worth of mentioning that price range of those is north of €5K for just base unit. From cheaper stuff they are MIPS and PPC seems to be rarer every year...

So PowerPC is losing market, but MIPS? Is it still on track, or losing market as well?
Quote:

Source - Me actually working in telecom.

Very reliable source.

@cdimauro

Quote:

Memory that must not be mapped, swapped, or otherwise made non-addressable. ALL MEMORY THAT IS REFERENCED VIA INTERRUPTS AND/OR BY OTHER TASKS MUST BE EITHER PUBLIC OR LOCKED INTO MEMORY! This includes both code and data.

I never use interrupts in my code. When writing single threaded applications I am not referencing memory by other tasks either.

So rule is: if you dont use interrupts and write single threaded applications you should never use MEMF_PUBLIC.

If you use interrupts or write multithreaded applications you should always use MEMF_PUBLIC everywhere unless you can properly isolate code portions not shared by multiple tasks.

Right?

(Trying to say those MEMF_PUBLIC rules never made any sense...)

Anyway, with 64-bit it is better make clean break.

Last edited by itix on 26-Jan-2014 at 11:47 AM.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@cdimauro

They're all losing market share to Intel... it even says so in the PowerPC roadmap document :D

_________________

@Birbo

I had a talk with someone @power.org board, talking about the requisites to become a power org member.

I sent the informations to A-eon, but A-Eon was already member at that time (A year ago, i guess). I sent the informations to ACube too, but don't know if they have done the necessary steps to partecipate (haven't asked again)

_________________
retired

@itix

Quote:

When writing single threaded applications I am not referencing memory by other tasks either.

Do you ever read any data from disk?

@itix

Quote:

itix wrote: @cdimauro Quote: Memory that must not be mapped, swapped, or otherwise made non-addressable. ALL MEMORY THAT IS REFERENCED VIA INTERRUPTS AND/OR BY OTHER TASKS MUST BE EITHER PUBLIC OR LOCKED INTO MEMORY! This includes both code and data. I never use interrupts in my code. When writing single threaded applications I am not referencing memory by other tasks either. So rule is: if you dont use interrupts and write single threaded applications you should never use MEMF_PUBLIC. If you use interrupts or write multithreaded applications you should always use MEMF_PUBLIC everywhere unless you can properly isolate code portions not shared by multiple tasks. Right?

Absolutely. It's the correct way to deal with the allocated memory.

My only concern is about the malloc() function. Many applications just use it as is when they require memory, so I think that the compiler will generate all MEMF_PUBLIC requests, which solves the problem (everything is shared), but it's not always desirable.
Quote:

(Trying to say those MEMF_PUBLIC rules never made any sense...)

I think that the main problem here is that there's a missing / unhandled scenario: sharing memory by a group of threads.

That's because an Amiga o.s.-/like has basically one big process which shares its address space with all the threads (our tasks), and can only allow to share resources with all threads or none.

So, since there's no process concept, there's also no group of threads concepts. Sharing resources between them becomes very difficult, since the only official way is using MEMF_PUBLIC for the allocated memory, which is not desirable.

There's something like MEMF_PRIVATE_SHARED (!) or MEMF_COMMON_TASKS flag which will be useful, but actually we don't have it and in any case the o.s. will not be able to handle this kind of memory allocation without introducing the proper process concept.

The very lightweight Amiga o.s. nature (all threads) is actually one of its big disadvantages on writing modern applications.
Quote:

Anyway, with 64-bit it is better make clean break.

I totally agree. Introducing patches to a 30 years old o.s. to enable / allow some new features will make it a big patchwork which will worsen the situation.

It's much better to intern the existing applications in a sandbox, and define a brand new and modern o.s. with its APIs/ABIs which are largely imported or inspired by the Amiga o.s..

@umisef

It is not documented. I know that dos.library is passing pointers to the file system task but it is not my fault. Either documentation is wrong or implementation is wrong.

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

@WolfToTheMoon

Quote:

WolfToTheMoon wrote: @cdimauro They're all losing market share to Intel... it even says so in the PowerPC roadmap document :D

ROFL

Again, unbelievable, since x86 was not a big win for the embedded market. May be the software complexity is so great now, that even a complex CISC is convenient in doing some tasks which in the past were in the complete RISC dominion.

Good.

P.S. I haven't downloaded the roadmap document. It's absolutely stupid to require an e-mail to download something which contains just public information. I can understand it for NDAs / confidentials information, but not for public ones.

@umisef

Quote:

umisef wrote: @itix Quote: When writing single threaded applications I am not referencing memory by other tasks either. Do you ever read any data from disk?

It's supposed that an o.s. is able to access all kind of memory,

So using its APis for such tasks IMO shouldn't require a MEMF_PUBLIC only memory to be passed.

@cdimauro

Quote:

the confusion about MEMF_PUBLIC

To solve the issue whit confusion of MEMF_PUBLIC,
MEMF_SHARED in AmigaOS4.0 introduced, latter on its no longer program that should define the memory to used, now whit AllocSysObjectTags(ASOT_LIST, TAG_END), the memory type is defined by the OS.

Quote:

It's no true. They are publicly accessible, and even reading them is enough to hurt compatibility.

That is not the biggest issue, the missing CIAA, CIAB, Paula, Alice chips is bigger issue, besides some of ExecBase structure are now forbidden to read from according to AmigaOS4.0 SDK.

Quote:

Matthey wrote: Quote: Couldn't new apps use a new memory allocation memory attribute like MEMF_64BIT to allocate 64 bit memory while old apps would continue to get allocated memory in the 32 bit addressing range? The OS and structures without changes (other than 64 bit memory allocation functions) would all be allocated in 32 bit address space for compatibility. There's no need for a new flag, and that's from Amiga o.s. version 1.0.

The thing is that AllocVec()/AllocVecTags() and AllocMem() return normal 32bit pointers, if MEMF_64BIT flag was introduced then function will need to return the memory byref in parameter (But normal tag lists are 32bit wide so it want work), my guess it that instead of creating this confusion, they probably make a new method for Exec.library called AllocVecTags64() or some thing like that.

By having more methods, I belive the issue of where the memory is going to be allocated is solved.

Quote:

But the performance will drop, as I stated before. PowerPC 64-bit ISA didn't provide huge benefits, like x64 and ARM64 gave,

Well, 64bit wont give faster programs as you say, is half true, but there is a issue that is different on PowerPC the issue about Risk architecture is that some instructions are missing on G5, Pa-Semi chips, and normal 32bit program compiled for older PowerPC chip is not running optimal on the newer G5, Pa-Semi because of illegal instruction hits.

Having binary’s that are optimized for G5, and Pa-Semi will be huge benefit, anyway the GCC 4.2.x we have is too old to take advantage of chips now.
Basically we are seeing the same issues we did have on 68060 processor on G5 / Pa-Semi PA6T, and possibly on the QorIQ serials of CPU's

Anyway Microsoft has moved in different direction when comes to programing langauge for years, C# .NET and VB .NET, this are programming languages that live inside a runtime, and so memory, DLL library’s and OCX files all become C# classes, moving away from CPU dependency to a generic binary format that is compiled on runtime, allowing programs to not think about memory and all that.

Last edited by NutsAboutAmiga on 26-Jan-2014 at 02:09 PM.
Last edited by NutsAboutAmiga on 26-Jan-2014 at 02:06 PM.

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS