This is a question for Hyperion developers.. I dont completely understand the situation, but here goes:

Correct me if I'm wrong, but full memory protection in AmigaOS where each program has its own isolated memory space will break all current applications as AmigaOS currently just has one big address space, which makes IPC possible?

When full memory protection finally makes it into AmigaOS (version 4.2 or so?), in what ways will AmigaOS be different to code for? How does AmigaOS currently work, and how will it work when full MP is implemented?

Pardon my ignorance, guys. :/

@jahc

Quote:

Correct me if I'm wrong, but full memory protection in AmigaOS where each program has its own isolated memory space will break all current applications as AmigaOS currently just has one big address space, which makes IPC possible?

Basically, yes. The current message passing IPC relies on any pointers in the message being absolute, and therefore may reference data outside the message (usually does; intuition's IDCMPWindow is a good/bad example).

Quote:

When full memory protection finally makes it into AmigaOS (version 4.2 or so?), in what ways will AmigaOS be different to code for? How does AmigaOS currently work, and how will it work when full MP is implemented?

Slightly. The basic API may stay, but there will be much stricter rules that need to be adhered to. Old programs can obviously never have full MP, so they will need to run "sandboxed" from the others. We have some ideas about how this can go about, but things need to be tried to see if this works out.

One thing that would need to change is the usage of absolute pointers in messages. Related to that we would like to gradually get rid of any open system structures. Their open-ness is already a problem when you want to extend them, and absolute pointers are a problem when you have different address spaces (of course, it doesn't affect local resources).

It will definitely not come overnight

_________________
Seriously, if you want to contact me do not bother sending me a PM here. Write me a mail

@Rogue

What is an "absolute" pointer?

Btw, will getting rid of open system structures help improve stability? i.e. make it harder to stuff up the system internally? Thats one thing the MOS guys were going on about a few years ago I think. Full memory protection should also help make the OS harder to crash as well, right?

@jahc

An absolute pointer:

Let's say, you have an address space of 4GB. You can address every byte in that address space with a 32 bit pointer. That is, you can put all the numbers from 0 to 4.294.967.295 in 32 bit. If you use a 32 bit pointer like that to address every byte in the 4GB address space, you have an absolute pointer. You see, no other program logic is needed to address any byte in your memory.

If you use a relative pointer on the other hand, it will address relative from a boundry (I'm not sure, if this is the right term). If your relative pointer is 24 bit, you can address 16MB. But if you still have a 4GB address space, you can have several of these 16MB areas, and every relative pointer will address in its own 16MB area. To do this, you need som program logic to take care of the final addressing.

Am I bad at explaining or what?
I hope, you get something from this.

Sincerely
John.

@Geomol

Oh okay, I think I understand. Thanks for explaining that for me.

@Geomol

Or you can (will, as it works that way ) have one 32bit address space per application (virtual memory, remember?), and map ANYTHING into memory. For instance say you're opening a 10MB file from harddisk, just mmap() it and read from memory. Run out of physical memory? Doesn't matter, the system will give you swap instead. Whenever you need to access something that exists in an absolute address map (say a serial port) you go via a driver in either IO or mmap() mode. Direct hardware access becomes impossible in a virtual address space...

If you on the other hand create a 64 bit address space, you can go back to using a single address space again. I saw a rather nice description here:

http://c2.com/cgi/wiki?SingleAddressSpaceOperatingSystem

Favorite quote:

Quote:

Consider that 40 bits can address a terabyte, two orders of magnitude beyond the primary and secondary storage capacity of all but the largest systems today, and that a 64-bit address space, consumed at a rate of 100 megabytes per second, would last five thousand years.

Basically you would be able to map a few million harddrives into memory addresses and still have enough addresses left to run quite a lot of applications


Edit:
I'm thinking maybe this is some of what Fleecy was talking about for OS5... Remember the talk about orthogonal persistence? 64 bit addresses is in no way needed for CURRENT systems, but the things it makes possible changes the whole theory behing operating systems...

_________________
This weeks pet peeve:
Using "voltage" instead of "potential", which leads to inventing new words like "amperage" instead of "current" (I, measured in A) or possible "charge" (amperehours, Ah or Coulomb, C). Sometimes I don't even know what people mean.

@jahc

Quote:

Full memory protection should also help make the OS harder to crash as well, right?

The problem right now is that if your program crashes, you don't even know if it's really a bug in your program or not. Somebody else might have killed off your data structures.

One extremely popular instance of this bug is the stack overflow: A program has it's stack allocated directly behind a block of data that belongs to your program. Now if the other program overshoots it's stack, it will kill off your data, and chances are the other program will continue to work, but your program will find corrupted data, and might crash (it might try to reference a value as a pointer that's now invalid).

Other instances are programs shooting down the system by killing internal system structures, either by accident or "design" (because they assume they know how they look like).

None of these problems can be fully cured in the current system. It's possible (and will be the first step) to make some private memory segments for programs (so that no other program sees them and nobody but the program owning them can access these).

The biggest issue is protection of the system itself, because a lot of programs still rely on access to the system strucutres...

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

@EntilZha

1:
Protecting your own structures. good idea, wouldn't mess up other programs to any degree (writing to illegal memory would either generate a warning or an error, depending on some setting by the user, perhaps?)

2:
Protecting the internal structures of the OS:
Also a good idea, but would break some programs.
The question is, do we care? Yes, some users might want to run some obscure application that is inherently broken and does things it shouldn't be doing. The solution is simple: Don't upgrade your OS. It's like Windows95/NT. If you need to run a program that accesses things in a DOS'y manner, don't install NT (serial ports, for instance).

So I personally do not see the problem. Yes, it would break some applications, but the idea is to have some progress, no? If there is enough interest in an application, it would just have to be updated to work with the newer versions of the OS. If not, then there is obviously no point in supporting that application (if noone is interested in it, I mean).

_________________
This weeks pet peeve:
Using "voltage" instead of "potential", which leads to inventing new words like "amperage" instead of "current" (I, measured in A) or possible "charge" (amperehours, Ah or Coulomb, C). Sometimes I don't even know what people mean.

@olegil

It may be a good idea to include in the preferences of the OS the possibility of turning on and off the protection of internal structures of the OS so people who wants a 100% stable system will be able to turn it on, but people who prefers better compatibility will be able to run their old apps.

Everyone wins

the OS4.2 API should produce binaries that use a different messaging system, so starting from 4.2 the user will have the option of only allowing pure 4.2 apps to run in the OS.

And we may have something like the compatibility settings in WindowsXP and allow concrete apps to access directly to the OS internal structures... (if we know that the App is very well programmed and won't fill with trash the internal structures).

But the new message system should be added as soon as possible so we can be sure that pure OS4 apps won't access directly to the OS internal structures and will use other ways to work.

I mean... I want an option to run memory protected OS4 apps only and I'd like to have the option of allowing concrete 3.x apps to read/write the internal structures

_________________
The only spanish amiga news web page/club: CUAZ

@EntilZha
Quote:

The biggest issue is protection of the system itself, because a lot of programs still rely on access to the system strucutres...

Idea:
How about protecting the system data structures for normal compliant programs and creating a special "wrapper" program only for pre-OS4 applications. It would mirror the data structures so that the app run under it could directly access them, and transmit modifications to the OS4 data structures. It would be invoked only at the user's discretion: directly run by the user (or set up for an app by an installer).
- Yes, it would violate system security, but it would be the user's decision to "live dangerously".
- There would be no OS/System overhead for applications that do not require it.
- I don't think that it would be a true "sandbox" because it wouldn't emulate the old OS, just provide an "adapter"

This is just an idea, I provide absolutely no warrantee to its usefulness, sanity (or mine ), and applicability

Regards,
JCC

@olegil

What's easy to forget is that the OS currently in many places practically requires applications to read certain system structures directly because it doesn't provide any alternatives . When I program I try to make it a rule use OS calls for reading/ writing whenever possible but often the OS just leaves no other choice.

@JCC

No, for me the best idea is use the JIT emulator for the pre OS4 applications and develop the new only in OS4 Native mode. Because the emulator is legacy to the memory protection system and the old application is executed in emulator process with memory protection system.

Sorry for my poor english

Isn't the main reason for implementing memory protection to reduce damage caused by badly behaving programs? If so, what's the point of adding it if it breaks all those programs that were the reason for adding it in the first place , along with most of the OS and most of the wellbehaved ones as well?

@JCC

"It would mirror the data structures so that the app run under it could directly access them, and transmit modifications to the OS4 data structures."

Yes, that would be a good idea. OS4 may notice modifications in the mirror structures with the MMU and evaluate if the program that is accessing these structures is doing nasty stuff or not... if it doesn't then the OS could change the real structures.

_________________
The only spanish amiga news web page/club: CUAZ

@salass00

Uhm, and is it impossible to fix that first?
No.

_________________
This weeks pet peeve:
Using "voltage" instead of "potential", which leads to inventing new words like "amperage" instead of "current" (I, measured in A) or possible "charge" (amperehours, Ah or Coulomb, C). Sometimes I don't even know what people mean.

@olegil

Did I ever say it was impossible ?

I just think this "feature" is a total waste of time and resources. I do see the point of less open OS structures, but do we really need memory protection ?

@salass00

I think in this day and age of viruses getting out of control, obscurity will not protect Amiga forever. We will absolutely need security in future versions of AmigaOS. Otherwise viruses can patch into device drivers and take over from there.

@EntilZha

Quote:

One extremely popular instance of this bug is the stack overflow: A program has it's stack allocated directly behind a block of data that belongs to your program. Now if the other program overshoots it's stack, it will kill off your data, and chances are the other program will continue to work, but your program will find corrupted data, and might crash (it might try to reference a value as a pointer that's now invalid).

Speaking about stack, how does AOS4 automatic stack enlarging code deal with evil cases like:

- stack allocated by app (StackSwap)

- stack as part of a library's libbase struct

- stack inside some struct "surrounded" by other data.

- manual stack pointer swapping (not through StackSwap)

Quote:

but do we really need memory protection?

I would say so. You just can't have in this day an age the ability for programs to trash each others memory spaces. I thought one of the Amiga's plus points is stability and security. When I was using 95 I had to reboot the machine three times a day. With NT4 it was once a month even though the apps crashed almost as much. I would like to think that was down to its memory protection. Whenever I did some Amiga coding I had to reboot often because of my dodgy pointer bugs. (Must try better. ) I would like to see the end of that.

Due to the better design of the Amiga OS it doesn't get its knickers in a twist as often as Windows so you don't get as much crashes. However I would still like the Amiga to handle crashes better (which it does with OS4).

I a bit confused now though. I thought OS4 had memory protection. When I see the Grim Reaper in action I thought that is usually when a program tries to access some memory it should'nt have. So what level of memory protection does OS4 have and what is there to come with OS4.1 and OS4.2?

_________________

@Intuitioned

I have to say I disagree with you here. Real stability should come from robust bugfree software not be added as some kind of afterthought. It just doesn't work that way.
Quote:

I a bit confused now though. I thought OS4 had memory protection. When I see the Grim Reaper in action I thought that is usually when a program tries to access some memory it should'nt have. So what level of memory protection does OS4 have and what is there to come with OS4.1 and OS4.2?

I would guess it's about the same as with Enforcer/ MuForce and MungWall under OS3.x, only Grim Reaper has a fancy GUI. AFAIR one of the Grim Reaper screenshots had 0xDEADF00D (I think that was the one) as the contents of a register, which is a sign of MungWall.