@pavlor

Quote:

pavlor wrote: I use PCI sound and Ethernet cards in WinUAE config.

OK, but those should take a really small amount of address space: losing around 1GB only for them is too much.
Quote:

Point of "Amiga NG" is to use Amiga enviroment as much as possible, not as mere another application on host OS. The less the host OS is visible, the better. This is AmigaWorld, you get used to that things doesn't make sense at all. ;) Logic, reason or common sense aren´t qualities one can expect form Amiga users. Pure passion, nothing less, nothing more.

And I cannot fight against irrationality. I give up here...

Quote:

cdimauro wrote: AFAIK they didn't turned on protection for code sections due to incompatibilities found with some applications. But I might recall wrong. It'll be good to have some reliable information here.

I don't know of any compiler that writes to code sections and PPC assembler code is rare. Self modifying code was frowned upon and rare already in the 68040 and 68060 days. I wouldn't think there would be any problem with making PPC code sections read only anyway. Existing violations are more likely to find bugs which is one of the reasons to make code sections read only in the first place.

Quote:

cdimauro wrote: Quote: Sadly, I don't think AmigaOS 4 can afford to drop 68k compatibility but moving to a little endian architecture (AArch64 or x86_64) would practically force them to. The same happens with 32-bit little-endian and 64-bit big-endian architectures (and that's why they cannot even move to 64-bit PowerPC).

Yes, 64 bit and 32 bit little endian without an instruction like ARM's SETEND will make 68k compatibility much less inefficient although the performance could be offset by much higher performance x86_64 CPUs than PPC CPUs. Moving to a big endian 64 bit ISA is trickier to say. Pointers will become 64 bits whichever data model is chosen (LLP64 or LP64) but this is not really a problem if all 68k pointers refer to the lower 4GB of memory (32 bit APTRs are sign extended to 64 bit pointers). It is possible to already get some benefit from a 64 bit architecture using a 32 bit kernel (all of the OS is in the lower 4GB of memory). Early versions of Mac OS X allowed 64 bit programs with a 32 bit kernel for example. AmigaOS 4 with minor changes could probably do the same on a 64 bit PPC CPU. Is it really worth it for a few hundred users though.

Quote:

OK, but previously I was talking only about existing applications, and without patching their binaries adding flags or their sources adding some new code. So, the worst (and likely common) scenario.

Well, the low memory protection could be for existing applications without changes.

Quote:

It's fine thinking about new o.s. and new apps, but here I think that it's better to turn everything on. However new code without message passing is not trivial at all to implement, both at the o.s. and apps level: it's a radical, totally different implementation.

Who is going to use your high security AmigaOS with no software? My idea provides compatibility, scalability and a transition path to higher security. It probably has some issues but AmigaOS security already has issues.

Message passing could be trapped and copied by the trap code which would provide security but would be the slowest way to pass messages. I'm not too concerned about message passing at this point. We have the fastest method of message passing and there are much larger security concerns in the AmigaOS to tackle first.

Quote:

Microsoft made a HUGE effort to optimize Windows in order to make it work/run on very low-end hardware platforms, like tablets and smartphones. I've a Compute Stick with just 32GB of eMMC, 2GB of RAM, and a BayTrail CPU, which runs "good enough" for regular/common jobs.

I have an Amiga which runs fine on 2MB of memory. What do you define as very low-end?

Quote:

This isn't about security but privacy, and we're talking about ANONYMOUS data which are transfered.

The data is only anonymous if you have good security everywhere and there are no backdoors allowing big brother in.

https://www.theverge.com/2013/6/6/4403868/nsa-fbi-mine-data-apple-google-facebook-microsoft-others-prism

Quote:

What do you mean with "darkness"?

I refer to the dark age of computing and the oppression bonds from large corporate entities squelching creativity and competition.

Quote:

kolla wrote: Are you sure? My OS4/MorphOS systems are rather clean for 68k cruft, there really isn't that much "must have" 68k software that doesn't have some OS4/MorphOS equivalent, and what "must have" 68k software that does exist mostly doesn't work well on OS4 anyways, due to lack of Amiga chipset.

You don't use ARexx? I wish it was possible to turn off 68k emulation and then we would see.

@cdimauro

Quote:

The problem with x64 is related to the chipset, and not to the processor/ISA. But if you compare the two ecosystems, ARM is in a much worse condition from this point-of-view: there are far more combinations of hardware.

I was thinking specifically of the RPi3, not the entire ARM ecosystem. It's cheap and widely available, and it doesn't get updated that often, so it would make a good reference system for developers.

There's not really anything like this in the x86_64 ecosystem; the Intel NUC probably comes the closest. It's more expensive and tends to get updated versions more often, which means more hardware to support, which makes it less attractive as a reference system.

_________________
"Unix is supposed to fix that." -- Jay Miner

Quote:

bison wrote: I was thinking specifically of the RPi3, not the entire ARM ecosystem. It's cheap and widely available, and it doesn't get updated that often, so it would make a good reference system for developers.

Even the Broadcom VideoCore IV GPU is standardized and well documented. It's lower performance than I would like for 3D but appropriate for the system energy efficiency. Why can't the Amiga have affordable standard hardware like this today? The end of Moore's law means electronic devices don't change as fast anymore. The standardized system model of the RPi (and C=) now may have more advantages than disadvantages, at least for the average computer user. OS programmers don't have to write drivers and support thousands of hardware add-ons. The software is usually better optimized for the hardware. The hardware can physically be located closer together for better performance. Sometimes technology changes too fast and the little guys end up constantly playing catch up. Maybe they should consider simpler standardized hardware.

@matthey

Quote:

Message passing could be trapped and copied by the trap code which would provide security but would be the slowest way to pass messages.

No, it could not. In order to copy the message, the trap code would need to know exact format of the message, and this is not always the case. How would message copying help if somewhere in the copied data there would be a pointer to another data in address space of the caller? How would you solve that?

Yes, there is a very simple solution. Turn off memory protection ;)

@matthey
Quote:

matthey wrote: Moving to a big endian 64 bit ISA is trickier to say. Pointers will become 64 bits whichever data model is chosen (LLP64 or LP64) but this is not really a problem if all 68k pointers refer to the lower 4GB of memory (32 bit APTRs are sign extended to 64 bit pointers).

68K pointers are by definition in the lower 4GB of memory.

However the problem with the Amiga o.s. (and ports/reimplementations) is that all o.s.' structures are public, and for applications you don't know if a memory allocation is used for a message or for an internal buffer. Even worse, if you use memory pools (like C stdlib implementations), the same buffer which was allocated with AllocMem can be used by malloc for practically every kind of memory allocation.

Now imagine how a 64-bit Amiga o.s. can work in this scenario, supporting both 32 and 64 bit applications. You can keep all o.s.' structures on the lower 4GB space (until it runs out of space), but the o.s. cannot make any assumption when a 64-bit applications allocates memory...
Quote:

It is possible to already get some benefit from a 64 bit architecture using a 32 bit kernel (all of the OS is in the lower 4GB of memory). Early versions of Mac OS X allowed 64 bit programs with a 32 bit kernel for example. AmigaOS 4 with minor changes could probably do the same on a 64 bit PPC CPU. Is it really worth it for a few hundred users though.

You're speaking about MacOS X Tiger I suppose, which was the first which introduced the support to 64-bit applications. I don't remember know if the kernel was 32 or 64 bit (I bet for the second), but this is irrelevant because the problems stay at the applications level.

Tiger used the lower 4GB of memory for most of the o.s. structures and 32-bit applications as well. But 64-bit applications weren't "first citizens" like the 32-bit one. In fact, ONLY 32-bit applications were able to use all o.s. APIs (not the 64-bit ones, of course), included the GUI ones. On the contrary, 64-bit applications only had a small subset (no GUI!) which allowed them to be used only to offload (heavy) computations.

Basically if you wanted to run a 64-bit application, you only had 2 chances:
- run from command line (no GUI at all). This was possible, if I remember correctly;
- run a 32-bit application (with GUI) which runs and controls a 64-bit one, passing data using ad-hoc APIs (copying back & forth data from 32 to 64-bit buffers, and viceversa).

So, as you can imagine, 64-bit applications weren't so useful, and Apple didn't recommended their use unless you had some number crunching code working on large set of data.

I don't think that you want to reproduce this crippled implementation, right?
Quote:

Quote: It's fine thinking about new o.s. and new apps, but here I think that it's better to turn everything on. However new code without message passing is not trivial at all to implement, both at the o.s. and apps level: it's a radical, totally different implementation. Who is going to use your high security AmigaOS with no software? My idea provides compatibility, scalability and a transition path to higher security. It probably has some issues but AmigaOS security already has issues.

It's not only a matter of security, but solidity and robustness comes first.

The problem with your idea is that, as I said before, it doesn't work in the real world, where you have several applications running, and they require/work only on some configurations which might (likely) be incompatible. This means that you're forced to run on the lowest common denominator, which means: all new features off. And this is clearly not acceptable.
Quote:

Message passing could be trapped and copied by the trap code which would provide security but would be the slowest way to pass messages.

michalsc already answered to this: it doesn't work.

As an additional detail, this trapping & copying doesn't work even in simple cases where messages have no pointers inside (e.g. a simply bytes array is passed, for example).

Since you don't know the message format (length, specifically), you don't know how much data to copy.
Quote:

I'm not too concerned about message passing at this point. We have the fastest method of message passing and there are much larger security concerns in the AmigaOS to tackle first.

It's not only a security problem, as I've already pointed out. The callee can do whatever it wants with the message that you passed, included altering / overriding data which the caller might still need.
Quote:

Quote: Microsoft made a HUGE effort to optimize Windows in order to make it work/run on very low-end hardware platforms, like tablets and smartphones. I've a Compute Stick with just 32GB of eMMC, 2GB of RAM, and a BayTrail CPU, which runs "good enough" for regular/common jobs. I have an Amiga which runs fine on 2MB of memory.

Then try run a modern Office-like suite on it, with all "bells & whistles". Try to run a modern browser. And so on.

You're comparing oranges and apples here. Not only the hardware evolved over the time, but the software did it as well, becoming more complex. A lightweight o.s. like the Amiga one can only partially help here, but can even create problems since he had no virtual memory, it's heavily affected by memory fragmentation, resources leaking, etc.
Quote:

What do you define as very low-end?

A Compute Stick is an example, as I've already reported.
Quote:

Quote: This isn't about security but privacy, and we're talking about ANONYMOUS data which are transfered. The data is only anonymous if you have good security everywhere and there are no backdoors allowing big brother in. https://www.theverge.com/2013/6/6/4403868/nsa-fbi-mine-data-apple-google-facebook-microsoft-others-prism

It's still a privacy problem.
Quote:

Quote: What do you mean with "darkness"? I refer to the dark age of computing and the oppression bonds from large corporate entities squelching creativity and competition.

According to the above link that you provided, the situation is even worse nowadays... :-/

@bison
Quote:

bison wrote: @cdimauro Quote: The problem with x64 is related to the chipset, and not to the processor/ISA. But if you compare the two ecosystems, ARM is in a much worse condition from this point-of-view: there are far more combinations of hardware. I was thinking specifically of the RPi3, not the entire ARM ecosystem. It's cheap and widely available, and it doesn't get updated that often, so it would make a good reference system for developers.

Yes, but the Raspberry Pi platform evolved over the time: the RPi3 is different from the first one. Sure, some things are still the same, and the new ones didn't changed so much the overall platform, but you have 3 different hardware flavors now. And more will come in future.
Quote:

There's not really anything like this in the x86_64 ecosystem; the Intel NUC probably comes the closest. It's more expensive and tends to get updated versions more often, which means more hardware to support, which makes it less attractive as a reference system.

Yes, the Intel platform is more stable a mostly backwards-compatible in the x86/x64 land. Even the integrated GPUs are very similar, and full documentation is available. That's why many open source fans prefer an Intel platform.

But it's more expensive. RPi is really, really low cost.

@cdimauro

Quote:

Yes, but the Raspberry Pi platform evolved over the time: the RPi3 is different from the first one. Sure, some things are still the same, and the new ones didn't changed so much the overall platform, but you have 3 different hardware flavors now. And more will come in future.

3 major versions (and a few minor ones) in 6 years does not seem excessive to me.

_________________
"Unix is supposed to fix that." -- Jay Miner

Quote:

michalsc wrote: No, it could not. In order to copy the message, the trap code would need to know exact format of the message, and this is not always the case. How would message copying help if somewhere in the copied data there would be a pointer to another data in address space of the caller? How would you solve that? Yes, there is a very simple solution. Turn off memory protection ;)

Memory protection does *not* need to be turned off if all shared memory is properly allocated with MEMF_SHARED. While not efficient of address space (not so important for >4GB of 64 bit address space) and not the most secure, memory protection should be good enough to catch and prevent most illegal address violations and probably even allow crash recovery (with resource tracking). Yes, existing programs will break but more secure AmigaOS settings would give a target and the improved memory protection could do its job of finding problems.

Quote:

cdimauro wrote: 68K pointers are by definition in the lower 4GB of memory.

An APTR could be redefined to be 64 bits but better compatibility may be possible with a new APTR64.

Quote:

However the problem with the Amiga o.s. (and ports/reimplementations) is that all o.s.' structures are public, and for applications you don't know if a memory allocation is used for a message or for an internal buffer. Even worse, if you use memory pools (like C stdlib implementations), the same buffer which was allocated with AllocMem can be used by malloc for practically every kind of memory allocation.

The flags for the memory allocation tell generally what it should be used for. Programmers need to follow the rules and we need a way to find problems when they don't or the problems grow. Linux/BSD style function calls should *not* be mixed with AmigaOS function calls (using malloc allocated memory for an AmigaOS message has never been kosher).

Quote:

You're speaking about MacOS X Tiger I suppose, which was the first which introduced the support to 64-bit applications. I don't remember know if the kernel was 32 or 64 bit (I bet for the second), but this is irrelevant because the problems stay at the applications level. Tiger used the lower 4GB of memory for most of the o.s. structures and 32-bit applications as well. But 64-bit applications weren't "first citizens" like the 32-bit one. In fact, ONLY 32-bit applications were able to use all o.s. APIs (not the 64-bit ones, of course), included the GUI ones. On the contrary, 64-bit applications only had a small subset (no GUI!) which allowed them to be used only to offload (heavy) computations. Basically if you wanted to run a 64-bit application, you only had 2 chances: - run from command line (no GUI at all). This was possible, if I remember correctly; - run a 32-bit application (with GUI) which runs and controls a 64-bit one, passing data using ad-hoc APIs (copying back & forth data from 32 to 64-bit buffers, and viceversa). So, as you can imagine, 64-bit applications weren't so useful, and Apple didn't recommended their use unless you had some number crunching code working on large set of data.

That would not have been very useful then. On average, 64 bit wide integer units are slower at processing data than 32 bit integer units but there are a few algorithms which are twice as fast. The big advantage of 64 bit is the increased address space and if you can't use that then you are better off with a 32 bit CPU. It should be possible to use the upper 64 bits of address space if the OS configures all the 64 bit space and the executable has a way to specify that the sections/hunks can be loaded into 64 bit address space. In any case, a 32 bit OS on a 64 bit CPU needs to become partially 64 bit aware (without a 32 bit compatibility mode) and passing 64 bit pointers of upper addresses to the OS could result in a crash. Interestingly, on a 64 bit CPU using register args, the pointers being passed and returned to/from functions of a 32 bit OS are 64 bits anyway (most address calculations would work too as they affect the whole register).

Quote:

It's not only a matter of security, but solidity and robustness comes first.

An AmigaOS with partial security provides more solidity and robustness than the AmigaOS with all security turned off. ThoR's MMU lib keeps some programs from crashing on my 68060 and I expect the minimal AmigaOS 4 memory protection does also. This includes code executing in the AmigaOS which was passed bad data.

Quote:

The problem with your idea is that, as I said before, it doesn't work in the real world, where you have several applications running, and they require/work only on some configurations which might (likely) be incompatible. This means that you're forced to run on the lowest common denominator, which means: all new features off. And this is clearly not acceptable.

Most of the security settings are transparent to the tasks/processes. Most of the changes should go in the AmigaOS. Why do you think ThoR's MMU library minimal memory protection works with most of my 68k programs? If I wrote a LoadSeg() patch to write protect code sections and/or SetFunction() many AmigaOS functions to add modular resource tracking, Amiga programs aren't going to care. Problematic programs would not work but they would be identified and some fixed which is half the purpose. Worst case, AmigaOS users are no better off than now but best case they get more security now and the possibility of better security in the future.

How is supporting different security settings any worse than supporting many different types of MMUs? Think of PPC where at least early MMUs didn't have the per page NX/XD bit. No execute protection requires designing all OS memory maps especially to get around this limitation and they are less efficient. Ouch! I don't think supporting variable security levels would be as obtrusive. Windows has some variable security levels already.

The performance and memory cost of security increases as the security level increases. AmigaOS 3.9 requires 6 MB of memory and AmigaOS 4.1 requires 128 MB of memory. I expect a significant amount of the difference is due to AmigaOS 4 minimal memory protection/mandatory MMU use. The MMU mapping tables require memory and significant padding to page boundaries is required. The difference between 6 MB and 128 MB on "very low-end" hardware is more than enough to make or break it and variable security levels allow some choice. Of course, if serious about going for a small footprint, PPC is the wrong architecture.

Quote:

As an additional detail, this trapping & copying doesn't work even in simple cases where messages have no pointers inside (e.g. a simply bytes array is passed, for example). Since you don't know the message format (length, specifically), you don't know how much data to copy.

The message structure contains a mn_Length field which gives the message length.

http://wiki.amigaos.net/wiki/Exec_Messages_and_Ports

Quote:

Quote: I have an Amiga which runs fine on 2MB of memory. Then try run a modern Office-like suite on it, with all "bells & whistles". Try to run a modern browser. And so on.

Try to run your office suite on your smart phone then too. I guess it is less than "very low-end" hardware too. Better toss it in the trash.

Quote:

You're comparing oranges and apples here. Not only the hardware evolved over the time, but the software did it as well, becoming more complex. A lightweight o.s. like the Amiga one can only partially help here, but can even create problems since he had no virtual memory, it's heavily affected by memory fragmentation, resources leaking, etc.

Memory fragmentation problems are much improved on most newer versions of the AmigaOS (AmigaOS 3.5+). Virtual memory is available on some and can be added to others. Surprisingly, full resource tracking seems to be elusive.



Quote:

It's still a privacy problem. ... According to the above link that you provided, the situation is even worse nowadays... :-/

Those who give up freedom for security shall have neither. Sadly, these backdoors really do exist. I have a friend who works at AT&T who knows. There are various court cases which shed limited light on it (some of the laws have been found Unconstitutional) and of course Edward Snowden who made the information more public.

@bison
Quote:

bison wrote: @cdimauro Quote: Yes, but the Raspberry Pi platform evolved over the time: the RPi3 is different from the first one. Sure, some things are still the same, and the new ones didn't changed so much the overall platform, but you have 3 different hardware flavors now. And more will come in future. 3 major versions (and a few minor ones) in 6 years does not seem excessive to me.

Indeed.


@matthey
Quote:

matthey wrote: Memory protection does *not* need to be turned off if all shared memory is properly allocated with MEMF_SHARED.

I don't know about 3.9, but until Amiga o.s. 3.5 there's no MEMF_SHARED flag defined (hence, usable) for memory allocation: include/exec/memory.h

So, there's no 68K binary which is using it, and no source code which can be updated with many applications.

I think that it was also quite normal to use MEMF_PUBLIC for memory allocations, which makes it quite hard to introduce proper protection and/or supporto to the virtual memory (public memory is not swappable, AFAIK).

What's your plan to improve this situation?
Quote:

While not efficient of address space (not so important for >4GB of 64 bit address space) and not the most secure, memory protection should be good enough to catch and prevent most illegal address violations and probably even allow crash recovery (with resource tracking). Yes, existing programs will break but more secure AmigaOS settings would give a target and the improved memory protection could do its job of finding problems.

True, but see above: the problem is represented by the existing applications, which are mostly without source, so cannot be changed to meet the new standards.

Just to clarify, do you think that losing compatibility with the existing Amiga o.s. applications is fine when talking about a new, modernized, Amiga o.s.?
Quote:

Quote: cdimauro wrote: 68K pointers are by definition in the lower 4GB of memory. An APTR could be redefined to be 64 bits but better compatibility may be possible with a new APTR64.

sizeof(APTR) should be enough to solve any problem without requiring a new pointer type.
Quote:

Quote: However the problem with the Amiga o.s. (and ports/reimplementations) is that all o.s.' structures are public, and for applications you don't know if a memory allocation is used for a message or for an internal buffer. Even worse, if you use memory pools (like C stdlib implementations), the same buffer which was allocated with AllocMem can be used by malloc for practically every kind of memory allocation. The flags for the memory allocation tell generally what it should be used for. Programmers need to follow the rules and we need a way to find problems when they don't or the problems grow. Linux/BSD style function calls should *not* be mixed with AmigaOS function calls (using malloc allocated memory for an AmigaOS message has never been kosher).

Not if c stdlib memory pool uses public memory: it should be "kosher enough", since messages will be public, as expected/required.
And you cannot expect that C/C++/Pascal (Delphi)/etc. applications, which normally use their memory pools, will all be changed in order to use a specific o.s. API for doing a common task, like allocating memory; think also to C++ with its new operator and you'll get crazy thinking about such changes.

However, and as I've already pointed out, the problem stays in the existing applications, which likely use MEMF_PUBLIC for all memory allocations, despite if the memory will be really used as public or not (private; only used by the application).
Quote:

That would not have been very useful then. On average, 64 bit wide integer units are slower at processing data than 32 bit integer units but there are a few algorithms which are twice as fast.

Not only few: there are many cases, common cases, where using 64-bit integers help a lot and improve performance. Memory copy/move/fill/scan/comparisons, string operations (not so well with the C str type, but 64-bit can be used as well with some tricks), decimals (fixed precision), color space conversions (32x32 bit multiplication -> 64-bit result), etc. I stop here but I can find others.
Quote:

The big advantage of 64 bit is the increased address space and if you can't use that then you are better off with a 32 bit CPU.

Not really: big pointers allow also pointer-tagging, which is very useful for managed resources (which are quite common today).
Quote:

It should be possible to use the upper 64 bits of address space if the OS configures all the 64 bit space and the executable has a way to specify that the sections/hunks can be loaded into 64 bit address space.

This is common. AFAIR the default entry point for 64-bit applications is >4GB on Windows.
Quote:

Quote: The problem with your idea is that, as I said before, it doesn't work in the real world, where you have several applications running, and they require/work only on some configurations which might (likely) be incompatible. This means that you're forced to run on the lowest common denominator, which means: all new features off. And this is clearly not acceptable. Most of the security settings are transparent to the tasks/processes. Most of the changes should go in the AmigaOS. Why do you think ThoR's MMU library minimal memory protection works with most of my 68k programs? If I wrote a LoadSeg() patch to write protect code sections and/or SetFunction() many AmigaOS functions to add modular resource tracking, Amiga programs aren't going to care. Problematic programs would not work but they would be identified and some fixed which is half the purpose. Worst case, AmigaOS users are no better off than now but best case they get more security now and the possibility of better security in the future.

The big problem is: who will update such applications, and how? See above.
Quote:

How is supporting different security settings any worse than supporting many different types of MMUs? Think of PPC where at least early MMUs didn't have the per page NX/XD bit. No execute protection requires designing all OS memory maps especially to get around this limitation and they are less efficient. Ouch! I don't think supporting variable security levels would be as obtrusive

No, but the problem is represented by the current situation.
Quote:

Windows has some variable security levels already.

But that's part of the o.s.. You have ACLs which allow to carefully and granularly define access to the resources, which Amiga o.s. is completely lacking.
Quote:

The performance and memory cost of security increases as the security level increases. AmigaOS 3.9 requires 6 MB of memory and AmigaOS 4.1 requires 128 MB of memory. I expect a significant amount of the difference is due to AmigaOS 4 minimal memory protection/mandatory MMU use. The MMU mapping tables require memory and significant padding to page boundaries is required.

Hum. AFAIR memory protection is only applied to not used/free memory pages, and you don't need to create a full page table hierarchy for it (keep only the entries & page directories which contain valid, allocated/used, pages).

I don't know if OS4 allocates memory on a per-page basis for different applications (e.g.: a page is used only for memory allocations of a specific task), but even in this case it shouldn't waste so much memory.

Memory is really wasted if every allocation gives back integral pages. But I don't know how OS4 works.
Quote:

Quote: As an additional detail, this trapping & copying doesn't work even in simple cases where messages have no pointers inside (e.g. a simply bytes array is passed, for example). Since you don't know the message format (length, specifically), you don't know how much data to copy. The message structure contains a mn_Length field which gives the message length. http://wiki.amigaos.net/wiki/Exec_Messages_and_Ports

Yes, you're right. I forgot this important detail. Nevermind.
Quote:

Quote: Then try run a modern Office-like suite on it, with all "bells & whistles". Try to run a modern browser. And so on. Try to run your office suite on your smart phone then too. I guess it is less than "very low-end" hardware too. Better toss it in the trash.

Well, that's what I can do with my smartphone (Motorola X4 with 64GB storage and 4GB RAM) as well, since Office is available for it. I can also use Office online from inside the Dropbox web site.

Now what about your Amiga with 2MB?
Quote:

Quote: You're comparing oranges and apples here. Not only the hardware evolved over the time, but the software did it as well, becoming more complex. A lightweight o.s. like the Amiga one can only partially help here, but can even create problems since he had no virtual memory, it's heavily affected by memory fragmentation, resources leaking, etc. Memory fragmentation problems are much improved on most newer versions of the AmigaOS (AmigaOS 3.5+).

They are still there, and without resource tracking every leak will worse the situation.

Imagine what happens with modern browser (which are affected by memory leaks). At least on a modern o.s. you can close and reopen the browser, solving such problems. With the Amiga o.s., it can only worse the situation...
Quote:

Virtual memory is available on some and can be added to others.

How, since usually you have public memory allocations, which aren't swappable?
Quote:

Surprisingly, full resource tracking seems to be elusive.

Eh. Quite hard to accomplish, since you don't know how the applications are using their resources...
Quote:

Quote: It's still a privacy problem. ... According to the above link that you provided, the situation is even worse nowadays... :-/ Those who give up freedom for security shall have neither.

Benjamin Franklin, if I recall correctly.

But this wasn't the case.
Quote:

Sadly, these backdoors really do exist. I have a friend who works at AT&T who knows. There are various court cases which shed limited light on it (some of the laws have been found Unconstitutional) and of course Edward Snowden who made the information more public.

What do you mean here? Backdoors inside o.ses, or public access to collected data?

@cdimauro

Quote:

do you think that losing compatibility with the existing Amiga o.s. applications is fine when talking about a new, modernized, Amiga o.s.?

Yes, yes and utter yes. The alternative is is that it is neither new nor modern.

Last edited by kolla on 15-Aug-2018 at 12:50 PM.

_________________
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC

@cdimauro

Quote:

True, but see above: the problem is represented by the existing applications, which are mostly without source, so cannot be changed to meet the new standards.

I think existing applications should be run via emulation. We already have this, and it works well, for the most part.

_________________
"Unix is supposed to fix that." -- Jay Miner

@bison

Quote:

We already have this, and it works well,

Yes and No, petunia works well, E-UAE Jit or E-UAE interpreted takes and takes CPU cycles, nothing left to host OS.

We really need SMP, I don't care about 64bit address space, and yes we need better memory protection, too many programs get away with to many things.

I think it be nice to have some kind of way to isolate bad programs, run in some kind of virtual host, some thing that don't share resources with host, so when you kill it, all the resources are freed, even if program sucks ass.

_________________
http://lifeofliveforit.blogspot.no/
Facebook::LiveForIt Software for AmigaOS

@kolla
Quote:

kolla wrote: @cdimauro Quote: do you think that losing compatibility with the existing Amiga o.s. applications is fine when talking about a new, modernized, Amiga o.s.? Yes, yes and utter yes. The alternative is is that it is neither new nor modern.

Then we need new applications as well. You can recycle most of the code, but a new Amiga o.s. with all modern features is and works quite different from the current one (and successors/reimplementations).


@bison
Quote:

bison wrote: @cdimauro Quote: True, but see above: the problem is represented by the existing applications, which are mostly without source, so cannot be changed to meet the new standards. I think existing applications should be run via emulation. We already have this, and it works well, for the most part.

That's what I was telling from long time. Let the legacy stuff be isolated in a sandbox.

It also means writing an o.s. which can run on any architecture.


@NutsAboutAmiga
Quote:

NutsAboutAmiga wrote: @bison Quote: We already have this, and it works well, Yes and No, petunia works well, E-UAE Jit or E-UAE interpreted takes and takes CPU cycles, nothing left to host OS.

It also depends on the hardware where such emulators are running. If it's not powerful enough, it can hog all CPU performance (from a single core/thread system).
Quote:

We really need SMP, I don't care about 64bit address space, and yes we need better memory protection, too many programs get away with to many things.

64-bit is good for addressing more memory (which isn't unlikely nowadays) and for pointer tagging.
Quote:

I think it be nice to have some kind of way to isolate bad programs, run in some kind of virtual host, some thing that don't share resources with host, so when you kill it, all the resources are freed, even if program sucks ass.

A modern o.s. with resource tracking.

@cdimauro

Quote:

Then we need new applications as well. You can recycle most of the code, but a new Amiga o.s. with all modern features is and works quite different from the current one (and successors/reimplementations).

Yes - and that is perfectly fine.

And yes, new applications too. The entire point is to have a platform that keeps the user interface/experience concepts and elements that Amiga _users_ are familiar with (screens that can have different resolutions, a souped up workbench, properly implemented amiga CLI, consistency between CLI and GUI, etc), yet a modern core and an operating system with all the features needed for the modern world, so that software developers - not Amiga developers, but _software developers_ - with as little effort as possible, can build their software for it.

Last edited by kolla on 15-Aug-2018 at 09:21 PM.

_________________
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC

@cdimauro

Quote:

Then we need new applications as well. You can recycle most of the code, but a new Amiga o.s. with all modern features is and works quite different from the current one (and successors/reimplementations).

I think so. It should run existing Amiga applications via emulation, Linux applications via a nested X-server, and new applications that are unencumbered by the requirements of existing Amiga and Linux applications. So the first day it is released it should have a large number of existing Amiga and Linux applications, and new applications that are being written using a new API that doesn't carry forward the mistakes of the past. (Or in the case of Linux, on mistakes that are still being made.)

_________________
"Unix is supposed to fix that." -- Jay Miner

@kolla

Quote:

...consistency between CLI and GUI...

I like the idea of a GUI application that is nothing but a front-end for a terminal app. The terminal app has to be written with hooks in it to make this seamless, but the same code (other than UI-specific code) is executed in either case.

Not all applications lend themselves to this approach, but I think it's natural for configuration utilities. For example, running 'foo' at the command line should take command line arguments, and running 'foo -g' should start the GUI version of the same application.

_________________
"Unix is supposed to fix that." -- Jay Miner

@bison

Quote:

For example, running 'foo' at the command line should take command line arguments, and running 'foo -g' should start the GUI version of the same application.

Or one could extend the concept of "Commodities" so that most programs are like commodities, in that they can show or hide the GUI via a standardised interface. Using icons for metadata is also perfectly fine to bring on, including tooltypes, and tooltypes can typically be passed on as command line arguments.

I would even like to see Amiga Shell implement some aspects of DCL, the CLI that comes with VMS and others. For example, on amiga one can type "foo ?", and if you are lucky, foo is kind enough to give you interactive help for its arguments. On DCL (and various others), the shell have access to these from the binary, without running the binary - meaning the shell can tab expand arguments and options, and you can build up a command line, and at any time break it with ctrl+c. For example.

A thorough clean-up and modernisation of the entire CLI would be great. And I am not talking about just replacing it with a posix shell, but keep it Amiga and just do it "right" - modular, extendable, feature rich, so that it is the obvious choice for all scripting on the OS - no more the duality with shell-scripts and rexx scripts - just scripts. Heck, did SHEEP go anywhere else? No.

_________________
B5D6A1D019D5D45BCC56F4782AC220D8B3E2A6CC

Quote:

cdimauro wrote: I don't know about 3.9, but until Amiga o.s. 3.5 there's no MEMF_SHARED flag defined (hence, usable) for memory allocation: include/exec/memory.h So, there's no 68K binary which is using it, and no source code which can be updated with many applications. I think that it was also quite normal to use MEMF_PUBLIC for memory allocations, which makes it quite hard to introduce proper protection and/or support to the virtual memory (public memory is not swappable, AFAIK). What's your plan to improve this situation?

MEMF_SHARED is new. MEMF_PUBLIC has been around (and abused) for a very long time.

http://wiki.amigaos.net/wiki/Obsolete_Exec_Memory_Allocation

Both of these flags should be respected which means that the shared/public memory can't be freed after a crash. MEMF_SHARED is more friendly and at least allows shared memory to be swapped out after a crash saving physical address space. Memory protection and resource tracking is better than turning it off in either case. Crash recovery with partial resource recovery is better than no crash recovery with no resource recovery. A 64 bit address space would make the loss of physical and virtual address space not a concern. Some kind of new API would likely need to be created to allow for all shared memory to be freed after a crash (existing programs would not benefit). I'm thinking it would be possible to allow shared memory messages instead of copying with a new API but it would be tricky.

Quote:

Just to clarify, do you think that losing compatibility with the existing Amiga o.s. applications is fine when talking about a new, modernized, Amiga o.s.?

It depends on the goal of the AmigaOS. If moving to an alien architecture and trying to compete with feature rich and mature operating systems then yes but realize that taking market share is an uphill battle even when their features are matched because of lack of software. I would prefer to aim lower end, smaller and more efficient where feature rich OSs can't go but embedded needs to go. Maybe it is possible to add features and retain software compatibility here. Don't underestimate the importance of software whatever the goal!

Quote:

sizeof(APTR) should be enough to solve any problem without requiring a new pointer type.

C programs should use sizeof(APTR) but that doesn't help compatibility in AmigaOS structures when redefining APTR's size. Perhaps you mean to break compatibility though.

Quote:

Quote: The flags for the memory allocation tell generally what it should be used for. Programmers need to follow the rules and we need a way to find problems when they don't or the problems grow. Linux/BSD style function calls should *not* be mixed with AmigaOS function calls (using malloc allocated memory for an AmigaOS message has never been kosher). Not if c stdlib memory pool uses public memory: it should be "kosher enough", since messages will be public, as expected/required. And you cannot expect that C/C++/Pascal (Delphi)/etc. applications, which normally use their memory pools, will all be changed in order to use a specific o.s. API for doing a common task, like allocating memory; think also to C++ with its new operator and you'll get crazy thinking about such changes.

I wouldn't use malloc for allocating AmigaOS structures and I don't recommend it. Programmers should not count on malloc allocated memory to be MEMF_PUBLIC or any other type. You are asking for trouble if you do as your code will not be portable from one C compiler to the next and may break if AmigaOS memory protection is improved.

Quote:

However, and as I've already pointed out, the problem stays in the existing applications, which likely use MEMF_PUBLIC for all memory allocations, despite if the memory will be really used as public or not (private; only used by the application).

MEMF_PUBLIC is no doubt long abused as there has not been enough to break the programs that abuse it.

Quote:

Quote: That would not have been very useful then. On average, 64 bit wide integer units are slower at processing data than 32 bit integer units but there are a few algorithms which are twice as fast. Not only few: there are many cases, common cases, where using 64-bit integers help a lot and improve performance. Memory copy/move/fill/scan/comparisons, string operations (not so well with the C str type, but 64-bit can be used as well with some tricks), decimals (fixed precision), color space conversions (32x32 bit multiplication -> 64-bit result), etc. I stop here but I can find others.

I can't remember where I saw the benchmark of 32 bit vs 64 bit compiled software results for a suit of benchmarks (not x86 vs x86_64 where 8 needed registers were added). On average the 64 bit applications were slower. There are algorithms which are much faster with 64 bit but most applications are a little slower with 64 bit (slower mul, div and shift, worse code density, more DCache stress, bigger misalignment penalties). AArch64 has 32 bit operations for this reason (helps low end CPUs more). It's kind of like 16 vs 32 registers where 32 can give a big boost to a few algorithms but I've seen results on benchmark suits where on average the performance gain is less than 1%. Register args instead of stack args with aggressive function inlining (GCC -O2 or higher), less than 1% performance gain too.

Quote:

Quote: The big advantage of 64 bit is the increased address space and if you can't use that then you are better off with a 32 bit CPU. Not really: big pointers allow also pointer-tagging, which is very useful for managed resources (which are quite common today).

Ugh. We cleaned up 32 bit dirty programs when we ran out of 32 bit address space so now we allow 64 bit dirty programs?

I thought it would be interesting for the high bit of 64 bit addresses to be 1 to specify a physical or virtual address avoiding a TLB lookup. The big advantage would be to avoid TLB cache misses and allow the TLB cache sizes to be reduced in an OS where many of the addresses were physical addresses.

Quote:

Quote: It should be possible to use the upper 64 bits of address space if the OS configures all the 64 bit space and the executable has a way to specify that the sections/hunks can be loaded into 64 bit address space. This is common. AFAIR the default entry point for 64-bit applications is >4GB on Windows.

The default code model for x86_64 is small code which requires programs to reside in the lower 2GB of (virtual) memory because RIP (PC) relative addressing is only 32 bits.

https://eli.thegreenplace.net/2012/01/03/understanding-the-x64-code-models

Quote:

The big problem is: who will update such applications, and how? See above.

Probably nobody with a niche Amiga market.

Quote:

But that's part of the o.s.. You have ACLs which allow to carefully and granularly define access to the resources, which Amiga o.s. is completely lacking.

Everything I proposed should be part of the AmigaOS. An OS which is modular is still an OS. It is the Amiga way.

Quote:

Quote: The performance and memory cost of security increases as the security level increases. AmigaOS 3.9 requires 6 MB of memory and AmigaOS 4.1 requires 128 MB of memory. I expect a significant amount of the difference is due to AmigaOS 4 minimal memory protection/mandatory MMU use. The MMU mapping tables require memory and significant padding to page boundaries is required. Hum. AFAIR memory protection is only applied to not used/free memory pages, and you don't need to create a full page table hierarchy for it (keep only the entries & page directories which contain valid, allocated/used, pages).

I agree that there shouldn't be too much MMU overhead for as little of memory protection as AmigaOS 4 has. My 68060 Amiga with MMU library active doesn't need much memory. The 40%-50% worse code density of PPC than 68k can't explain it either. Neither the much larger stack needs of the PPC. I expect disc caches are bigger but my 68060 Amiga has plenty of memory to spare with generous caches.

Quote:

Quote: Try to run your office suite on your smart phone then too. I guess it is less than "very low-end" hardware too. Better toss it in the trash. Well, that's what I can do with my smartphone (Motorola X4 with 64GB storage and 4GB RAM) as well, since Office is available for it. I can also use Office online from inside the Dropbox web site. Now what about your Amiga with 2MB?

A 68k Amiga could have done it with less but probably not 2MB.

Quote:

Quote: Memory fragmentation problems are much improved on most newer versions of the AmigaOS (AmigaOS 3.5+). They are still there, and without resource tracking every leak will worse the situation. Imagine what happens with modern browser (which are affected by memory leaks). At least on a modern o.s. you can close and reopen the browser, solving such problems. With the Amiga o.s., it can only worse the situation...

The AmigaOS has memory pools. Freeing a memory pool gives back all the memory allocated within at once. The total memory I lose from memory leaks over time is usually not a problem. The performance degradation is a bigger concern because the CPU is not fast. The newer AmigaOS functions are much better at reducing fragmentation but are also slower on an unfragmented system. TLSFMem gives significantly faster memory functions and low fragmentation despite the relatively slow BFFFO instruction on the 68060 (the Apollo Core BFFFO is a fraction of the cycles).

Quote:

Quote: Virtual memory is available on some and can be added to others. How, since usually you have public memory allocations, which aren't swappable?

Not all allocations are MEMF_PUBLIC. Program sections/hunks should be swappable at least. It sounds like virtual memory swapping is working in AmigaOS 4 but I can't say how good it is. The 68k has virtual memory add-ons like GigaMem, VMEM and XMEM. GigaMem worked reliably but did use a whitelist/blacklist setup for best compatibility. Perhaps some MEMF_PUBLIC flags were corrected from bug reports by virtual memory users.

Quote:

Quote: Surprisingly, full resource tracking seems to be elusive. Eh. Quite hard to accomplish, since you don't know how the applications are using their resources...

It is not necessary to know how resources are used in order to track them.

Quote:

Quote: Those who give up freedom for security shall have neither. Benjamin Franklin, if I recall correctly.

I believe the generalized saying predates our U.S. Forefathers and likely comes from Europe. Several of our Forefathers may have used it although Benjamin Franklin's famous quote was taken out of context. The saying does logically make sense.

Quote:

What do you mean here? Backdoors inside o.ses, or public access to collected data?

Fiber optic lines run to the NSA. They collect data in bulk but need secret court approval for searches and what filters can be used (broad search warrants are supposed to be Unconstitutional). Foreigners get less "security" than U.S. citizens.

@kolla

Quote:

Or one could extend the concept of "Commodities" so that most programs are like commodities, in that they can show or hide the GUI via a standardised interface.

Commodities are a very useful feature. One thing that I'm not certain of is if they can be implemented in a secure way. It seems like a very opportune way for someone with malicious intent to pass information along to a program that shouldn't be getting it, or to get information from a program that shouldn't be giving it.

Quote:

And I am not talking about just replacing it with a posix shell, but keep it Amiga and just do it "right" - modular, extendable, feature rich, so that it is the obvious choice for all scripting on the OS - no more the duality with shell-scripts and rexx scripts - just scripts.

Yes, let not have yet another POSIX shell. I don't think that a really good shell has ever been written, so there's probably opportunity there.

_________________
"Unix is supposed to fix that." -- Jay Miner