Quote:

@gregthecanuck 64bit memory pointers and the ability to use 64bit files are two totally seperate issues, with different problems and issues. Just becaus DOS has support for 64bit file I/O, doesn't suddenly make OS have a 64bit memory model.

Well duh! However it did prompt the next logical question. Don't forget that they can do tricky things like support more that 4GB of memory in a 32bit system. Lots of UNIX systems do that, for example.

I'm a bit grumpy after doing non-stop code reviews for hours on end so forgive my abrubtness... FYI I am working with "AccuRev" which is quite a cool product but unfortunately they are using a Java GUI client that sucks so badly, particularly its bad handling of memory. A new version should be out soon but in the meantime...

Quote:

However, I have one point of curiosity that remained unsatisfied: what about the logical subdivision (virtual, chip, fast, etc.) of the memory? I remember that that was bound to chance... has it changed? Is there a cleaner system in place now?

The new system only discerns between MEMF_SHARED and MEMF_PRIVATE. The old flags are still supported because of backward compatibility.

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

Quote:

Any hints as to which release this would be available in? 4.0? 4.n?

4.0

Quote:

Does this 64bit I/O imply a 64bit memory model as well?

No.

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

@EntilZha

Quote:

The new system only discerns between MEMF_SHARED and MEMF_PRIVATE. The old flags are still supported because of backward compatibility.

Veeery nice
This is exactly what I wanted to hear

saimo

_________________
RETREAM - retro dreams for Amiga, Commodore 64 and PC

Quote:

Does this 64bit I/O imply a 64bit memory model as well?

Only if you want to load a +4 GB file into memory, I think

_________________
Philippe 'Elwood' Ferrucci
Sam460 1.10 Ghz
AmigaOS 4 betatester
Amiga Translator Organisation

Quote:

The new system only discerns between MEMF_SHARED and MEMF_PRIVATE. The old flags are still supported because of backward compatibility.

Does this mean MEMF_PUBLIC is obsolete now? I.e. MEMF_SHARED/ANY is used for message ports etc?

_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

Quote:

Does this mean MEMF_PUBLIC is obsolete now? I.e. MEMF_SHARED/ANY is used for message ports etc?

MEMF_SHARED, yes. MEMF_ANY doesn't qualify as it can also be MEMF_PRIVATE, which is a rather bad idea for message ports.

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

@Entilzha

We were discussing Message ports and how they fit in an memory protective environment on the dev channel yesterday.

Shouldn't it be possible to use private memory for message ports if they are just going to be accessed thru OS functions anyway? The most extreme thing is that addresses need to be identical throughout appications if requested, but shouldn't the normal memory abstration (logical memory space) be able to cope with that? You could even don't replicate the msgports for each program unless they are written to (copy-on.write).

Feel free to enlighten me if you have time ;)

_________________
Nicolas Mendoza

Nicely written article. Good to see the developers have concerned themselves with such fundametal improvements of operation in addition to the already vast task of porting the original functionaility.

Can someone explain to me though, how this system materially differs from Memory Pools? (AllocPooled() etc.)

Quote:

How does the memory-system of OS4 compare to that of Windows/Linux/Mac OSX?

For Windows XP and Server refer to
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/low_fragmentation_heap.asp

_________________
Amiga 1200 (rev 1D1, KS 3.2, PiStorm32/RPi CM4/Emu68)
Amiga 500 (rev 6A, ECS, KS 3.2, PiStorm/RPi 4B/Emu68)
Ryzen 9 7950X, DDR5-6000 64 GB RAM, GeForce RTX 4080 16 GB

It seems Window's bucklets get rid of External Fragmentation, wich is good, but you'll have a lot of small internal fragmentations, that are not as bad as External ones, but if you do several little allocations instead of a big one you'll allocate a lot of unused memory.
They pratically used an evolution of the cluster allocation method that is used in common file systems.

IMHO, the one used by OS4, even if appears to be much more complicated, is uncomparably efficient .

Last edited by ErikBauer on 14-Dec-2005 at 01:24 PM.

_________________
God created Paula so that Allister Brimble and Dave Whittaker could do music

Check my Amiga gameplays (ITA)!

Quote:

Shouldn't it be possible to use private memory for message ports if they are just going to be accessed thru OS functions anyway?

In theory, yes, but that would require the OS to "change" into the virtual address space of another task, or it would require all memory to be accessible in supervisor state (or "kernel mode", like in Linux).

If others would only use PutMsg to put messages into the port, then it would work, but looking at the recent discussion on UtilityBase about abusing undocumented features in the system, you can imagine that some "clever" guys will queue up the message directly...

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

@Erik
However, from the article it's not clear how does this system deal with internal fragmentation.

Slab allocators are not general purpose allocators, you generally have a pool of free objects which you take your objects from, all of the same size. In the article there's no actual explanation on how do the free list get built: are they all "slab" lists? If so, are the allocatable objects sizes decided once and for all, or dynamically chosen depending on the usage pattern? Answering this question would also give an hint on what inpact internal fragmentation has on this system.

Last edited by falemagn on 14-Dec-2005 at 03:24 PM.
Last edited by falemagn on 14-Dec-2005 at 03:23 PM.

_________________
“It is well enough that people of the nation do not understand our banking and monetary
system, for if they did, I believe there would be a revolution before tomorrow morning.”
~~ Henry Ford

@falemagn

The sizes are selected at system startup, and currently, they are static. But since this functionality is hidden, dynamic cache sizes can be implemented.

The real strength of the slab allocator comes of course from using it to manage distinct objects. The system will gradually use more distinct caches. However, even with the current implementation (there are roughly 10 specialized caches), it's already more space efficient than the old one (not to mention the fact that it'S much faster).

The caches also keep statistics about how much waste they produce, the data is just not used yet for any dynamic adaption... I'll see that I can come up with some data later.

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

So, if I'm following it right, this puts OS4's Slab allocation method more or less in peer with Window's 'Basket' allocation method speaking of pure memory waste. But it is transparently adaptable to a dynamic memory handling that would allow less if not zero memory wasting.

Moreover it seems more flexible and has some other nice features that the other one lacks.

Am I right?

Last edited by ErikBauer on 14-Dec-2005 at 03:52 PM.

_________________
God created Paula so that Allister Brimble and Dave Whittaker could do music

Check my Amiga gameplays (ITA)!

Quote:

So, if I'm following it right, this puts OS4's Slab allocation method more or less in peer with Window's 'Basket' allocation method speaking of pure memory waste. But it is transparently adaptable to a dynamic memory handling that would allow less if not zero memory wasting.

If you only look at the AllocVec/FreeVec idea, and only at the memory waste, then yes, although OS4 doesn't use strict power of two buckets, but buckets that are ascending by roughly 20 % in size.

Quote:

Moreover it seems more flexible and has some other nice features that the other one lacks.

Yes, especially since the Windows allocator is not "typed" in any way. Slab allocators are typed, with the added benefit of making a segregated memory list approach very simple.

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

Thanx alot, I'm always trying to keep up my mind with new technologies and implementations, if you do not go like this in Computer Area you're lost in less than one year.

Btw these are very infos! You guys are doing a very smart job with OS4 (altough a lot of things seem to be 'borrowed' from Linux-Unix, not that it is a bad thing though... )!

_________________
God created Paula so that Allister Brimble and Dave Whittaker could do music

Check my Amiga gameplays (ITA)!

@ErikBauer:

Wasn't the Amiga philosophy something like:"if it works well for others,we'll benefit&improve it".
Likewise it was the case with Rexx/ARexx,for instance?

_________________
Cuida tus piedras gordas.

A1200/030 32Mb
A4000D
A1-X1000.

Quote:

altough a lot of things seem to be 'borrowed' from Linux-Unix, not that it is a bad thing though..

Actually, it's borrowed from Solaris, if at all..

Note that no code was used from other sources than my own. I based the implementation wholly on the published papers...

_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

@ErikBauer :

Quote:

(altough a lot of things seem to be 'borrowed' from Linux-Unix, not that it is a bad thing though... )!

This 'borrowing' takes place all the time, it's called 'taking advantage of current research'. If you have a the choice of doing something yourself from the ground up or using the freely available research from a group of PhDs with years of experience, which one would you do? (this doesn't really apply if you're coding something just for fun though...)

Not razzing you, just pointing out that Hyperion is not doing anything lazy or sneaky. Researchers want commercial implementations of their work, it validates it and increases their prestige in the scientific community.