Click Here
home features news forums classifieds faqs links search
6071 members 
Amiga Q&A /  Free for All /  Emulation /  Gaming / (Latest Posts)
Login

Nickname

Password

Lost Password?

Don't have an account yet?
Register now!

Support Amigaworld.net
Your support is needed and is appreciated as Amigaworld.net is primarily dependent upon the support of its users.
Donate

Menu
Main sections
» Home
» Features
» News
» Forums
» Classifieds
» Links
» Downloads
Extras
» OS4 Zone
» IRC Network
» AmigaWorld Radio
» Newsfeed
» Top Members
» Amiga Dealers
Information
» About Us
» FAQs
» Advertise
» Polls
» Terms of Service
» Search

IRC Channel
Server: irc.amigaworld.net
Ports: 1024,5555, 6665-6669
SSL port: 6697
Channel: #Amigaworld
Channel Policy and Guidelines

Who's Online
16 crawler(s) on-line.
 150 guest(s) on-line.
 1 member(s) on-line.


 clint

You are an anonymous user.
Register Now!
 clint:  2 mins ago
 vox:  8 mins ago
 Gunnar:  12 mins ago
 pixie:  20 mins ago
 NutsAboutAmiga:  24 mins ago
 zipper:  1 hr ago
 Templario:  1 hr 5 mins ago
 RobertB:  1 hr 25 mins ago
 GPTNederlands:  1 hr 40 mins ago
 janelancy:  1 hr 41 mins ago

Internet News   Internet News : Memory Management in AmigaOS4.0 Explained
   posted by Rogue on 13-Dec-2005 21:05:03 (17168 reads)
A new article is available on os4.hyperion-entertainment.biz outlining the new and improved memory system on AmigaOS4.0. It describes the method of "slab allocators" and "object caching" that greatly reduce external memory fragmentation while keeping internal fragmentation to a guaranteed minimum and typically provides a constant-time memory allocation.

Read all about it on os4.hyperion-entertainment.biz.
    

STORYID: 2754
Related Links
· More about Internet News
· News by Rogue


Most read story about Internet News
IBM confirms POWER5 server release details

Last news about Internet News
Tom's Hardware run a story on AMIGA
Printer Friendly Page  Send this Story to a Friend

Goto page ( 1 | 2 | 3 | 4 | 5 | 6 )

PosterThread
gregthecanuck 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 10:53:14
#41 ]
Cult Member
Joined: 30-Dec-2003
Posts: 846
From: Vancouver, Canada

Quote:
@gregthecanuck

64bit memory pointers and the ability to use 64bit files are two totally seperate issues, with different problems and issues.

Just becaus DOS has support for 64bit file I/O, doesn't suddenly make OS have a 64bit memory model.


Well duh! However it did prompt the next logical question. Don't forget that they can do tricky things like support more that 4GB of memory in a 32bit system. Lots of UNIX systems do that, for example.

I'm a bit grumpy after doing non-stop code reviews for hours on end so forgive my abrubtness... FYI I am working with "AccuRev" which is quite a cool product but unfortunately they are using a Java GUI client that sucks so badly, particularly its bad handling of memory. A new version should be out soon but in the meantime...

 Status: Offline
Profile     Report this post  
EntilZha 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 11:03:21
#42 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
However, I have one point of curiosity that remained unsatisfied: what about the logical subdivision (virtual, chip, fast, etc.) of the memory? I remember that that was bound to chance... has it changed? Is there a cleaner system in place now?


The new system only discerns between MEMF_SHARED and MEMF_PRIVATE. The old flags are still supported because of backward compatibility.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
EntilZha 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 11:07:03
#43 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
Any hints as to which release this would be available in? 4.0? 4.n?


4.0

Quote:
Does this 64bit I/O imply a 64bit memory model as well?


No.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
saimo 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 11:16:59
#44 ]
Elite Member
Joined: 11-Mar-2003
Posts: 2450
From: Unknown

@EntilZha

Quote:
The new system only discerns between MEMF_SHARED and MEMF_PRIVATE. The old flags are still supported because of backward compatibility.

Veeery nice
This is exactly what I wanted to hear

saimo


_________________
RETREAM - retro dreams for Amiga, Commodore 64 and PC

 Status: Offline
Profile     Report this post  
elwood 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 11:25:56
#45 ]
Elite Member
Joined: 17-Sep-2003
Posts: 3428
From: Lyon, France

Quote:
Does this 64bit I/O imply a 64bit memory model as well?

Only if you want to load a +4 GB file into memory, I think


_________________
Philippe 'Elwood' Ferrucci
Sam460 1.10 Ghz
AmigaOS 4 betatester
Amiga Translator Organisation

 Status: Offline
Profile     Report this post  
itix 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 11:53:39
#46 ]
Elite Member
Joined: 22-Dec-2004
Posts: 3398
From: Freedom world

Quote:
The new system only discerns between MEMF_SHARED and MEMF_PRIVATE. The old flags are still supported because of backward compatibility.


Does this mean MEMF_PUBLIC is obsolete now? I.e. MEMF_SHARED/ANY is used for message ports etc?


_________________
Amiga Developer
Amiga 500, Efika, Mac Mini and PowerBook

 Status: Offline
Profile     Report this post  
EntilZha 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 11:56:09
#47 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
Does this mean MEMF_PUBLIC is obsolete now? I.e. MEMF_SHARED/ANY is used for message ports etc?


MEMF_SHARED, yes. MEMF_ANY doesn't qualify as it can also be MEMF_PRIVATE, which is a rather bad idea for message ports.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
nicomen 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 12:55:29
#48 ]
Cult Member
Joined: 5-Nov-2003
Posts: 539
From: Trondheim, Norway

@Entilzha

We were discussing Message ports and how they fit in an memory protective environment on the dev channel yesterday.

Shouldn't it be possible to use private memory for message ports if they are just going to be accessed thru OS functions anyway? The most extreme thing is that addresses need to be identical throughout appications if requested, but shouldn't the normal memory abstration (logical memory space) be able to cope with that? You could even don't replicate the msgports for each program unless they are written to (copy-on.write).

Feel free to enlighten me if you have time ;)


_________________
Nicolas Mendoza

 Status: Offline
Profile     Report this post  
Laser 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 12:58:23
#49 ]
Regular Member
Joined: 19-Jul-2003
Posts: 333
From: Norwich, UK

Nicely written article. Good to see the developers have concerned themselves with such fundametal improvements of operation in addition to the already vast task of porting the original functionaility.

Can someone explain to me though, how this system materially differs from Memory Pools? (AllocPooled() etc.)

 Status: Offline
Profile     Report this post  
Hammer 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 12:59:20
#50 ]
Elite Member
Joined: 9-Mar-2003
Posts: 5246
From: Australia

Quote:
How does the memory-system of OS4 compare to that of Windows/Linux/Mac OSX?

For Windows XP and Server refer to
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/low_fragmentation_heap.asp


_________________
Ryzen 9 7900X, DDR5-6000 64 GB RAM, GeForce RTX 4080 16 GB
Amiga 1200 (Rev 1D1, KS 3.2, PiStorm32lite/RPi 4B 4GB/Emu68)
Amiga 500 (Rev 6A, KS 3.2, PiStorm/RPi 3a/Emu68)

 Status: Offline
Profile     Report this post  
ErikBauer 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 13:23:53
#51 ]
Super Member
Joined: 25-Feb-2004
Posts: 1141
From: Italy

It seems Window's bucklets get rid of External Fragmentation, wich is good, but you'll have a lot of small internal fragmentations, that are not as bad as External ones, but if you do several little allocations instead of a big one you'll allocate a lot of unused memory.
They pratically used an evolution of the cluster allocation method that is used in common file systems.

IMHO, the one used by OS4, even if appears to be much more complicated, is uncomparably efficient .

Last edited by ErikBauer on 14-Dec-2005 at 01:24 PM.


_________________
God created Paula so that Allister Brimble and Dave Whittaker could do music

Check my Amiga gameplays (ITA)!

 Status: Offline
Profile     Report this post  
EntilZha 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 15:08:48
#52 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
Shouldn't it be possible to use private memory for message ports if they are just going to be accessed thru OS functions anyway?


In theory, yes, but that would require the OS to "change" into the virtual address space of another task, or it would require all memory to be accessible in supervisor state (or "kernel mode", like in Linux).

If others would only use PutMsg to put messages into the port, then it would work, but looking at the recent discussion on UtilityBase about abusing undocumented features in the system, you can imagine that some "clever" guys will queue up the message directly...


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
falemagn 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 15:15:11
#53 ]
Super Member
Joined: 24-Nov-2003
Posts: 1126
From: Italy

@Erik
However, from the article it's not clear how does this system deal with internal fragmentation.

Slab allocators are not general purpose allocators, you generally have a pool of free objects which you take your objects from, all of the same size. In the article there's no actual explanation on how do the free list get built: are they all "slab" lists? If so, are the allocatable objects sizes decided once and for all, or dynamically chosen depending on the usage pattern? Answering this question would also give an hint on what inpact internal fragmentation has on this system.

Last edited by falemagn on 14-Dec-2005 at 03:24 PM.
Last edited by falemagn on 14-Dec-2005 at 03:23 PM.


_________________
It is well enough that people of the nation do not understand our banking and monetary
system, for if they did, I believe there would be a revolution before tomorrow morning.

~~ Henry Ford

 Status: Offline
Profile     Report this post  
EntilZha 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 15:47:32
#54 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

@falemagn

The sizes are selected at system startup, and currently, they are static. But since this functionality is hidden, dynamic cache sizes can be implemented.

The real strength of the slab allocator comes of course from using it to manage distinct objects. The system will gradually use more distinct caches. However, even with the current implementation (there are roughly 10 specialized caches), it's already more space efficient than the old one (not to mention the fact that it'S much faster).

The caches also keep statistics about how much waste they produce, the data is just not used yet for any dynamic adaption... I'll see that I can come up with some data later.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
ErikBauer 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 15:51:32
#55 ]
Super Member
Joined: 25-Feb-2004
Posts: 1141
From: Italy

So, if I'm following it right, this puts OS4's Slab allocation method more or less in peer with Window's 'Basket' allocation method speaking of pure memory waste. But it is transparently adaptable to a dynamic memory handling that would allow less if not zero memory wasting.

Moreover it seems more flexible and has some other nice features that the other one lacks.

Am I right?

Last edited by ErikBauer on 14-Dec-2005 at 03:52 PM.


_________________
God created Paula so that Allister Brimble and Dave Whittaker could do music

Check my Amiga gameplays (ITA)!

 Status: Offline
Profile     Report this post  
EntilZha 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 16:08:23
#56 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
So, if I'm following it right, this puts OS4's Slab allocation method more or less in peer with Window's 'Basket' allocation method speaking of pure memory waste. But it is transparently adaptable to a dynamic memory handling that would allow less if not zero memory wasting.


If you only look at the AllocVec/FreeVec idea, and only at the memory waste, then yes, although OS4 doesn't use strict power of two buckets, but buckets that are ascending by roughly 20 % in size.

Quote:
Moreover it seems more flexible and has some other nice features that the other one lacks.


Yes, especially since the Windows allocator is not "typed" in any way. Slab allocators are typed, with the added benefit of making a segregated memory list approach very simple.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
ErikBauer 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 16:23:42
#57 ]
Super Member
Joined: 25-Feb-2004
Posts: 1141
From: Italy

Thanx alot, I'm always trying to keep up my mind with new technologies and implementations, if you do not go like this in Computer Area you're lost in less than one year.

Btw these are very infos! You guys are doing a very smart job with OS4 (altough a lot of things seem to be 'borrowed' from Linux-Unix, not that it is a bad thing though... )!


_________________
God created Paula so that Allister Brimble and Dave Whittaker could do music

Check my Amiga gameplays (ITA)!

 Status: Offline
Profile     Report this post  
Ami603 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 16:53:47
#58 ]
Cult Member
Joined: 7-Mar-2003
Posts: 580
From: Valencia,Spain 8-)

@ErikBauer:

Wasn't the Amiga philosophy something like:"if it works well for others,we'll benefit&improve it".
Likewise it was the case with Rexx/ARexx,for instance?


_________________
Cuida tus piedras gordas.

A1200/030 32Mb
A4000D
A1-X1000.

 Status: Offline
Profile     Report this post  
EntilZha 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 17:31:10
#59 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
altough a lot of things seem to be 'borrowed' from Linux-Unix, not that it is a bad thing though..


Actually, it's borrowed from Solaris, if at all..

Note that no code was used from other sources than my own. I based the implementation wholly on the published papers...


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
CodeSmith 
Re: Memory Management in AmigaOS4.0 Explained
Posted on 14-Dec-2005 17:33:00
#60 ]
Elite Member
Joined: 8-Mar-2003
Posts: 3045
From: USA

@ErikBauer :

Quote:
(altough a lot of things seem to be 'borrowed' from Linux-Unix, not that it is a bad thing though... )!


This 'borrowing' takes place all the time, it's called 'taking advantage of current research'. If you have a the choice of doing something yourself from the ground up or using the freely available research from a group of PhDs with years of experience, which one would you do? (this doesn't really apply if you're coding something just for fun though...)

Not razzing you, just pointing out that Hyperion is not doing anything lazy or sneaky. Researchers want commercial implementations of their work, it validates it and increases their prestige in the scientific community.

 Status: Offline
Profile     Report this post  

Goto page ( 1 | 2 | 3 | 4 | 5 | 6 )

[ home ][ about us ][ privacy ] [ forums ][ classifieds ] [ links ][ news archive ] [ link to us ][ user account ]
Copyright (C) 2000 - 2019 Amigaworld.net.
Amigaworld.net was originally founded by David Doyle