Click Here
home features news forums classifieds faqs links search
6066 members 
Amiga Q&A /  Free for All /  Emulation /  Gaming / (Latest Posts)
Login

Nickname

Password

Lost Password?

Don't have an account yet?
Register now!

Support Amigaworld.net
Your support is needed and is appreciated as Amigaworld.net is primarily dependent upon the support of its users.
Donate

Menu
Main sections
» Home
» Features
» News
» Forums
» Classifieds
» Links
» Downloads
Extras
» OS4 Zone
» IRC Network
» AmigaWorld Radio
» Newsfeed
» Top Members
» Amiga Dealers
Information
» About Us
» FAQs
» Advertise
» Polls
» Terms of Service
» Search

IRC Channel
Server: irc.amigaworld.net
Ports: 1024,5555, 6665-6669
SSL port: 6697
Channel: #Amigaworld
Channel Policy and Guidelines

Who's Online
19 crawler(s) on-line.
 18 guest(s) on-line.
 2 member(s) on-line.


 klx300r,  Musashi5150

You are an anonymous user.
Register Now!
 Musashi5150:  27 secs ago
 klx300r:  53 secs ago
 kolla:  7 mins ago
 kriz:  23 mins ago
 BSzili:  26 mins ago
 AP:  30 mins ago
 MEGA_RJ_MICAL:  1 hr 2 mins ago
 m0lebrain:  1 hr 13 mins ago
 amigadave:  1 hr 39 mins ago
 Hammer:  1 hr 46 mins ago

News   News : ExecSG in Detail: Memory by Thomas Frieden
   posted by MikeB on 10-Sep-2003 9:17:22 (3337 reads)
Amiga Inc has released an article written by Thomas Frieden of Hyperion for CAM Issue #7. The article shares more details about AmigaOS4's ExecSG kernel.
    

Related Links
· More about News
· News by MikeB


Most read story about News
AmigaOS 4.0 Status Report

Last news about News
New articles on Obligement
Printer Friendly Page  Send this Story to a Friend

PosterThread
Anonymous 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 10-Sep-2003 10:10:16
# ]



Im glad this became public, because now I can ask questions publically

Right so you use the MMU to implement virtual addressing, and recognise
the performance issues that can occur when the MMU is satisfying
so many requests by paging the address space. All well and good, can this
paging be variable in size for performance tuning ( as hardware gets
better you may want to hit the algorithm less and maybe constrain
the amount of entries in the table the MMU uses so it doesnt go
into memory accessed across the bus )?

Does each "object" have dominion over a page or does how many pages
an object own depend on what requests it has sated?

You say that memory objects have a series of exits that get called
if a violation occurs by a task inside addresses that it owns. Is there
a default implementation registered with that exit?

If ( and this depends on your answer to my first question ) the object
owns a page, and you implement per application objects, does this
mean that an application occupies at least one page ( giving us a
theoretical limit ) OR does this mean that the architecture completely
changes OR do you see these complementing each other ( in which
case what is the priority order for memory object - per application then
per page - ) e.g. what order do exits get called in?.

 
     Report this post  
EntilZha 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 10-Sep-2003 12:17:20
#2 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
All well and good, can this paging be variable in size for performance tuning ( as hardware gets better you may want to hit the algorithm less and maybe constrain the amount of entries in the table the MMU uses so it doesnt go into memory accessed across the bus )?


Currently, the size is fixed. However, the current implementation can be easily changed to cope with different sizes.

I have to add that this part of the memory architecture is internal to Exec, and will not be accessible from the outside. It's current implementation will probably change in the course of the AG2 migration (I can't get more specific in public, sorry )

Quote:
Does each "object" have dominion over a page or does how many pages an object own depend on what requests it has sated?


These objects are nothing more than glorified MemHeaders, i.e. they span a certain amount of address space and handle all organisation of that area.

Basically, the currently active set of memory objects defines the current address space for the running program (and currently, with only one addres space, they are all global).

Quote:
You say that memory objects have a series of exits that get called if a violation occurs by a task inside addresses that it owns. Is there a default implementation registered with that exit?


Yes. The current default is to generate the equivalent of a segmentation fault.

The different objects implement their own set of functionality (for example, memory objects might map a delayed-mapped page).

There's also the possibility to hook into this functionality with a new Exec functionality: Instead of having one global trap handler for all sort of exceptions, tasks can now install trap handlers for specific events. One of them is the page fault handler (You are a beta tester, so you can look up the function in the SDK: It's called SetTaskTrap).


Quote:
If ( and this depends on your answer to my first question ) the object owns a page, and you implement per application objects, does this mean that an application occupies at least one page ( giving us a theoretical limit ) OR does this mean that the architecture completely changes OR do you see these complementing each other in which case what is the priority order for memory object - per application then per page - ) e.g. what order do exits get called in?.


The current implementation only implements global objects, not per application. Therefore, it's limited to the amount of virtual address space. With 4.1, we want to introduce the possibility for isolated address spaces, and that will allow an unlimited amount.

Generally, don't expect too much from this yet. As I said above, these objects are nothing more than a struct MemHeader with a bit of intelligence. The implementation is internal and will change a lot with AG2.

I hope this answers your questions.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
Toaks 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 10-Sep-2003 14:04:41
#3 ]
Elite Member
Joined: 10-Mar-2003
Posts: 8042
From: amigaguru.com

hmm is this just to clear off some flame's or? , this is old news to me :)))

keep it on hyperion! , love it..


_________________
See my blog and collection website! . https://www.blog.amigaguru.com

 Status: Offline
Profile     Report this post  
Anonymous 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 10-Sep-2003 14:28:48
# ]



@Toaks

The article was recently released to the public. And not everyone has read it from within the CAM before.

/Björn

 
     Report this post  
Laser 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 10-Sep-2003 21:52:18
#5 ]
Regular Member
Joined: 19-Jul-2003
Posts: 333
From: Norwich, UK

Neat...

Two questions:

1) I assume that there's some sort of "memory pools" type of thing, so that several dozen allocations of ten bytes (frequent occurrence in the Amiga system) doesn't go and allocate a separate X Kb RAM page for each one?

2) Can third parties hook into these memory object header wotsits, or define areas of their own under control of one? So that, for example, someone could implement a VMM-alike program (if the OS didn't provide one) using this mechanism. Or does it need to be built in to the OS?

 Status: Offline
Profile     Report this post  
Toaks 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 10-Sep-2003 23:56:45
#6 ]
Elite Member
Joined: 10-Mar-2003
Posts: 8042
From: amigaguru.com

orgin:yeah i know, i belived many had bought the coupon like me..

anyway its good that something gets out this way to get less fud and such from you know whom...

keep it up hyperion!


_________________
See my blog and collection website! . https://www.blog.amigaguru.com

 Status: Offline
Profile     Report this post  
EntilZha 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 11-Sep-2003 12:01:47
#7 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

@ Laser
Quote:
1) I assume that there's some sort of "memory pools" type of thing, so that several dozen allocations of ten bytes (frequent occurrence in the Amiga system) doesn't go and allocate a separate X Kb RAM page for each one?


Yeah, I know that someone on ANN brought up this nonsense that a simple "Hello World" type program would already take up 256 KB of memory.

But that's wrong. Memory allocation is *NOT* done on a per-page basis. If you allocate 10 bytes, you will use 10 bytes of memory, not 64 KB. It is in fact using a memory-pool like structure, although chances are that this will be replaced by something more sophisticated before or shortly after release (I'm thinking about a boundary tag list with a roving pointer, if you want to know).


Quote:
2) Can third parties hook into these memory object header wotsits, or define areas of their own under control of one? So that, for example, someone could implement a VMM-alike program (if the OS didn't provide one) using this mechanism. Or does it need to be built in to the OS?


No, it's a black box.

The reason for this is simply that the implementation isn't very advanced, and will be replaced in the course of the AG2 migration. Therefore, we do not want to open up such low-level functionality to the user. It might be hard to change it otherwise.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
Laser 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 11-Sep-2003 13:22:24
#8 ]
Regular Member
Joined: 19-Jul-2003
Posts: 333
From: Norwich, UK

Quote:
Yeah, I know that someone on ANN brought up this nonsense that a simple "Hello World" type program would already take up 256 KB of memory.


Yeah, I figured that was kind of unlikely, given the attention to detail evident in other released snippets.

Quote:
(I'm thinking about a boundary tag list with a roving pointer, if you want to know).


Ahem... No idea what that means, sorry! Never mind...


I liked the idea of both the unix-like "huge file access as if it was in memory" and the "allocated memory not actually used up until accessed". (And obviously the swap-memory, like everyone else.) Will these possible uses of the new memroy objects actually be in OS4.0 (4.x?) or were they simply examples plucked from the air?

 Status: Offline
Profile     Report this post  
EntilZha 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 11-Sep-2003 14:14:34
#9 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
Will these possible uses of the new memroy objects actually be in OS4.0 (4.x?) or were they simply examples plucked from the air?


The delayed memory mapping is already implemented (and works ).

The next thing will be the automatically enlarging stack. The swapping is planned, but there are still issues to be resolved. If these are resolve, the memory mapped files are just an application of the swapping mechanism.

So no, the examples were not just plucked from the air


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
Laser 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 11-Sep-2003 17:34:05
#10 ]
Regular Member
Joined: 19-Jul-2003
Posts: 333
From: Norwich, UK

Quote:
So no, the exmples where not just plucked from the air


Heheh! Cool.

Once again, congrat's to you and the rest of the OS4 team for some great work.

 Status: Offline
Profile     Report this post  
Georg 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 12-Sep-2003 11:41:16
#11 ]
Regular Member
Joined: 14-May-2003
Posts: 446
From: Unknown

Quote:
If you allocate 10 bytes, you will use 10 bytes of memory, not 64 KB


10 bytes? Not a multiple of 8 (struct MemChunk), or
bigger. Ie. at least 16 Bytes in the case of allocating
10 Bytes?

 Status: Offline
Profile     Report this post  
Georg 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 12-Sep-2003 11:44:33
#12 ]
Regular Member
Joined: 14-May-2003
Posts: 446
From: Unknown

Quote:
The next thing will be the automatically enlarging stack


What about the (not few) cases where stack is allocated
by hand using AllocMem (with parts of allocation possibly
used for other things), or is part of a library base structure,
or is part of an app's BSS hunk. What about StackSwap()?

 Status: Offline
Profile     Report this post  
EntilZha 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 12-Sep-2003 12:33:55
#13 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
10 bytes? Not a multiple of 8 (struct MemChunk), or bigger. Ie. at least 16 Bytes in the case of allocating 10 Bytes?


Of course, this is still true. In fact, all memory allocations (also those with AllocVec) are now aligned to 8 bytes, due to alignment restrictions of the PowerPC. So in fact, allocating 10 bytes would in the worst case need 24 bytes.

However, not 64 KB. That was the point I tried to make.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
EntilZha 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 12-Sep-2003 12:35:59
#14 ]
OS4 Core Developer
Joined: 27-Aug-2003
Posts: 1679
From: The Jedi Academy, Yavin 4

Quote:
What about the (not few) cases where stack is allocated by hand using AllocMem (with parts of allocation possibly used for other things), or is part of a library base structure, or is part of an app's BSS hunk.


What about it ? They will work as before. These automatic stacks have to be specifically asked for.

Quote:
What about StackSwap()?


See above, works like before.


_________________
Thomas, the kernel guy

"I don't have a frigging clue. I'm norwegian" -- Ole-Egil

All opinions expressed are my own and do not necessarily represent those of Hyperion Entertainment

 Status: Offline
Profile     Report this post  
Georg 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 13-Sep-2003 21:01:22
#15 ]
Regular Member
Joined: 14-May-2003
Posts: 446
From: Unknown

Quote:
These automatic stacks have to be specifically asked for


I see. Didn't know that. I was remembering to have read
that automatic stack extension in AOS4 is ~"transparent to
68k programs", ie. automatically also working with those,
but had doubts on how this could possibly work with
things mentioned (stackswap, hand allocated stack, etc.)

 Status: Offline
Profile     Report this post  
Georg 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 13-Sep-2003 21:09:15
#16 ]
Regular Member
Joined: 14-May-2003
Posts: 446
From: Unknown

Quote:
Of course, this is still true. In fact, all memory allocations (also those with AllocVec) are now aligned to 8 bytes, due to alignment restrictions of the PowerPC. So in fact, allocating 10 bytes would in the worst case need 24 bytes.

However, not 64 KB. That was the point I tried to make.


Ok. What about stack memory, the one with automatic
stack enlargment. This eats physical memory in
64 KBytes chunks, right? So running say 10 programs
(which have automatic stack enlargment "activated")
eats at least 10*64 KBytes physical memory, right?

 Status: Offline
Profile     Report this post  
CodeSmith 
Re: ExecSG in Detail: Memory by Thomas Frieden
Posted on 14-Sep-2003 21:41:00
#17 ]
Elite Member
Joined: 8-Mar-2003
Posts: 3045
From: USA

Quote:
These automatic stacks have to be specifically asked for.

How do you ask for auto-enlarging stacks? is it something the program itself must do, or something like a new flag in Task::tc_Flags? if it's the former, how will older programs benefit from automatically enlarged stacks? if it's the latter, will programs calling StackSwap "turn off" the autostacks? will StackSwap()'ing autostacks cause the memory to be leaked (since I guess you can't FreeMem an autostack)?

We need autodocs darn it!

 Status: Offline
Profile     Report this post  
[ home ][ about us ][ privacy ] [ forums ][ classifieds ] [ links ][ news archive ] [ link to us ][ user account ]
Copyright (C) 2000 - 2019 Amigaworld.net.
Amigaworld.net was originally founded by David Doyle