Since the QuickFix, OS4.1 has been running long enough without crashing that I've gradually noticed a problem: After a few hours (say 4-8) without a reset, the available memory drops very low, and I have to reboot (or else OS4 probably freezes).

I thought this was probably a leaky program, but I finally got around to leaving it running after booting, without running anything (nor any screen blankers, nor moving the mouse), and got a bit of a shock: In 1 hour it has lost about 8% of it's free memory, or roughly 40MB. As it starts at 77% free, that means it will reach 0% in about 11 hours. This figure is pretty close to my experience of OS4 tending to run out of memory after 8 hours (when I am running some programs that use a bit of memory), so I am inclined to believe it is the same problem.

I can even see the memory slowly drop in the Workbench's title bar, which is why I first considered the possibility.

I will leave it running longer, to get a more accurate estimate of the memory loss, but does anybody have any idea what could cause it? Possibly it is still a 3rd-party program I have in the WbStartup folder or User-Startup, so I will try disabling those next...

Just a quick note: I am using the Debug kernel, but I don't see why that should cause the problem.

Last edited by ChrisH on 16-Sep-2009 at 09:52 PM.
Last edited by ChrisH on 16-Sep-2009 at 09:52 PM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@ChrisH

Bear in mind that memory allocation and deallocation is taking place all the time, even when the system appears idle.

In light of this, I shall reitterate what has been said many times before, free memory figures on OS4.1 are meaningless, so don't believe what you read in the WB titlebar.

Simon

_________________
Simon

Comments made by me on any public fora are not representative of, or on behalf of, any company I may have, or assumed by the reader to have, any association with.

Any comments are a personal opinion, and should be accepted as such.

@ChrisH

Hello friend! I do not have any idea what could be causing the problem but I can verify that it does exists. I have experienced the same problem you are describing on my SamEp667.

_________________

@rigo Quote:

free memory figures on OS4.1 are meaningless, so don't believe what you read in the WB titlebar.

That is not the problem! I have *seen* that when free memory reaches 0%, the virtual memory pager will kick-in (if I am lucky), or OS4 will freeze (if I am not lucky). That's why I have to reboot when memory gets low.

If I am compiling a large program, I need most of that 512MB free. If I have had OS4 running for some time, it doesn't have enough free, so compilation runs very slowly. Therefore I have to reboot before compiling something.


Rant: If OS4.1 had a proper reporting of free memory (it should not be rocket science!) then we would not even be having this argument. This is one area where MorphOS definitely has the advantage, even if it's TLSF allocator isn't quite as advanced as OS4's Slab allocator.

Quote:

Bear in mind that memory allocation and deallocation is taking place all the time, even when the system appears idle.

Of course. Which is partly why I am so worried - is some OS4.1 component allocating memory, but not freeing it? Or is there some bug in OS4.1's new (super-complicated) memory allocator? Both seem unlikely, but are getting more likely as I am gradually ruling-out 3rd-party software...

Last edited by ChrisH on 16-Sep-2009 at 10:47 PM.
Last edited by ChrisH on 16-Sep-2009 at 10:09 PM.
Last edited by ChrisH on 16-Sep-2009 at 10:07 PM.
Last edited by ChrisH on 16-Sep-2009 at 10:04 PM.
Last edited by ChrisH on 16-Sep-2009 at 10:01 PM.
Last edited by ChrisH on 16-Sep-2009 at 10:00 PM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@rigo
BTW, other people have previously reported a similar-sounding problem. I had dismissed them for the same reasons you dismissed me. I was probably mistaken - certainly I am sure that you are about me.

Last edited by ChrisH on 16-Sep-2009 at 10:14 PM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

UPDATE:

After 2.5 hours the memory loss was at 19%. My most accurate measurements showed that it was loosing memory at very close to 38MB/hour.

I have now rebooted, with the WbStartup & User-Startup disabled, and the early signs are that free memory is decreasing at the same rate... So this seems to disprove the "3rd-party software is to blame" hypothesis, and points the finger rather strongly towards OS4.1 itself

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@ChrisH

Or at your interpretation of the "figures".

What happens if you just leave it to stew by itself? Does the system freeze after so many hours? I think you'll find that the reading stabilises after a while. It does here, anyway.

And in any case, you still haven't claimed that there is anything wrong except the figure reported.

_________________
cheers
tony

Hyperion Support Forum: http://forum.hyperion-entertainment.biz/index.php

@ChrisH

Quote:

Rant: If OS4.1 had a proper reporting of free memory (it should not be rocket science!) then we would not even be having this argument. This is one area where MorphOS definitely has the advantage, even if it's TLSF allocator isn't quite as advanced as OS4's Slab allocator.

I think we already discussed it once. And what's sure is that absolutely nothing indicates SLAB would be more advanced, quite the contrary, in fact:
- SLAB/SLUB/SLOB wastes much more memory (almost 50% more).
- SLAB/SLUB/SLOB allocation/deallocation time is not constant, unlike TLSF.
- Nothing indicates SLAB/SLUB/SLOB performs better than TLSF regarding fragmentation (which is quite low in TLSF).
- TSLF is more recent. You can see TLSF patches for linux kernels (which originally uses SLAB/SLUB). These patches aren't written just because people are bored. :)

@ChrisH
Quote:

I have now rebooted, with the WbStartup & User-Startup disabled, and the early signs are that free memory is decreasing at the same rate... So this seems to disprove the "3rd-party software is to blame" hypothesis, and points the finger rather strongly towards OS4.1 itself .

I have a µA1 and a SAM Flex and I can tell you for sure that memory management problems exist on the SAM.
For one thing SAM Flex USB eats memory that is not returned to the system. If you monitor free memory while you move your pointer around you will see memory decline in small amounts and never be freed. If you try to back up back up 10+ GB of data to a USB mass storage device (a 16GB Flash drive in my case) your memory will eventually disappear and the system will freeze. The USB memory eating problem was confirmed on the OS4 ML and I was told that SAM USB continuously allocates new memory; expecting that the OS 4.1 memory management system would free it up if needed. I was told that SAM USB will reuse buffers in a future update to eliminate the memory consumption.

When I create a low memory situation on my µA1 by copying large amounts of data to the RAM drive, the system will start removing unused resources like libraries to free memory and begin using the SWAP partition if necessary. On my SAM Flex the system just freezes or crashes under the same condidtions. There is something wrong with memory management on the SAM Flex.

I get weary of hearing the OS 4.1 cheerleaders blame it on 3rd party software and repeat the promotional claims that OS 4.1 manages the memory automatically and that we should pay no attention to the free memory listed by WorkBench. Maybe that's true to some extent on the A1/µA1 (I'm slightly skeptical about how well it works there too) but SAM memory management is broken and I have the freezes and crashes to prove it.

_________________
X1000 with 2GB memory & OS4.1FE

a possible problem could be a logfile that is written to and thus it's growing and stealing ram if it's placed in t:

i had this happen with samba once.

_________________
AOS4 Betatester, Peg2, G4@1ghz, Radeon 9250 256mb, 1gb RAM.

http://www.asciiarena.com
http://www.uprough.net

@ChrisH

It could be the USB Stack. Not the size of the usblog in t: being generated.

When you notice your ram going down kill the usbstack and see if the memory leak stops.

C:usbctrl stop

Or unplug all HID devices and see if the problem still exists.

Last edited by JurassicC on 17-Sep-2009 at 07:30 AM.

_________________
A1200T 603e 330Mhz - Mediator TX OS4.1 F.E.
CDTV 8MB Fast, OS3.1, SCSI, MicroSD SCSI & CD32 FMV
X5000, X1000, A1XE with OS4.1F.E.

@ChrisH

Quote:

Rant: If OS4.1 had a proper reporting of free memory (it should not be rocket science!) then we would not even be having this argument. This is one area where MorphOS definitely has the advantage, even if it's TLSF allocator isn't quite as advanced as OS4's Slab allocator.

If WB bar memory report is so inaccurate, just remove it. Probably OS4 team didn't have an idea of what usability means. Like having both AHI and sound prefs.... You rewrote all the OS, why don't modernize and reorganize it at the same time?

_________________
ikir

@ikir

Quote:

ikir wrote: @ChrisH Quote: Rant: If OS4.1 had a proper reporting of free memory (it should not be rocket science!) then we would not even be having this argument. This is one area where MorphOS definitely has the advantage, even if it's TLSF allocator isn't quite as advanced as OS4's Slab allocator. If WB bar memory report is so inaccurate, just remove it. Probably OS4 team didn't have an idea of what usability means. Like having both AHI and sound prefs.... You rewrote all the OS, why don't modernize and reorganize it at the same time?

Don't assume that just because something hasn't been done yet means that the OS4 team is clueless. There are many things on the to-do lists of the OS4 team, and they all take time.

Hans

_________________
http://hdrlab.org.nz/ - Amiga OS 4 projects, programming articles and more. Home of the RadeonHD driver for Amiga OS 4.x project.
https://keasigmadelta.com/ - More of my work.

UPDATE:

After about 9.5 hours the memory loss was at 69%. Using more accurate memory & time measurements than before (plus a long time interval) put the memory loss at 37.7MB/hour.

So with 10% free memory, I ran User-startup & the programs in WbStartup. Then started Tunenet. That put free memory down to 3%. I then ran OWB... At that point the USB mouse stopped working, which is a pretty damn good indication that the VM pager had kicked-in (due to a previously seen bug on OS4.1 Sam). Free memory was at 0%, surprise surprise!


Using the keyboard I was able to tell OWB to go to it's Homepage. At that point I got a GuruMeditation-like flashing box at the top of the screen indicating indicating a problem. Tunenet is still running, and I can still use the keyboard, so I click on an AmiDock icon... The icon disappears, but nothing else happens - AmiDock seems frozen. I guess that OWB wants some non-paged memory (maybe only a few bytes), but since that has all been used-up it crashed. And that then caused the subsequent problems.

Last edited by ChrisH on 17-Sep-2009 at 09:24 AM.
Last edited by ChrisH on 17-Sep-2009 at 09:14 AM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@tonyw Quote:

I think you'll find that the reading stabilises after a while. It does here, anyway.

As previously explained, I had unscientifically noticed that memory ran out after 4-8 hours of use (VM pager kicked-in, etc). With my most recent more scientific measurement (see post above), it is clear that it does NOT stabilise. This clearly indicates a memory leak.

Quote:

And in any case, you still haven't claimed that there is anything wrong except the figure reported.

Then please re-read what I said. I think I made it QUITE clear that I thought the free memory figures were basically CORRECT - and that they showed a leakage problem.

@Fab Quote:

I think we already discussed it once. And what's sure is that absolutely nothing indicates SLAB would be more advanced

Possibly we have previously discussed it long ago, but I have since gained a better understanding of OS4.1's Slab implementation: Please go read-up on the "VMem" allocator in relation to Slabs.

You will see that VMem is O(1) & therefore modern Slab implementation's like OS4's is also O(1). VMem is actually very similar to TLSF, but predates it by a few years & so is not quite so clever (or memory efficient).

I do NOT intend to debate the pros & cons of Slab allocators vs TLSF *here*. Please start another thread if you wish to do that.

@Xenic Quote:

There is something wrong with memory management on the SAM Flex.

Thanks for the confirmation! And nice to know that OS4.1 is NOT bugged on the MicroA1, so there is hope for OS4.1 on Sam.

@Spot-up
Problem is NOT a log file, unless a clean OS4 installation creates such a log file (which I doubt).

@JurassicC
I had to reboot, but after 16 mins running, I would have expected 10MB of lost memory. Yet T:usb.log is only 3KB in size, and has not increased.

(inaccurate memory reporting)
@Hans Quote:

Don't assume that just because something hasn't been done yet means that the OS4 team is clueless.

I don't think I ever assumed that it WON'T be done. Just that it has NOT been done yet - and that I found this quite surprising. My short rant was supposed to provide a little extra incentive to get it fixed sooner.

Last edited by ChrisH on 17-Sep-2009 at 09:29 AM.
Last edited by ChrisH on 17-Sep-2009 at 09:26 AM.

_________________
Author of the PortablE programming language.
It is pitch black. You are likely to be eaten by a grue...

@ChrisH

Hi!
I don't see this problem on my A1-XE. Never did. The only thing eating memory over time in my system is all my crashed programs in the background after failed programming attempts The pager swaps them away when needed though and I get my memory back.

My guess is that some log buffer is getting larger and larger over time. Check T: if there is an actual logfile there that is getting huge. But since you're running the debug kernel perhaps the "DumpDebugBuffer" log grows all the time.

Do you have any "kdebug" line active in startup-sequence? Mine is commented out. Perhaps its possible to limit any such buffer to be a roll-over buffer with a fixed maximum size?

EDIT: About the free memory: I also think that the cached free memory should be included in the freemem reported since it *is* free to be allocated even if it hasn't been put back into the slabs or whatever.

Last edited by Deniil715 on 17-Sep-2009 at 10:24 AM.

_________________
- Don't get fooled by my avatar, I'm not like that (anymore, mostly... maybe only sometimes)
> Amiga Classic and OS4 developer for OnyxSoft.

Just my experience here:

My SAM runs since 6 hours, I started of with 410 MB memory at the moment I am at 341MB. I did nothing than moving an clicking mouse once an hour. I did not even open a directory. BUT

"memory loss" seems to become less with the time

It lost 50 MB the first hour (you could watch the countdown nearly lineraly), the rest was only a loss of 17 MB.

Please keep in mind, this proofs nothing as
1. WBStartup was not disabled
2. got a few backgroundtasks running (FTP Server e.g.)
3. User-Startup was not disabled
4. not a clean/new install
5. USB is still running
6. I have NOT installed the QuickFix !!!

If I have got time, I will go into deeper testing..

_________________
Geomarketing at
www.geobiz.de
www.gebietsplanung.net
www.geomarketing-consultant.de

@gerograph

Quote:

6. I have NOT installed the QuickFix !!!

Install the QuickFix first. It's not a good idea to report bugs of older software which might be fixed in the update already.

@Deniil715
Quote:

About the free memory: I also think that the cached free memory should be included in the freemem reported since it *is* free to be allocated even if it hasn't been put back into the slabs or whatever.

I second that.

Last edited by Tomppeli on 17-Sep-2009 at 02:11 PM.
Last edited by Tomppeli on 17-Sep-2009 at 02:10 PM.

_________________
Rock lobster bit me. My Workbench has always preferences. X1000 + AmigaOS4.1 FE
"Anyone can build a fast CPU. The trick is to build a fast system." -Seymour Cray

@rigo

Quote:

@ChrisH Bear in mind that memory allocation and deallocation is taking place all the time, even when the system appears idle. In light of this, I shall reitterate what has been said many times before, free memory figures on OS4.1 are meaningless, so don't believe what you read in the WB titlebar. Simon

That is the same answer that i received when i pointed out the problem, but if it was true the system shouldn't freeze for a memory problem ( there is not enough memory available ), instead the system freezes. Infact more than one time, i found my self ( i use a lot the ram disk to pack and unpack/download archives ) with 512kb of free mem and the system freezed for obvious reasons..

If it is only a problem about " system notification " it should be fine and works without problem but that's not..

_________________
GDG-Entertainment
OS4Games
Iksnet
Bitplane

@Xenic


Quote:

If you try to back up back up 10+ GB of data to a USB mass storage device (a 16GB Flash drive in my case) your memory will eventually disappear and the system will freeze. The USB memory eating problem was confirmed on the OS4 ML and I was told that SAM USB continuously allocates new memory; expecting that the OS 4.1 memory management system would free it up if needed. I was told that SAM USB will reuse buffers in a future update to eliminate the memory consumption.

i had a problem like this, but wit Windows.
i have saved many video files from WinXP to Vista into lan and some times Vista alerts me with an Out of Memory error.
Vista coud not operate because there aren't more memory to go on.

But in the system i had more than 2 GB of free RAM ...

_________________