Every now and again a it seems a that a program lauching a new thread with pthreads will hang with both the prgram and thread apparently waiting on the same semaphore. I've found that if you identify the thread with Scout and send it a signal the lockup can be freed. But you must get the right thread or it crashes!!!!

I've notice this happening a few time with my blender port, but just now with the test program for OpenEXR is happened a lot.

In the blender case it happens whilst rendering animations and trying use other apps (say web browsing) at the same time. In fact I've most often noticed it when browsing with AWeb (but also OWB) whilst tesing a render sequance.

The OpenEXR test program I was just running has a routine that spawns literally 10000's of threads (not simultaneously) and every 10000 or so it would stop, and I had to release it with the signaling tecnque described above. Once the routine was other the test program thought it had passed

Some kind of race condition bug in pthreads? Or even in the systems semaphores?


THis is on AmigaOne EX G4 800 - OS4.1 (53.5)

Last edited by broadblues on 17-Jun-2009 at 03:34 PM.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

I vaguely recall a post on amigans.net describing the old pthreads implementation as "broken". It has actually been updated in the newest SDK. Bean was encouraged to update Tunenet to use the newer pthread because of the problems.. but he actually removed the pthread dependency altogether. But the point is.. it had problems earlier. Please search amigans.net if you want more info.

Hello,

Blender 2.5 got flexible.

http://vimeo.com/5137477

Sorry if not related to the thread subject, just can't resist to post the link

@broadblues

You port blender to AOS4 ???

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@broadblues

I don't know much about semaphores but if you're using the static one from SDK:local/newlib/lib/libpthread.a (5.1.2009 144,370 bytes) it should be the latest version.

If you're using the dynamic one from WB4.1:SObjs/libpthread.so (2.7.2007 155,465 bytes) that one is older.

The one in SDK:local/newlib/lib/libpthread.so (5.1.2009 148,741) is newer.

Good luck with the port and thanks for doing it, I can't wait to try it!

_________________
AlexC's free OS4 software collection

AmigaOne XE/X1000/X5000/UAE-PPC OS4 laptop/X-10 Home Automation

@jahc

Yes I remember the thread. I am building with the latest SDK so am using the recent version of pthreads.library so the problem (probably) doesn't come from that angle.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@Mrodfr

I'll have a look at that vimeo when I've got time to fire up my linux box (for the flash).

Yes, I'm porting blender, I have most things working, except a few that require depth buffer read access, which is not supported by minigl, and is not working in warp3d for radeon drivers. Basically that means no sculpt mode ;-(

I'm working on the 2.48a source. 2.5 has been rewritten so likely would require porting from scratch, although I haven't looked at the source.

[edit]
changed have to haven't!
[/edit]

Last edited by broadblues on 17-Jun-2009 at 01:35 PM.
Last edited by broadblues on 17-Jun-2009 at 12:36 PM.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@AlexC

Quote:

don't know much about semaphores but if you're using the static one from SDK:local/newlib/lib/libpthread.a (5.1.2009 144,370 bytes) it should be the latest version.

I'm using that version. Satically linked. although IIUC the both the staic and dynamic ones are wrapers for the pthreads.library

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

Great news

BTW, are you in touch with Yomgui, the developper of the morphOS version ???

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@Mrodfr

No. The main issues are GL related. Morphos uses a different gl implementation, so will have different bugs.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

OK, BTW, thanks for your development for porting blender to AOS4.x

Now with my SAM, this subject (3D) and this software (a modern 3D software for AOS4.x) interest me a lot. Thanks again for your actually developped port of Blender 3D for AOS4.x.

_________________
BTW, what you have done for the amiga today ????

-A1200+Mediator+VooDoo3+060/50+96mo+SCSI-KIT
-SAM440EP-667mhz-on MapowerKC3000+AOS4.1

Amiga Docs Disks Preservation Project

@broadblues

Do you assume the Morphos version has bugs or what? :)

>The OpenEXR test program I was just running has a routine that spawns literally >10000's of threads (not simultaneously) and every 10000 o

where is the source of it, so can compile on other systems if it work.

@bernd_afa

http://www.openexr.com/downloads.html

in the openEXR-1.6.1 package but you'll also need the ilmbase package too.

It built more or less out of the box, apart from the ocaasional unix filepath (in the test progs) and the fact that the configure script had default settings for amigaos built in. You'd think that would be agood thing but it assumed a 68020 cpu!

I would think this is likely a OS4 issue though, although proving that works elsewhere can't do any harm


NOTE: This issue is with amigaos semphores being used inside pthreads not with pthread semaphores as described by semaphore.h, these aren't suported by AmigaOS4 at present and so if you try and test openEXR you will need to disable semaphore.h on any system that has it, to get an similar setup.

Last edited by broadblues on 17-Jun-2009 at 04:01 PM.
Last edited by broadblues on 17-Jun-2009 at 03:35 PM.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

>NOTE: This issue is with amigaos semphores being used inside pthreads not with >pthread semaphores as described by semaphore.h, these aren't suported by >AmigaOS4 at present and so if you try and test openEXR

i think thats not good, use amiga semaphore and pthreads, which use own task scheduling and own semaphores.because teh semaphore and tasks must fit.look in task manger of OS4.Are there when pthread is used this 1000 amiga tasks show or not ?

only when there is every amiga thread show amiga semaphore can signal the right task or you need the pthread semaphores that know about all tasks pthread create.

I see in the pthread source no amiga task create, so i think pthread use its own child task scheduler and so it need semaphore system that know about this intern create tasks

either pthread must change to use amiga tasks or must use pthread samaphore

@bernd_afa

But you don't have the os4 pthread sources. I doubt it's implemented with its own task scheduler like the old ixemul one (that really doesn't work well, btw).

Last edited by Fab on 17-Jun-2009 at 07:02 PM.

@Fab

>But you don't have the os4 pthread sources. I doubt it's implemented with its own >task scheduler like the old ixemul one (that really doesn't work well, btw).

Yes, so i ask in my last Post if there are all pthread tasks show in OS4 task manager.
using for pthread amiga tasks is the best way i think only so its possible to singlestep programs with more tasks in GUI debugger and find bugs fast.

i dont know wy here often is tell that ixemul work not good.its the only lib i see that work thread safe with best Unix compatibility.

when i do a simple sdl program that start 5 tasks and every task do a SDL_Wait(10);and print out in a loop hello world\n

and link with libnix, then you can see that this is not threadsafe and between the text helloworld is writen text from other tasks.with ixemul work 100% ok

output on libnix look sometimes as this

hellhelloworldoworld

this happen when the task is switch during output.On V61 ixemul this work ok, so i dont know wy ixemul task scheduler not work well

if OS4 pthreads work well, its also possible to try this simple test.

EDIT:

pthreads is LGPL licence, so where can get the OS4 pthread sources ?

Last edited by bernd_afa on 18-Jun-2009 at 10:14 AM.

@bernd_afa

A task shows in scout for each active OS4 pthread

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@bernd_afa

Quote:

when i do a simple sdl program that start 5 tasks and every task do a SDL_Wait(10);and print out in a loop hello world\n and link with libnix, then you can see that this is not threadsafe and between the text helloworld is writen text from other tasks.with ixemul work 100% ok

Er, if every task executes SDL_Wait(10); and then prints, then they will start printing at approximately the same time and a task switch or real multiple hardware threads will make the text a mess. You can't expect printf for instance to be aware of threads in that way on any system, if you want singular access to the console or any other resource the only way to be certain is to use some form of locking.

Quote:

pthreads is LGPL licence, so where can get the OS4 pthread sources ?

A pthreads implementation does not have to use the pthreads source code.

Last edited by Zardoz on 18-Jun-2009 at 10:42 AM.

_________________

@bernd_afa

access to file descriptor is not atomic, it never was on any platform. it's up to the multi thread programmer to handle the case and do what he wants to.
If you are adding such thing both into file descriptor handling and the task scheduler than you are breaking things because it means you are preventing one task from being switched while there is ongoing output if the task is writing a huge amount of data to the file descriptor (or if the file descriptor is connected to a slow, synchronized size-limited medium - say a socket for example - then you are seriously hitting multitasking which is bad, very bad !!

_________________
AOS 4.1 : I dream it, Hyperion did it !
Now dreaming AOS 4.2...
Thank you to all devs involved for this great job !

@broadblues

Quote:

Some kind of race condition bug in pthreads? Or even in the systems semaphores?

I don't know OS4, but assuming the system (Exec) semaphores work correctly, then the situation where two tasks wait for the same semaphore con only happen if A) there's a third task (maybe some helper task) which owns it, or B) something messed up SysBase->TDNestCnt (Forbid()/Permit() nesting counter) causing the semaphore internal protection/locking to fail.