@abalaban
>access to file descriptor is not atomic, it never was on any platform. it's up to the >multi thread programmer to handle the case and do what he wants to.

ixemul is a BSD Kernel, and when it work here then on evry linux it work too i guess

and btw if a printf or a fprintf(stdout, (stderr)) work not atomic, then its impossible to get usefull log output in multithreaded apps.you get then only text output mix by task.

and i see in the linux soft no log output command that do a extra semaphore look.

Last edited by bernd_afa on 18-Jun-2009 at 11:17 AM.
Last edited by bernd_afa on 18-Jun-2009 at 11:16 AM.

@bernd_afa

stdout and stderr are not file descriptors. They are filehandles provided by the c runtime library. It is up to the c runtime library to guarantee thread safety or not. 0 1 and 2 are file descriptors and the reason why you usually dont see output interleaved is that the c runtime library will send the output from fprintf() buffered via a line buffer to the kernel. Once write() gets the buffer to output, it is very likely that it will output the entire buffer before the next call to write() on the same kernel file descriptor is made. But it is not guaranteed, nor needed.

regards,
Stefan

@bernd_afa

Quote:

and i see in the linux soft no log output command that do a extra semaphore look.

Maybe because in a correctly designed multithread app, one would have only one thread responsible of the log operations, other threads would only signal this log thread when they have something to log.
The fact it worked during your tests is not a proof of anything, either it worked by accident either your c runtime was handling this by using buffered filehandle as Yabba pointed above, but even in this latest case you are relying on a c runtime implementation hence you would not be portable...

_________________
AOS 4.1 : I dream it, Hyperion did it !
Now dreaming AOS 4.2...
Thank you to all devs involved for this great job !

@bernd_afa

Quote:

and btw if a printf or a fprintf(stdout, (stderr)) work not atomic, then its impossible to get usefull log output in multithreaded apps.you get then only text output mix by task.

Er, do you seriously want to tell me that you'd never seen that on any other system? I've had that on Win32 and I've had that on OpenMP code running on both Windows and Linux.

Last edited by Zardoz on 18-Jun-2009 at 02:37 PM.

_________________

@Zardoz

>Er, do you seriously want to tell me that you'd never seen that on any other system? >I've had that on Win32 and I've had that on OpenMP code running on both Windows >and Linux.

on winuae i notice in write_log entercritical commands.so here seem blovking need,

seem really implemention specific, on Darwin, fprintf is thread safe, read here and i think to avoid thread problems, best is use a thread safe lib.

http://lists.apple.com/archives/Darwin-dev/2008/Jan/msg00139.html

for bsd i have no confirm in that way find, but the stdio in ixemul do before any actions always a lock with that code

static int __do_sync_write(struct file *f, char *buf, int len)
{
usetup;
int err=errno, res = 0;
int omask;

omask = syscall (SYS_sigsetmask, ~0);
__get_file (f);


-------


}

int __get_file (struct file *f)
{
int res = 0;

Forbid ();
for (;;)
{
if (!(f->f_sync_flags & FSFF_LOCKED))
{
f->f_sync_flags |= FSFF_LOCKED;
/* got it ! */
break;
}
f->f_sync_flags |= FSFF_WANTLOCK;
if (ix_sleep((caddr_t)&f->f_sync_flags, "get_file") < 0) /* error in sleep, might be INTR */
{
res = -1;
break;
}
/* have to always recheck whether we really got the lock */
}
Permit ();
return res;
}

@bernd_afa

Quote:

bernd_afa wrote: @Zardoz Er, do you seriously want to tell me that you'd never seen that on any other system? I've had that on Win32 and I've had that on OpenMP code running on both Windows and Linux. on winuae i notice in write_log entercritical commands.so here seem blovking need,

Yes and again if you rethink one second to what Yabba told some posts above and the fact that vast majority of current OS are using *buffered* I/O you'll certainly convey that bufferization operations should be protected by semaphore or mutex operations....

Quote:

seem really implemention specific, on Darwin, fprintf is thread safe, read here and i think to avoid thread problems, best is use a thread safe lib. http://lists.apple.com/archives/Darwin-dev/2008/Jan/msg00139.html

yea sure, just read the three following posts in this thread and you'll notice how best it is.

Quote:

for bsd i have no confirm in that way find, but the stdio in ixemul do before any actions always a lock with that code

I don't know how ixemul works internally but looking at your code I have the impression that you are switching off AmigaOS multitasking while you are waiting for acquisition of the lock on the file handle. In the situation described by your link the situation would be worst because not only your multithreaded app would be dead locked, but the whole system will be also...

Just a side question are file handles shared amongst ixemul programs ?

EDIT: bad things that broke the page layout

Last edited by abalaban on 18-Jun-2009 at 04:32 PM.

_________________
AOS 4.1 : I dream it, Hyperion did it !
Now dreaming AOS 4.2...
Thank you to all devs involved for this great job !

@abalaban

>I don't know how ixemul works internally but looking at your code I have the >impression that you are switching off AmigaOS multitasking while you are waiting
>for acquisition of the lock on the file handle. In the situation described by your link >the situation

the code is not from me written, it is in since long time in and i assume it is from Freebsd.the func is also simular in Darwin, (called swrite).here are other commands in to do atomic.

of course for a short period multitasking must be disabled at least if the CPU need more than 1 instruction.

but look at the code, if the handle is in use by another lock, the func call ix_sleep.Here the task is set to waiting and other tasks can run, until the file get free.

so the time that is on forbid state is very very short.

>Just a side question are file handles shared amongst ixemul programs ?

no, of course not, every main thread have its own handles.the handle table is in task userdata.
only subthreads (with the ixemul SDL Version) share the handles.thats need to get most Unix programs work and the new feature in V61

Last edited by bernd_afa on 18-Jun-2009 at 05:47 PM.
Last edited by bernd_afa on 18-Jun-2009 at 05:44 PM.
Last edited by bernd_afa on 18-Jun-2009 at 05:43 PM.

@broadblues

To add something to the discussion that is actually on-topic, if you are using libpthread.so, the problem might be related to an issue in dynamic linking that was resolved in the latest quickfix. However, new binutils are required to make use of this - I'll try to make these available ASAP, and we're also looking at updating the SDK and a few of the support libraries in SOBJS:

The issue with the dynamic linking was that I had thought it a good idea to look up the symbol a shared procedure call was referencing (a PLTREL24 reloc) and make it a direct jump if the symbol was found, thus bypassing the PLT. This was based on the assumption that the linker would leave PLTREL24 relocs when -fPIC code was linked statically, which it doesn't. The result is that the symbol was always found (because the shared object was already loaded) and thus the jump NEVER went via the PLT.

The implication of that was two-fold: For one thing, I didn't notice that the PLT wasn't even in an executable segment in some cases (which would cause an ISI), and if the code was sufficiently large OR sufficiently far apart after loading, the 24 bit offset wouldn't be enough to perform the jump (that's what normally is done via .PLTresolv / .PLTcall) so it would basically jump somewhere randomly.

This is also the reason why some people had trouble with shared objects and missing libraries. For example, linking against libpng would fail to work if you didn't link against libz as well (which is no longer the case with the quickfix/new binutils).

The problems you describe (random/erratic behavior) are very likely caused by this. So it might be the new binutils will solve two of your issues at once :--)

_________________
Seriously, if you want to contact me do not bother sending me a PM here. Write me a mail

@Rogue

Thanks for the detailed info on dynaic linking and the sobjs problem. Unfortunatly in this case I don't think it can be the problem as whilst blender was using dynamic linking (for plugin loading at least) the ilmthread test from the openEXR package was statically linked.

I sent you some additional info by email late last night, and will get a cut down test program together as soon as I can.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues

I'll have a look at the stuff. Thank you.

_________________
Seriously, if you want to contact me do not bother sending me a PM here. Write me a mail

@Rogue

Could the relocation issues you mention be responsible for "cc1" crashes on my 1GB Sam Flex? I notice that "gcc" appears to have some pthreads. The Grim Reaper reports:

Task 0x566b6aa0 ("gcc:libexec/gcc/ppc-amigaos/4.2.4/cc1")
generated an error of type ISI (Instruction Storage Interrupt)
on address 0x00000000

which looks like the task has jumped to a bogus location to me.

If this is a related issue, could we get a new GCC along with binutils?

Last edited by Xenic on 09-Jul-2009 at 07:25 PM.

_________________
X1000 with 2GB memory & OS4.1FE

@Xenic

I think (but may be wrong) that gcc itself is statically linked.

_________________
BroadBlues On Blues BroadBlues On Amiga Walker Broad

@broadblues
Quote:

I think (but may be wrong) that gcc itself is statically linked.

I think you're right. Somehow I got the impression that the issues mentioned could apply to statically linked objects but a second reading makes me think otherwise. My problem is probably unrelated.

Last edited by Xenic on 09-Jul-2009 at 10:56 PM.

_________________
X1000 with 2GB memory & OS4.1FE

@Xenic

As broadblues pointed out, gcc is statically linked, and it doesn't use pthreads at all.

_________________
Seriously, if you want to contact me do not bother sending me a PM here. Write me a mail