[mpich-discuss] Hydra issues

[Scott Atchley]

On Aug 26, 2009, at 10:35 AM, Pavan Balaji wrote:

>> Ok, I was not patient enough. If I let it run, it eventually  
>> starts. The actual walltime is nearly the same as when I use  
>> proxies, but the stdout is delayed until the application completes.
>
> Hydra itself does not do any buffering of stdout/stderr as it comes.  
> The problem is that hydra never gets the stdout/stderr events till  
> the application process flushes stdout/stderr (or libc's internally  
> buffering is full). In fact, this is the same problem with mpd as  
> well. For example, you can try a simple program which does
>
> for (5 loops) { printf("foo\n"); sleep(1); }
>
> ... and you'll notice that the stdout doesn't get printed till the  
> end (both with hydra and mpd). Instead, if you change the above test  
> to:
>
> for (5 loops) { printf("foo\n"); fflush(stdout); sleep(1); }
>
> ... this should show the output as it appears.
>
> We, obviously, can't expect users to go around adding fflush() in  
> their code, so this is not a good solution. Guillaume @ INRIA and I  
> had tried some experiments with this, but there wasn't any solution  
> that we could use to work around it. Maybe it's time to look into  
> this some more.
>
> -- Pavan

Pavan,

Is there a downside to using:

setbuf (stdout, NULL);
setbuf (stderr, NULL);

?

Also, why does using persistent proxies allow immediate output?

Scott