[mpich-discuss] Redirection and Pipes

[Dave Goodell]

Is your read done repeatedly, or just once?  Your code won't be portable in general if it assumes that you'll always get MAX_LEN out of the read() call.  Do you flush your stdout explicitly at any point?

I'm not sure why the buffering behavior would change when run under MPI or not.  If you are on Linux you can try to use strace to watch for relevant calls (ioctl, fcntl, etc). I think you can also use ltrace or a debugger to watch for calls to fflush, setvbuf, and similar, which will affect the buffering behavior of the standard library.

You might also try to isolate whether it is being caused by using the MPI library (i.e. calling MPI_Init/MPI_Finalize/etc), or by being executed via mpiexec, or both.

It looks like we call "setbuf(stdout,NULL)" at MPI_Init time, so that might be relevant.  Do you perform this redirection before or after MPI_Init time?

-Dave

On Aug 20, 2010, at 9:30 AM CDT, Diego Fernández Slezak wrote:

> Hello,
> 
> I have a working code that redirects stdout to a pipe, runs some code and then process the stdout (through the pipe):
> 
> 
> 		/// Connect STDOUT to pipe
> 		// Save old stdout.
> 		saved_stdout = dup(STDOUT_FILENO);  // save stdout for display later
> 		// create pipe
> 		if( pipe(out_pipe) != 0 ) {          // make a pipe 
> 			exit(1);
> 		}
> 		// connect stdout to pipe.
> 		dup2(out_pipe[1], STDOUT_FILENO);   // redirect stdout to the pipe
> 
> ...
> ...
> //at some point:
> 		int read_bytes = read(out_pipe[0], buffer, MAX_LEN); // read from pipe into buffer 
> 
> ...
> ...
> 
> 		dup2(saved_stdout, STDOUT_FILENO);  // reconnect stdout for testing 
> 
> 
> Now, if I try to run exactly the same code in MPI environment, the "read" function only retrieves part of the message.
> I think this could be because of some MPI redefinition of pipe buffer sizes or similar.
> 
> Any ideas?