[mpich-discuss] mpich2 and pvfs2 error

[Dorian Krause]

Hi,

there has been a lot discussion of discussion concerning pvfs2 and 
mpich2/romio on this list but I couldn't find a hint for the following 
problem:

I'm working on a CentOs Rocks-distribution cluster and we have the pvfs2 
roll installed (which basically set's up a pvfs2 2.7.1). The output of 
pvfs2-ping shows that everything should work find and I can use all the 
unix commands like ls, touch, ...

However, MPI-IO fails:

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/mnt/pvfs2 kraused$ cat /home/kraused/bnchmk/Filesystem/mpio.c

#include <stdio.h>
#include <stdlib.h>

#include <mpi.h>

int main(int argc, char** argv)
{
    MPI_File fh;
    int      err, N;
    char     msg[MPI_MAX_ERROR_STRING];

    MPI_Init(&argc, &argv);

    if(2 != argc) {
        MPI_Abort(MPI_COMM_WORLD, 1);
    }

    err = MPI_File_open(MPI_COMM_WORLD, argv[1], MPI_MODE_WRONLY | 
MPI_MODE_CREATE /* | MPI_MODE_UNIQUE_OPEN */, MPI_INFO_NULL, &fh);
    printf(" MPI_File_open returned %d\n", err);

    MPI_Error_string(err, msg, &N);
    printf(" MPI_Error_string(%d) = %s\n", err, msg);

    MPI_File_close(&fh);

    return MPI_Finalize();
}

/mnt/pvfs2 kraused$ /home/kraused/mpich2-1.1.1/bin/mpiexec -np 1 
/home/kraused/bnchmk/Filesystem/mpio pvfs2:///mnt/pvfs2/abc
[E 14:42:04.891854] invalid (NULL) required argument
[E 14:42:04.891992] PVFS_isys_create call: Invalid argument
[E 14:42:04.892686] invalid (NULL) required argument
[E 14:42:04.892717] PVFS_isys_create call: Invalid argument
 MPI_File_open returned -524286
 MPI_Error_string(-524286) = Undefined dynamic error code
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The problem seams to be the creation of new files as touching and 
afterwards opening works:

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/mnt/pvfs2 kraused$ touch abc
/mnt/pvfs2 kraused$ /home/kraused/mpich2-1.1.1/bin/mpiexec -np 1 
/home/kraused/bnchmk/Filesystem/mpio pvfs2:///mnt/pvfs2/abc
 MPI_File_open returned 0
 MPI_Error_string(0) = No MPI error
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

The mpich2 version is 1.1.1 and the configure flags are

./configure --prefix=/home/kraused/mpich2-1.1.1 --enable-romio 
--with-file-system=pvfs2+ufs+nfs CFLAGS=-I/usr/local/src/pvfs-2.8.1/include/

Any hints/ideas where to look at?

Thanks,
Dorian