[mpich-discuss] mpich2-1.1.1p1/nemesis error

[Dave Goodell]

Bryan, thanks again for reporting this bug.  It should be fixed by  
r5374: https://trac.mcs.anl.gov/projects/mpich2/changeset/5374

If you want to cherry pick the change you will probably also need to  
grab r5372.  Alternatively you can play with one of the nightly  
tarballs (once tonight's script is finished running) if you don't want  
to deal with SVN: http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/nightly/trunk/

-Dave

On Sep 14, 2009, at 5:41 PM, Bryan Putnam wrote:

> Hi All,
>
> We've run across a piece of C code that appears to run successfully  
> with
> mpich, mvapich2-1.4rc2, mpich2-1.1.1p1 (built with ssm or sockets),  
> but
> fails with mpich2-1.1.1p1 (built with nemesis). The code is meant to  
> be
> run with two processors, and it either hangs or give an error  
> message in
> the case that the two processors are on separate nodes. It appears  
> to work
> OK if both processors are on the same node. I've included it here  
> for your
> enjoyment if you like to try it out.
>
> Thanks!
> Bryan
>
> =======================================================
>
>
>
>
>
> #include <stdio.h>
> #include <string.h>
> #include <mpi.h>
>
> double array[256][256][512];
>
> int main(int argc, char *argv[])
> {
>    int myrank;
>    MPI_Request req[2];
>    MPI_Status status1, status2, stats[2];
>    MPI_Datatype subarray1, subarray2, subarray3, subarray4;
>    int array_size[] = {256, 256, 512};
>    int array_subsize1[] = {128,128,512};
>    int array_subsize2[] = {128,128,512};
>
>    int array_start1[] = {0, 0, 0};
>        int array_start2[] = {128, 128, 0};
>    int i;
>        double *ptr = (double *)array;
>
>    MPI_Init(&argc, &argv);
>    MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
>
>        printf("MPI_Init okay...\n");
>
>        if(myrank == 0)
>        {
>                /* Create a subarray datatype */
>                MPI_Type_create_subarray(3, array_size, array_subsize1,
> array_start1, MPI_ORDER_C, MPI_DOUBLE, &subarray1);
>                MPI_Type_commit(&subarray1);
>                MPI_Type_create_subarray(3, array_size, array_subsize2,
> array_start2, MPI_ORDER_C, MPI_DOUBLE, &subarray2);
>                MPI_Type_commit(&subarray2);
>
>                for(i = 0; i < 256*256*512; i++)
>                        ptr[i] = 0.0;
>
>                printf("data init ok\n");
>                fflush(stdout);
>
>                MPI_Send(array, 1, subarray1, 1, 123, MPI_COMM_WORLD);
>
>                printf("sent ok\n");
>                fflush(stdout);
>
>                MPI_Recv(array, 1, subarray2, 1, 124, MPI_COMM_WORLD,
> &stats[1]);
>
>                printf("received ok\n");
>                printf("array: [0][0][0] = %f, [128][128][0] = %f\n",
> array[0][0][0], array[128][128][0]);
>                fflush(stdout);
>        }
>        else
>        {
>                /* Create a subarray datatype */
>                MPI_Type_create_subarray(3, array_size, array_subsize1,
> array_start1, MPI_ORDER_C, MPI_DOUBLE, &subarray3);
>                MPI_Type_commit(&subarray3);
>                MPI_Type_create_subarray(3, array_size, array_subsize2,
> array_start2, MPI_ORDER_C, MPI_DOUBLE, &subarray4);
>                MPI_Type_commit(&subarray4);
>
>                for(i = 0; i < 256*256*512; i++)
>                        ptr[i] = 1.0;
>
>                printf("data init ok\n");
>                fflush(stdout);
>
>                MPI_Recv(array, 1, subarray3, 0, 123, MPI_COMM_WORLD,
> &stats[0]);
>
>                printf("received ok\n");
>                printf("array: [0][0][0] = %f, [128][128][0] = %f\n",
> array[0][0][0], array[128][128][0]);
>                fflush(stdout);
>
>                MPI_Send(array, 1, subarray4, 0, 124, MPI_COMM_WORLD);
>
>                printf("sent ok\n");
>
>                fflush(stdout);
>        }
>
>    MPI_Finalize();
>    return 0;
> }