[mpich2-dev] [opa-core] Problem with MPICH2 1.1 during MPI_Init() on PPC Linux 32bit

[Brian Smith]

Joe is building on one of our front-ends because we are seeing some 
weirdness in datatype stuff inside MPIR_Gather. We wanted to see if it was 
having problems on multiple platforms.

But, yes, on BGP for the compute nodes it needs to be at least 8-byte 
aligned (and 16 might be better for performance), but you probably want to 
generalize the type sizing on other platforms.


Brian Smith
BlueGene MPI Development
IBM Rochester
Phone: 507 253 4717
smithbr at us.ibm.com




Darius Buntinas <buntinas at mcs.anl.gov> 
Sent by: opa-core-bounces at lists.mcs.anl.gov
06/09/2009 03:23 PM

To
Joe Ratterman <jratt0 at gmail.com>
cc
OpenPA Core <opa-core at lists.mcs.anl.gov>, MPICH2 Dev 
<mpich2-dev at mcs.anl.gov>
Subject
Re: [opa-core] Problem with MPICH2 1.1 during MPI_Init() on PPC Linux 
32bit






I see that there is a comment in opa_gcc_ppc.h that "these need to be
aligned on an 8-byte boundary to work on a BG/P."  Do you know if this
is true?

Dave, this looks like your comment.  Can you comment?

Should this be changed to "OPA_ATTRIBUTE((aligned (SIZEOF_VOID_P)))" to
handle different pointer sizes (similarly for SIZEOF_INT for ints)?

-d

On 06/09/2009 03:04 PM, Joe Ratterman wrote:
> This is the error I'm seeing:
> 
>     Assertion failed in file mpid_nem_init.c at line 89:
>     sizeof(MPID_nem_cell_rel_ptr_t) == 4
> 
> 
> 
> This appears to be the important code from the pre-processed version
> of mpid_nem_init.c
> 
> 
>     # 11
> 
"/bglhome/usr1/jratt/src-sles10/mpich2/mpich2-1.1/src/openpa/src/primitives/opa_gcc_ppc.h"
>     typedef struct { volatile int v __attribute__ ((aligned (8))); }
>     OPA_int_t;
>     typedef struct { void * volatile v __attribute__ ((aligned (8))); }
>     OPA_ptr_t;
> 
>     # 188 "../include/mpid_nem_datatypes.h"
>     typedef struct MPID_nem_cell_rel_ptr
>     {
>         OPA_ptr_t p;
>     }
>     MPID_nem_cell_rel_ptr_t;
> 
>     # 135 mpid/ch3/include/mpidi_ch3_conf.h
> 
>     #define SIZEOF_VOID_P 4
> 
>     # 85 mpid_nem_init.c
>         /* The MPID_nem_cell_rel_ptr_t defined in mpid_nem_datatypes.h
>          * should only contain a pointer. Also, we don't support 
compilers
>          * which add random padding to the start of the structure. The
>          * below assert should blow up if any of these happens. */
>         MPIU_Assert(sizeof(MPID_nem_cell_rel_ptr_t) == SIZEOF_VOID_P);
> 
> 
> 
> Given the above code, the size of that type appears to be 8 in both GCC
> and XLC.  I see that the alignment is hard-coded to 8 in
> src/openpa/src/primitives/opa_gcc_ppc.h.  Changing that first line to an
> alignment of 4, the type size shrinks to 4 (correspondingly, increasing
> it to 16 increases the type size as well).
> 
> Changing the file to 4 and recompiling things allows a few simple tests
> to work fine--I haven't tried too much more.
> 
> 
> 
> Thanks,
> Joe Ratterman
> jratt at us.ibm.com <mailto:jratt at us.ibm.com>
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> opa-core mailing list
> opa-core at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/opa-core
_______________________________________________
opa-core mailing list
opa-core at lists.mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/opa-core

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich2-dev/attachments/20090609/521d2dca/attachment-0001.htm>