<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7036.0">
<TITLE>RE: [mpich-discuss] [mpich2-maint] #428: MPI_Win_fencememoryconsumption</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>Hi,<BR>
<BR>
>> But the memory layout of an array of double3 and...<BR>
Any *tricks* like the one you tried would make your code non-portable.<BR>
<BR>
>> Is MPI_Type_create_struct able to infer from the input data, whether the struct is a contiuguous block ...<BR>
Looks like MPICH2 does. However I don't think the standard says anything about this (Don’t make any assumptions which are not in the std)...<BR>
<BR>
>> ...I'm just sending my array as a MPI_DOUBLE array...<BR>
I think this is the *right* approach in your case.<BR>
<BR>
Regards,<BR>
Jayesh<BR>
<BR>
-----Original Message-----<BR>
From: mpich-discuss-bounces@mcs.anl.gov [<A HREF="mailto:mpich-discuss-bounces@mcs.anl.gov">mailto:mpich-discuss-bounces@mcs.anl.gov</A>] On Behalf Of Dorian Krause<BR>
Sent: Wednesday, March 04, 2009 2:52 PM<BR>
To: mpich-discuss@mcs.anl.gov<BR>
Cc: mpich2-maint@mcs.anl.gov<BR>
Subject: Re: [mpich-discuss] [mpich2-maint] #428: MPI_Win_fencememoryconsumption<BR>
<BR>
Hi<BR>
<BR>
(sorry for sending the message twice ...)<BR>
<BR>
Jayesh Krishna wrote:<BR>
> Hi,<BR>
> A contiguous MPI derived type consisting of 3 MPI_DOUBLEs is not<BR>
> equivalent to a C structure with 3 doubles. Try using an array of 3<BR>
> doubles (double[3])<BR>
<BR>
But the memory layout of an array of double3 and an array of double[3] is the same since all data is aligned to an 8 byte boundary, so no padding is necessary (I checked it).<BR>
But in the end, MPI gets a void* passed so the actual data layout in the buffer is irrelevant since it just greps what it thinks is the correct data (if it isn't (e.g., for padding reasons), this is the users fault). It this is the problem I would expect the problems to be on the origin side, not on the target side?!<BR>
<BR>
> or using an equivalent MPI datatype (eg:<BR>
> MPI_Type_create_struct()).<BR>
> <BR>
<BR>
With MPI_Type_create_struct I don't observe the mentioned problems.<BR>
<BR>
Is MPI_Type_create_struct able to infer from the input data, whether the struct is a contiuguous block or data? I read that strided data still yields a significant drop down of performance so I this wouldn't be a good alternative if the overhead is too large.<BR>
<BR>
At the moment I'm just sending my array as a MPI_DOUBLE array of 3x the size. This works for the moment ...<BR>
<BR>
Thanks + Regards,<BR>
Dorian<BR>
<BR>
> Let us know if it works for you.<BR>
><BR>
> Regards,<BR>
> Jayesh<BR>
><BR>
> -----Original Message-----<BR>
> From: mpich-discuss-bounces@mcs.anl.gov<BR>
> [<A HREF="mailto:mpich-discuss-bounces@mcs.anl.gov">mailto:mpich-discuss-bounces@mcs.anl.gov</A>] On Behalf Of Rajeev Thakur<BR>
> Sent: Friday, February 27, 2009 9:07 PM<BR>
> To: mpich-discuss@mcs.anl.gov<BR>
> Subject: Re: [mpich-discuss] MPI_Win_fence memory consumption<BR>
><BR>
> OK, thanks. We will look into it.<BR>
><BR>
> Rajeev<BR>
> <BR>
>> -----Original Message-----<BR>
>> From: mpich-discuss-bounces@mcs.anl.gov<BR>
>> [<A HREF="mailto:mpich-discuss-bounces@mcs.anl.gov">mailto:mpich-discuss-bounces@mcs.anl.gov</A>] On Behalf Of Dorian Krause<BR>
>> Sent: Friday, February 27, 2009 7:44 PM<BR>
>> To: mpich-discuss@mcs.anl.gov<BR>
>> Subject: Re: [mpich-discuss] MPI_Win_fence memory consumption<BR>
>><BR>
>> Rajeev Thakur wrote:<BR>
>> <BR>
>>> Does that happen only with Nemesis or even with ch3:sock?<BR>
>>> <BR>
>> The behaviour is the same with the configure flag<BR>
>> --with-device=ch3:sock.<BR>
>><BR>
>> Dorian<BR>
>><BR>
>> <BR>
>>> Rajeev<BR>
>>><BR>
>>> <BR>
>>>> -----Original Message-----<BR>
>>>> From: mpich-discuss-bounces@mcs.anl.gov<BR>
>>>> [<A HREF="mailto:mpich-discuss-bounces@mcs.anl.gov">mailto:mpich-discuss-bounces@mcs.anl.gov</A>] On Behalf Of<BR>
>>>> <BR>
>> Dorian Krause<BR>
>> <BR>
>>>> Sent: Friday, February 27, 2009 7:16 AM<BR>
>>>> To: mpich-discuss@mcs.anl.gov<BR>
>>>> Subject: [mpich-discuss] MPI_Win_fence memory consumption<BR>
>>>><BR>
>>>> Hi List,<BR>
>>>><BR>
>>>> the attached test program uses<BR>
>>>> <BR>
>> MPI_Accumulate/MPI_Win_fence for one<BR>
>> <BR>
>>>> sided communication with derived datatype.<BR>
>>>> The program runs fine with mpich2-1.1a2 except for my debugging<BR>
>>>> version of MPICH2 compiled with<BR>
>>>><BR>
>>>> ./configure --with-device=ch3:nemesis --enable-g=dbg,mem,meminit<BR>
>>>><BR>
>>>> In this case the MPI_Win_fence on the target side comuses<BR>
>>>> <BR>
>> about 90%<BR>
>> <BR>
>>>> of main memory (e.g. > 3 GB). As the behaviour is completely<BR>
>>>> different for predefined datatypes, I suspect that the memory<BR>
>>>> consumption is related to the construction of the derived<BR>
>>>> <BR>
>> datatype on<BR>
>> <BR>
>>>> the target side.<BR>
>>>><BR>
>>>> Is there a workaround for this?<BR>
>>>><BR>
>>>> Thanks + Best regards,<BR>
>>>> Dorian<BR>
>>>><BR>
>>>><BR>
>>>><BR>
>>>><BR>
>>>><BR>
>>>> <BR>
>> <BR>
><BR>
><BR>
> <BR>
<BR>
</FONT>
</P>
</BODY>
</HTML>