Thanks Jeff,<div><br></div><div>the code is quite large to post here. It is from graph500 benchmark's mpi implementation. multiple files are required to build this code. I can send the whole directory for debugging. what do you say?</div>
<div><br></div><div>Thanks,</div><div>Ziaul.<br><br><div class="gmail_quote">On Thu, May 31, 2012 at 8:51 PM, Jeff Hammond <span dir="ltr"><<a href="mailto:jhammond@alcf.anl.gov" target="_blank">jhammond@alcf.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">This is an interesting question.<br>
<br>
"Is it okay that I am freeing memory before the closing fence?"<br>
<br>
With respect to the memory associated with the MPI_Accumulate itself,<br>
the answer is clearly "no".<br>
<br>
MPI-2.2 11.3:<br>
<br>
"The local communication buffer of an RMA call should not be updated,<br>
and the local communication buffer of a get call should not be<br>
accessed after the RMA call, until the subsequent synchronization call<br>
completes."<br>
<br>
Freeing memory is obviously a modification.<br>
<br>
However, it seems you are freeing the MPI_Datatype used in this<br>
operation and the input arrays associated with it.<br>
<br>
>From MPI-2.2 4.1.9:<br>
<br>
"Marks the datatype object associated with datatype for deallocation<br>
and sets datatype to MPI_DATATYPE_NULL. Any communication that is<br>
currently using this datatype will complete normally. Freeing a<br>
datatype does not affect any other datatype that was built from the<br>
freed datatype. The system behaves as if input datatype arguments to<br>
derived datatype constructors are passed by value."<br>
<br>
It would seem that your usage is correct.<br>
<br>
I cannot debug your code because it is incomplete and has a few bugs<br>
according to the C standard. Can you post the code that actually<br>
results in the error you're seeing?<br>
<br>
There may be other issues with your code that others may point out.<br>
My email is not intended to capture all possible sources of error.<br>
<br>
Best,<br>
<br>
Jeff<br>
<div><div><br>
<br>
On Thu, May 31, 2012 at 6:23 PM, Ziaul Haque Olive <<a href="mailto:mzh.olive@gmail.com" target="_blank">mzh.olive@gmail.com</a>> wrote:<br>
> Hello,<br>
><br>
> I am not sure, if my code is correct according to MPICH2(v1.4.1p1.). the<br>
> code is given as follows, I am doing MPI one-sided communication inside a<br>
> function - data_transfer. this function is being called inside a fence<br>
> epoch. inside data_transfer, I am allocating memory for non-contiguous data,<br>
> creating derived data type, using this datatype in MPI_Accumulate, and after<br>
> calling MPI_Accumulate, freeing the indexed data type and also freeing the<br>
> memory containing indices for indexed data type. is it okay that I am<br>
> freeing memory before the closing fence.<br>
><br>
> void data_transfer(void *data, int *sources_disp, int *targets_disp, int<br>
> *target, int size, int *blength, int func, MPI_Op op, MPI_Win win,<br>
> MPI_Datatype dtype){<br>
><br>
> int i,j, index;<br>
> int tmp_target;<br>
> int *flag;<br>
> int *source_disp;<br>
> int *target_disp;<br>
> MPI_Datatype source_type, target_type;<br>
> MPI_Alloc_mem( size*sizeof(int), MPI_INFO_NULL, &source_disp);<br>
> MPI_Alloc_mem( size*sizeof(int), MPI_INFO_NULL, &target_disp);<br>
> MPI_Alloc_mem( size*sizeof(int), MPI_INFO_NULL, &flag );<br>
><br>
> memset(flag, 0, size*sizeof(int));<br>
><br>
> for(i=0;i<size;i++){<br>
> if(flag[i]==0){<br>
> tmp_target = target[i];<br>
> index = 0;<br>
> for(j=i; j<size; j++){<br>
> if(flag[j]==0 && tmp_target == target[j] ){<br>
> source_disp[index] = sources_disp[j];<br>
> target_disp[index] = targets_disp[j];<br>
> //printf("src, target disp %d %d\n", j, disp[j]);<br>
> index++;<br>
> flag[j] = 1;<br>
> }<br>
> }<br>
><br>
> MPI_Type_indexed(index, blength , source_disp, dtype, &source_type);<br>
> MPI_Type_commit(&source_type);<br>
> MPI_Type_indexed(index, blength , target_disp, dtype, &target_type);<br>
> MPI_Type_commit(&target_type);<br>
> MPI_Accumulate( data, 1, source_type, tmp_target, 0, 1, target_type , op,<br>
> win);<br>
><br>
> MPI_Type_free(&source_type);<br>
> MPI_Type_free(&target_type);<br>
> }<br>
> }<br>
> MPI_Free_mem(source_disp);<br>
> MPI_Free_mem(target_disp);<br>
> MPI_Free_mem(flag);<br>
><br>
> }<br>
><br>
> void main(){<br>
> int i;<br>
> while(i<N){<br>
> MPI_Win_fence(MPI_MODE_NOPRECEDE, queue2_win);<br>
><br>
> data_transfer();<br>
><br>
> MPI_Win_fence(MPI_MODE_NOSUCCEED, queue2_win);<br>
> }<br>
> }<br>
><br>
> Thanks<br>
> Ziaul<br>
><br>
</div></div>> _______________________________________________<br>
> mpich-discuss mailing list <a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
> To manage subscription options or unsubscribe:<br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
><br>
<span><font color="#888888"><br>
<br>
<br>
--<br>
Jeff Hammond<br>
Argonne Leadership Computing Facility<br>
University of Chicago Computation Institute<br>
<a href="mailto:jhammond@alcf.anl.gov" target="_blank">jhammond@alcf.anl.gov</a> / (630) 252-5381<br>
<a href="http://www.linkedin.com/in/jeffhammond" target="_blank">http://www.linkedin.com/in/jeffhammond</a><br>
<a href="https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond" target="_blank">https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond</a><br>
_______________________________________________<br>
mpich-discuss mailing list <a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
To manage subscription options or unsubscribe:<br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
</font></span></blockquote></div><br></div>