<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.5730.11" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=387045518-24012007><FONT face=Arial
color=#0000ff size=2>You should see a difference with bigger files, larger
number of processes, and fast file systems (not NFS).</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=387045518-24012007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=387045518-24012007><FONT face=Arial
color=#0000ff size=2>Rajeev</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Luiz Mendes
[mailto:luizmendesw@gmail.com] <BR><B>Sent:</B> Tuesday, January 23, 2007 6:37
PM<BR><B>To:</B> Rajeev Thakur<BR><B>Cc:</B> mpich-discuss<BR><B>Subject:</B>
Re: [MPICH] Re: Problems with allreduce<BR></FONT><BR></DIV>
<DIV></DIV>Hi all, Rajeev<BR><BR>Well it is still early in Argonne now, here
is 10pm lol<BR><BR>I was testing some reads, and i see that Collective
read_all perform slower times to complete than read only operation. Is this
correct? or it would be notable only in great files? <BR><BR>
<DIV id=result_box dir=ltr>Below, it follows the code:</DIV><BR><SPAN
style="FONT-STYLE: italic">#include <stdio.h></SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">#include
<stdlib.h ></SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">#include <unistd.h></SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">#include
<string.h></SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">#include <mpi.h></SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">#include
<string.h></SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">#define FILESIZE 1048576</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">#define
INTS_PER_BLK 16</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">int main(int argc, char **argv) </SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">{</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
int *buf, rank, nprocs, nints, bufsize;</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
double tempo_total, tempo1, tempo2, tempo;</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
MPI_File fh;</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_Datatype filetype;</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
MPI_Init(&argc,&argv);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_Comm_rank(MPI_COMM_WORLD,
&rank); </SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_Comm_size(MPI_COMM_WORLD,
&nprocs);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> bufsize = FILESIZE/nprocs;
</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> buf = (int *)
malloc(bufsize);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> nints =
bufsize/sizeof(int);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_File_open(MPI_COMM_WORLD,
"datafile", MPI_MODE_RDONLY,</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_INFO_NULL,
&fh);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Type_vector(nints/INTS_PER_BLK, INTS_PER_BLK,INTS_PER_BLK*nprocs, MPI_INT,
&filetype);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_Type_commit(&filetype);
</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_File_set_view(fh,
INTS_PER_BLK*sizeof(int)*rank, MPI_INT,</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
filetype, "native", MPI_INFO_NULL); </SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
tempo1=MPI_Wtime();</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_File_read_all(fh, buf,
nints, MPI_INT, MPI_STATUS_IGNORE); </SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
tempo2=MPI_Wtime();</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> tempo=tempo2-tempo1;</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> printf("\nPrinting Time of
current process--> foi %.10f",tempo);</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
MPI_Allreduce(&tempo, &tempo_total, 1, MPI_DOUBLE,
MPI_MAX,MPI_COMM_WORLD); </SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> if(rank==0)</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
{</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> printf("\nThe
longest time is %.10f\n",tempo_total);</SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
}</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_File_close(&fh);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Type_free(&filetype);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> free(buf); </SPAN><BR
style="FONT-STYLE: italic"><SPAN style="FONT-STYLE: italic">
MPI_Finalize();</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> return 0;</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">}</SPAN><BR><BR><BR>Thanks.<BR>Luiz.<BR><BR><BR><BR><BR><BR><BR>
<DIV><SPAN class=gmail_quote>2007/1/23, Rajeev Thakur <<A
href="mailto:thakur@mcs.anl.gov">thakur@mcs.anl.gov</A>>:</SPAN>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff size=2>Unless
dimensao is 1, the Allreduce won't work because the input and output buffers
are single element doubles (not arrays).</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2>Rajeev</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: rgb(0,0,255) 2px solid; MARGIN-RIGHT: 0px">
<DIV lang=en-us dir=ltr align=left>
<HR>
<FONT face=Tahoma size=2><B>From:</B> Luiz Mendes [mailto:<A
onclick="return top.js.OpenExtLink(window,event,this)"
href="mailto:luizmendesw@gmail.com"
target=_blank>luizmendesw@gmail.com</A>] <BR><B>Sent:</B> Tuesday, January
23, 2007 6:10 PM<BR><B>To:</B> mpich-discuss<BR><B>Cc:</B> Rajeev
Thakur<BR><B>Subject:</B> Re: [MPICH] Re: Problems with
allreduce<BR></FONT><BR></DIV>
<DIV><SPAN class=e id=q_110517b06b2c2833_1>
<DIV></DIV>Hi all, Rajeev<BR><BR>I correct this, and i made some
simplifications, and the problems appears to be caused by the fact of i
add double variable declarations.<BR><BR>It is very strange. Some light to
this?<BR><BR>Thanks<BR>Luiz<BR><BR>
<DIV><SPAN class=gmail_quote>2007/1/23, Rajeev Thakur <<A
onclick="return top.js.OpenExtLink(window,event,this)"
href="mailto:thakur@mcs.anl.gov"
target=_blank>thakur@mcs.anl.gov</A>>:</SPAN>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2>Where have you allocated space for
arquivo_nome?</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN><FONT face=Arial color=#0000ff
size=2>Rajeev</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: rgb(0,0,255) 2px solid; MARGIN-RIGHT: 0px">
<DIV lang=en-us dir=ltr align=left>
<HR>
<FONT face=Tahoma size=2><B>From:</B> <A
onclick="return top.js.OpenExtLink(window,event,this)"
href="mailto:owner-mpich-discuss@mcs.anl.gov"
target=_blank>owner-mpich-discuss@mcs.anl.gov</A> [mailto:<A
onclick="return top.js.OpenExtLink(window,event,this)"
href="mailto:owner-mpich-discuss@mcs.anl.gov" target=_blank>
owner-mpich-discuss@mcs.anl.gov</A>] <B>On Behalf Of </B>Luiz
Mendes<BR><B>Sent:</B> Tuesday, January 23, 2007 5:05 PM<BR><B>To:</B>
mpich-discuss<BR><B>Subject:</B> [MPICH] Re: Problems with
allreduce<BR></FONT><BR></DIV>
<DIV><SPAN>
<DIV></DIV>Discard variable "tempo", without this continue getting
error.<BR><BR>thanks<BR><BR>
<DIV><SPAN class=gmail_quote>2007/1/23, Luiz Mendes <<A
onclick="return top.js.OpenExtLink(window,event,this)"
href="mailto:luizmendesw@gmail.com"
target=_blank>luizmendesw@gmail.com</A>>:</SPAN>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">Hi
all, i am with a problem in operation MPI_Allreduce. <BR><BR>The
code is the following below: <BR><BR><SPAN
style="FONT-STYLE: italic">int main(int argc, char
*argv[])</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">{</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> int opcao,
dimensao,nome_tamanho, processo, num_procs;</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> int coluna,
contador_bytes, contador_bytes_parcial, interacao,
contador_total_bytes; </SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> char *palavra =
NULL, *arquivo_nome;</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
char *arquivo_pv;</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> char
procname[MPI_MAX_PROCESSOR_NAME];</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> double inicio,
termino, tempo_parcial, soma_tempo_total, valor;</SPAN> <BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> long long
offset;</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_Datatype
linhaMatriz;</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_File
arquivo;</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_Status
status;</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> MPI_Init(&argc,
&argv);</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Comm_rank(MPI_COMM_WORLD, &processo); </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Comm_size(MPI_COMM_WORLD, &num_procs);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Get_processor_name(procname,&nome_tamanho); </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> while ((opcao =
getopt(argc, argv, "A:D:P:h")) != EOF)</SPAN> <BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> {</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
switch(opcao) </SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
{</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
case 'A': </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
arquivo_pv =optarg;
</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
sprintf(arquivo_nome,"%s",arquivo_pv);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
break; </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
case 'D': </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
sscanf (optarg, "%d",
&dimensao); </SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
break;</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
case 'P':</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
sprintf(arquivo_nome,"pvfs2:/mnt/pvfs2/%s",arquivo_pv);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
}
</SPAN> <BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> }</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> </SPAN><BR
style="FONT-STYLE: italic"><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Type_contiguous(dimensao,MPI_DOUBLE,&linhaMatriz);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Type_commit(&linhaMatriz); </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_File_open(MPI_COMM_WORLD, arquivo_nome, MPI_MODE_CREATE |
MPI_MODE_WRONLY, MPI_INFO_NULL, &arquivo);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> for(interacao=0;
interacao<(dimensao/num_procs);interacao++)</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> {</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
double matrizlocal[dimensao];</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
valor=processo*(dimensao/num_procs)*dimensao+interacao*dimensao;
</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
offset=valor*sizeof(double);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
for(coluna=0; coluna<dimensao; coluna++) </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
{</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
matrizlocal[coluna]=valor+coluna;</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
}</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_File_set_view(arquivo, offset, MPI_DOUBLE, linhaMatriz,
"native", MPI_INFO_NULL); </SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
inicio=MPI_Wtime();</SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_File_write(arquivo, matrizlocal, 1, linhaMatriz,
&status); </SPAN><BR style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
termino=MPI_Wtime();</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
tempo_parcial=termino-inicio;</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
contador_bytes_parcial+=sizeof(matrizlocal);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
soma_tempo_total+=tempo_parcial;</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic"> }</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_File_close(&arquivo);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-WEIGHT: bold; FONT-STYLE: italic">
MPI_Allreduce(&soma_tempo_total, &tempo, dimensao,
MPI_DOUBLE, MPI_MAX,MPI_COMM_WORLD);</SPAN><BR
style="FONT-STYLE: italic"><SPAN
style="FONT-STYLE: italic">
MPI_Finalize();
<BR>.......<BR><BR></SPAN><BR>Do you know what could be
happening?<BR><BR>Thanks<BR><SPAN><BR>Luiz
Mendes<BR></SPAN></BLOCKQUOTE></DIV><BR></SPAN></DIV></BLOCKQUOTE></DIV></BLOCKQUOTE></DIV><BR></SPAN></DIV></BLOCKQUOTE></DIV></BLOCKQUOTE></DIV><BR></BLOCKQUOTE></BODY></HTML>