<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v =
"urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word"><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>Are there limits to MPICH2 Jobs?</TITLE>
<META content="MSHTML 6.00.2900.3314" name=GENERATOR><!--[if !mso]>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
font-family: Tahoma;
}
@page Section1 {size: 8.5in 11.0in; margin: .3in .3in .3in .3in; }
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline
}
P {
FONT-SIZE: 12pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; FONT-FAMILY: "Times New Roman"; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto
}
SPAN.EmailStyle18 {
FONT-WEIGHT: normal; COLOR: navy; FONT-STYLE: normal; FONT-FAMILY: Arial; TEXT-DECORATION: none; mso-style-type: personal-reply
}
DIV.Section1 {
page: Section1
}
</STYLE>
</HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=567491914-08052008>Thank
you Lester and Rajeev for your feedback.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=567491914-08052008></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=567491914-08052008>We
change the stack size to unlimited. We were able to run a 4.4 GB
job. However, when we tried to run a 6.7 GB job we got the following
error:</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=567491914-08052008></SPAN></FONT><FONT color=#0000ff><SPAN
class=567491914-08052008><FONT size=2></FONT> </DIV>
<DIV><FONT face=Arial color=#000000 size=2>master sending static
commons...</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>master sending dynamic
commons...</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1917782060)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1891740044)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1871790948)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1871396372)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1871396340)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1858966996)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1777645156)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1777447868)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1777250572)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[pe:0] **DOTCOMM Error**
DOTCOMMI_PACK (dotcommi_pack.c:104[!(( dotcommp_sbuf.data != ((void *)0) ))])
(alloc(dotcommp_sbuf.data)) (1777250568)</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>[cli_0]: aborting job:</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>Fatal error in MPI_Bcast: Invalid
buffer pointer, error stack:</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>MPI_Bcast(784): MPI_Bcast(buf=(nil),
count=1855241236, MPI_BYTE, root=0, comm=0x84000000) failed</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>MPI_Bcast(735): Null buffer
pointer</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>Terminated</FONT></DIV>
<DIV><FONT face=Arial color=#000000 size=2>Signal Received, exiting
...</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial></FONT> </DIV>
<DIV><FONT face=Arial><SPAN class=567491914-08052008><FONT size=2>Are
there parameters in the environment that we need to change o ther than
the stack limit?</FONT></SPAN></FONT></DIV>
<DIV><FONT face=Arial><SPAN class=567491914-08052008></SPAN></FONT> </DIV>
<DIV><FONT><SPAN class=567491914-08052008></SPAN><FONT face=Arial><SPAN
class=567491914-08052008><FONT size=2>Your feedback
is appreciated.</FONT></SPAN></FONT></FONT></DIV>
<DIV><FONT><FONT face=Arial><SPAN
class=567491914-08052008></SPAN></FONT></FONT> </DIV>
<DIV><FONT><FONT><SPAN class=567491914-08052008></SPAN><FONT face=Arial><SPAN
class=567491914-08052008><FONT
size=2>Ibrahim</FONT></SPAN> </FONT></FONT></FONT></SPAN></FONT></DIV>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader dir=ltr align=left><FONT face=Tahoma
size=2>-----Original Message-----<BR><B>From:</B>
owner-mpich-discuss@mcs.anl.gov [mailto:owner-mpich-discuss@mcs.anl.gov]<B>On
Behalf Of </B>Rajeev Thakur<BR><B>Sent:</B> May 3, 2008 8:49 PM<BR><B>To:</B>
mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> RE: [mpich-discuss] Are there
limits to MPICH2 Jobs?<BR><BR></FONT></DIV>
<DIV dir=ltr align=left><SPAN class=156304700-04052008><FONT face=Arial
color=#0000ff size=2>Nothing in MPICH2 should prevent the job from running on
> 2GB memory. The problem is likely to be in the environment running
MPICH2, such as the stack size limit Lester mentions.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=156304700-04052008><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=156304700-04052008><FONT face=Arial
color=#0000ff size=2>Rajeev</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=156304700-04052008><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> owner-mpich-discuss@mcs.anl.gov
[mailto:owner-mpich-discuss@mcs.anl.gov] <B>On Behalf Of </B>Cavey,
Lester<BR><B>Sent:</B> Saturday, May 03, 2008 3:33 PM<BR><B>To:</B>
mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> RE: [mpich-discuss] Are there
limits to MPICH2 Jobs?<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Ibrahim,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">I'm not an expert
on this, but we used the ulimit command, in our bash rc files, to increase
our stack size limit. Then our 32-bit linux system was able iterate
through more loops while running our application (a particle-in-cell
simulator). Eventually though there was a limit to the number of loops
that ran, before a "segmentation fault" was reported. In our case,
this meant we had a stack overflow, since there is always going to be a
maximum stack size that can not be exceeded.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">I'm sure the ANL
guys can give you more advice, besides just trying the ulimit
command.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial color=navy size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<DIV>
<P class=MsoNormal><FONT face=Arial color=blue size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: Arial">Lester</SPAN></FONT><o:p></o:p></P></DIV>
<DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center><FONT
face="Times New Roman" size=3><SPAN style="FONT-SIZE: 12pt">
<HR tabIndex=-1 align=center width="100%" SIZE=2>
</SPAN></FONT></DIV>
<P class=MsoNormal><B><FONT face=Tahoma size=2><SPAN
style="FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">From:</SPAN></FONT></B><FONT
face=Tahoma size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Tahoma">
owner-mpich-discuss@mcs.anl.gov [mailto:owner-mpich-discuss@mcs.anl.gov]
<B><SPAN style="FONT-WEIGHT: bold">On Behalf Of </SPAN></B>Attieh,
Ibrahim<BR><B><SPAN style="FONT-WEIGHT: bold">Sent:</SPAN></B> Saturday, May
03, 2008 12:03 PM<BR><B><SPAN style="FONT-WEIGHT: bold">To:</SPAN></B>
mpich-discuss@mcs.anl.gov<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> [mpich-discuss] Are there
limits to MPICH2 Jobs?</SPAN></FONT><o:p></o:p></P></DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<P><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Hello,</SPAN></FONT>
<o:p></o:p></P>
<P><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">We are trying to run the Monte
Carlo code MCNP with MPICH2. However, we are having problems with jobs
bigger than 2 gigabytes. The jobs we want to run are as big as 9
gigabytes. We are running the jobs on a linux system with 64-bit
operating system.</SPAN></FONT><o:p></o:p></P>
<P><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Are there any limits to the size
of the files that MPICH2 could handle? Are there any options that we
need to set for MPICH2 to handle such large
jobs?</SPAN></FONT><o:p></o:p></P>
<P><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Your help would be greatly
appreciated.</SPAN></FONT> <o:p></o:p></P>
<P><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Thanks,</SPAN></FONT> <BR><FONT
face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Ibrahim
Attieh</SPAN></FONT> <o:p></o:p></P>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><FONT face="Times New Roman"
size=3><SPAN style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P>
<TABLE class=MsoNormalTable style="BACKGROUND: white" cellPadding=0
bgColor=white border=0>
<TBODY>
<TR>
<TD
style="PADDING-RIGHT: 0.75pt; PADDING-LEFT: 0.75pt; PADDING-BOTTOM: 0.75pt; PADDING-TOP: 0.75pt">
<P class=MsoNormal><FONT face="Times New Roman" color=black
size=3><SPAN
style="FONT-SIZE: 12pt; COLOR: black"><BR>CONFIDENTIAL AND PRIVILEGED INFORMATION NOTICE<BR><BR>This e-mail, and any attachments, may contain information that<BR>is confidential, subject to copyright, or exempt from disclosure.<BR>Any unauthorized review, disclosure, retransmission, <BR>dissemination or other use of or reliance on this information <BR>may be unlawful and is strictly prohibited. <BR><BR>AVIS D'INFORMATION CONFIDENTIELLE ET PRIVILÉGIÉE<BR><BR>Le présent courriel, et toute pièce jointe, peut contenir de <BR>l'information qui est confidentielle, régie par les droits <BR>d'auteur, ou interdite de divulgation. Tout examen, <BR>divulgation, retransmission, diffusion ou autres utilisations <BR>non autorisées de l'information ou dépendance non autorisée <BR>envers celle-ci peut être illégale et est strictement interdite.<o:p></o:p></SPAN></FONT></P></TD></TR></TBODY></TABLE>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P></DIV></BLOCKQUOTE></BLOCKQUOTE><br><br><table bgcolor=white style="color:black"><tr><td><br>CONFIDENTIAL AND PRIVILEGED INFORMATION NOTICE<br>
<br>
This e-mail, and any attachments, may contain information that<br>
is confidential, subject to copyright, or exempt from disclosure.<br>
Any unauthorized review, disclosure, retransmission, <br>
dissemination or other use of or reliance on this information <br>
may be unlawful and is strictly prohibited. <br>
<br>
AVIS D'INFORMATION CONFIDENTIELLE ET PRIVILÉGIÉE<br>
<br>
Le présent courriel, et toute pièce jointe, peut contenir de <br>
l'information qui est confidentielle, régie par les droits <br>
d'auteur, ou interdite de divulgation. Tout examen, <br>
divulgation, retransmission, diffusion ou autres utilisations <br>
non autorisées de l'information ou dépendance non autorisée <br>
envers celle-ci peut être illégale et est strictement interdite.</td></tr></table></BODY></HTML>