I tried:<br><ol><li>mpiexec -n 2 ./nek5000</li><li>./nekbmpi blah 2</li></ol>and I get the same problem for both. There is no big deal if it doesn't work on parallel since it is on my laptop, but it would be cool if it worked :)<br>
I'll continue to look for the problem. It might perhaps come from my mpich implementation. I'll try to run some other parallel codes and get back to you after.<br><br><div class="gmail_quote">On 18 October 2011 19:22, <span dir="ltr"><<a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">How do you launch Nek in parallel? Can you try to run Nek using<br>
mpiexec or mpirun w/o calling a script.<br>
<div><div></div><div class="h5">-Stefan<br>
<br>
On 10/18/11, <a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a><br>
<<a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a>> wrote:<br>
> Here it is:<br>
><br>
> #!/bin/bash<br>
> # Nek5000 build config file<br>
> # (c) 2008,2009,2010 UCHICAGO ARGONNE, LLC<br>
><br>
> # source path<br>
> SOURCE_ROOT="$HOME/Codes/nek5_svn/trunk/nek"<br>
><br>
> # Fortran compiler<br>
> F77="mpif77"<br>
><br>
> # C compiler<br>
> CC="mpicc"<br>
><br>
> # pre-processor symbol list<br>
> # (set PPLIST=? to get a list of available symbols)<br>
> #PPLIST="?"<br>
><br>
> # plug-in list<br>
> PLUGIN_LIST=""<br>
><br>
><br>
> # OPTIONAL SETTINGS<br>
> # -----------------<br>
><br>
> # enable MPI (default true)<br>
> #IFMPI="false"<br>
><br>
> # auxilliary files to compile<br>
> # NOTE: source files have to located in the same directory as makenek<br>
> # a makefile_usr.inc has to be provided containing the build rules<br>
> #USR="foo.o"<br>
><br>
> # linking flags<br>
> #USR_LFLAGS="-L/usr/lib -lfoo"<br>
><br>
> # generic compiler flags<br>
> #G="-g"<br>
><br>
> # optimization flags<br>
> #OPT_FLAGS_STD=""<br>
> #OPT_FLAGS_MAG=""<br>
><br>
> # enable AMG coarse grid solver (default XXT)<br>
> #IFAMG="true"<br>
> #IFAMG_DUMP="true"<br>
><br>
> # CVODE path<br>
> #CVODE_DIR=$HOME/cvode/lib<br>
><br>
> # MOAB/iMESH path<br>
> #MOAB_DIR="$HOME/moab"<br>
><br>
><br>
> ###############################################################################<br>
> # DONT'T TOUCH WHAT FOLLOWS !!!<br>
> ###############################################################################<br>
> # assign version tag<br>
> mver=1<br>
> # overwrite source path with optional 2nd argument<br>
> if [ -d $2 ] && [ $# -eq 2 ]; then<br>
> SOURCE_ROOT="$2"<br>
> echo "change source code directory to: ", $SOURCE_ROOT<br>
> fi<br>
> # do some checks and create makefile<br>
> source $SOURCE_ROOT/makenek.inc<br>
> # compile<br>
> make -j4 -f makefile 2>&1 | tee compiler.out<br>
> exit 0<br>
><br>
><br>
> On 18 October 2011 19:13, <<a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a>> wrote:<br>
><br>
>> Please post your makenek.<br>
>> -Stefan<br>
>><br>
>> On 10/18/11, <a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a><br>
>> <<a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a>> wrote:<br>
>> > Hi,<br>
>> ><br>
>> > Sorry for this very late response, it completely got out of my mind.<br>
>> > Here<br>
>> > are the different outputs I get for these commands:<br>
>> ><br>
>> > jean-christophe@dyf-corto:~/Calculs/Cavite2D/BaseFlow$ ulimit -a<br>
>> > core file size (blocks, -c) 0<br>
>> > data seg size (kbytes, -d) unlimited<br>
>> > scheduling priority (-e) 0<br>
>> > file size (blocks, -f) unlimited<br>
>> > pending signals (-i) 31533<br>
>> > max locked memory (kbytes, -l) 64<br>
>> > max memory size (kbytes, -m) unlimited<br>
>> > open files (-n) 1024<br>
>> > pipe size (512 bytes, -p) 8<br>
>> > POSIX message queues (bytes, -q) 819200<br>
>> > real-time priority (-r) 0<br>
>> > stack size (kbytes, -s) 8192<br>
>> > cpu time (seconds, -t) unlimited<br>
>> > max user processes (-u) 31533<br>
>> > virtual memory (kbytes, -v) unlimited<br>
>> > file locks (-x) unlimited<br>
>> ><br>
>> > jean-christophe@dyf-corto:~/Calculs/Cavite2D/BaseFlow$ size nek5000<br>
>> > text data bss dec hex filename<br>
>> > 2232834 3484 185835044 188071362 b35bdc2 nek5000<br>
>> ><br>
>> > jean-christophe@dyf-corto:~/Calculs/Cavite2D/BaseFlow$ cat SIZE<br>
>> > C Dimension file to be included<br>
>> > C<br>
>> > C HCUBE array dimensions<br>
>> > C<br>
>> > parameter (ldim=2)<br>
>> > parameter (lx1=11,ly1=lx1,lz1=1,lelt=225,lelv=lelt)<br>
>> > parameter (lxd=17,lyd=lxd,lzd=1)<br>
>> > parameter (lelx=20,lely=60,lelz=1)<br>
>> > c<br>
>> > parameter (lzl=3 + 2*(ldim-3))<br>
>> > c<br>
>> > parameter (lx2=lx1-2)<br>
>> > parameter (ly2=ly1-2)<br>
>> > parameter (lz2=1)<br>
>> > parameter (lx3=lx2)<br>
>> > parameter (ly3=ly2)<br>
>> > parameter (lz3=lz2)<br>
>> > c<br>
>> > c parameter (lpelv=lelv,lpelt=lelt,lpert=3) ! perturbation<br>
>> > c parameter (lpx1=lx1,lpy1=ly1,lpz1=lz1) ! array sizes<br>
>> > c parameter (lpx2=lx2,lpy2=ly2,lpz2=lz2)<br>
>> > c<br>
>> > parameter (lpelv=1,lpelt=1,lpert=1) ! perturbation<br>
>> > parameter (lpx1=1,lpy1=1,lpz1=1) ! array sizes<br>
>> > parameter (lpx2=1,lpy2=1,lpz2=1)<br>
>> > c<br>
>> > c<br>
>> > c parameter (lbelv=lelv,lbelt=lelt) ! MHD<br>
>> > c parameter (lbx1=lx1,lby1=ly1,lbz1=lz1) ! array sizes<br>
>> > c parameter (lbx2=lx2,lby2=ly2,lbz2=lz2)<br>
>> > c<br>
>> > parameter (lbelv=1,lbelt=1) ! MHD<br>
>> > parameter (lbx1=1,lby1=1,lbz1=1) ! array sizes<br>
>> > parameter (lbx2=1,lby2=1,lbz2=1)<br>
>> > c<br>
>> > C LX1M=LX1 when there are moving meshes; =1 otherwise<br>
>> > parameter (lx1m=lx1,ly1m=ly1,lz1m=lz1)<br>
>> > parameter (ldimt= 4) ! 3 passive scalars + T<br>
>> > parameter (ldimt1=ldimt+1)<br>
>> > parameter (ldimt3=ldimt+3)<br>
>> > parameter (lp = 8)<br>
>> > parameter (lelg = 300)<br>
>> > c<br>
>> > c Note: In the new code, LELGEC should be about sqrt(LELG)<br>
>> > c<br>
>> > PARAMETER (LELGEC = 1)<br>
>> > PARAMETER (LXYZ2 = 1)<br>
>> > PARAMETER (LXZ21 = 1)<br>
>> > c<br>
>> > PARAMETER (LMAXV=LX1*LY1*LZ1*LELV)<br>
>> > PARAMETER (LMAXT=LX1*LY1*LZ1*LELT)<br>
>> > PARAMETER (LMAXP=LX2*LY2*LZ2*LELV)<br>
>> > PARAMETER (LXZ=LX1*LZ1)<br>
>> > PARAMETER (LORDER=3)<br>
>> > PARAMETER (MAXOBJ=4,MAXMBR=LELT*6,lhis=100)<br>
>> > C<br>
>> > C Common Block Dimensions<br>
>> > C<br>
>> > PARAMETER (LCTMP0 =2*LX1*LY1*LZ1*LELT)<br>
>> > PARAMETER (LCTMP1 =4*LX1*LY1*LZ1*LELT)<br>
>> > C<br>
>> > C The parameter LVEC controls whether an additional 42 field arrays<br>
>> > C are required for Steady State Solutions. If you are not using<br>
>> > C Steady State, it is recommended that LVEC=1.<br>
>> > C<br>
>> > PARAMETER (LVEC=1)<br>
>> > C<br>
>> > C Uzawa projection array dimensions<br>
>> > C<br>
>> > parameter (mxprev = 80)<br>
>> > parameter (lgmres = 40)<br>
>> > C<br>
>> > C Split projection array dimensions<br>
>> > C<br>
>> > parameter(lmvec = 1)<br>
>> > parameter(lsvec = 1)<br>
>> > parameter(lstore=lmvec*lsvec)<br>
>> > c<br>
>> > c NONCONFORMING STUFF<br>
>> > c<br>
>> > parameter (maxmor = lelt)<br>
>> > C<br>
>> > C Array dimensions<br>
>> > C<br>
>> > COMMON/DIMN/NELV,NELT,NX1,NY1,NZ1,NX2,NY2,NZ2<br>
>> > $,NX3,NY3,NZ3,NDIM,NFIELD,NPERT,NID<br>
>> > $,NXD,NYD,NZD<br>
>> ><br>
>> > c automatically added by makenek<br>
>> > parameter(lxo = lx1) ! max output grid size (lxo>=lx1)<br>
>> ><br>
>> > c automatically added by makenek<br>
>> > parameter(lpart = 1 ) ! max number of particles<br>
>> ><br>
>> > c automatically added by makenek<br>
>> > integer ax1,ay1,az1,ax2,ay2,az2<br>
>> > parameter (ax1=lx1,ay1=ly1,az1=lz1,ax2=lx2,ay2=ly2,az2=lz2) !<br>
>> running<br>
>> > averages<br>
>> ><br>
>> > c automatically added by makenek<br>
>> > parameter (lxs=1,lys=lxs,lzs=(lxs-1)*(ldim-2)+1) !New Pressure<br>
>> > Preconditioner<br>
>> ><br>
>> > c automatically added by makenek<br>
>> > parameter (lfdm=0) ! == 1 for fast diagonalization method<br>
>> ><br>
>> > Concerning Nek, yes I do get an output on the screen when lauching the<br>
>> > calculation. Attached is the logfile for one example (2D cavity, only<br>
>> with<br>
>> > 100 elements though but the problem is the same as I increase the number<br>
>> of<br>
>> > elements).<br>
>> ><br>
>> > Cheers,<br>
>> ><br>
>> > On 18 October 2011 18:03, Jean-Christophe Loiseau<br>
>> > <<a href="mailto:loiseau.jc@gmail.com">loiseau.jc@gmail.com</a>>wrote:<br>
>> ><br>
>> >> Hi,<br>
>> >><br>
>> >> Sorry for this very late response, it completely got out of my mind.<br>
>> Here<br>
>> >> are the different outputs I get for these commands:<br>
>> >><br>
>> >> jean-christophe@dyf-corto:~/Calculs/Cavite2D/BaseFlow$ ulimit -a<br>
>> >> core file size (blocks, -c) 0<br>
>> >> data seg size (kbytes, -d) unlimited<br>
>> >> scheduling priority (-e) 0<br>
>> >> file size (blocks, -f) unlimited<br>
>> >> pending signals (-i) 31533<br>
>> >> max locked memory (kbytes, -l) 64<br>
>> >> max memory size (kbytes, -m) unlimited<br>
>> >> open files (-n) 1024<br>
>> >> pipe size (512 bytes, -p) 8<br>
>> >> POSIX message queues (bytes, -q) 819200<br>
>> >> real-time priority (-r) 0<br>
>> >> stack size (kbytes, -s) 8192<br>
>> >> cpu time (seconds, -t) unlimited<br>
>> >> max user processes (-u) 31533<br>
>> >> virtual memory (kbytes, -v) unlimited<br>
>> >> file locks (-x) unlimited<br>
>> >><br>
>> >> jean-christophe@dyf-corto:~/Calculs/Cavite2D/BaseFlow$ size nek5000<br>
>> >> text data bss dec hex filename<br>
>> >> 2232834 3484 185835044 188071362 b35bdc2 nek5000<br>
>> >><br>
>> >> jean-christophe@dyf-corto:~/Calculs/Cavite2D/BaseFlow$ cat SIZE<br>
>> >> C Dimension file to be included<br>
>> >> C<br>
>> >> C HCUBE array dimensions<br>
>> >> C<br>
>> >> parameter (ldim=2)<br>
>> >> parameter (lx1=11,ly1=lx1,lz1=1,lelt=225,lelv=lelt)<br>
>> >> parameter (lxd=17,lyd=lxd,lzd=1)<br>
>> >> parameter (lelx=20,lely=60,lelz=1)<br>
>> >> c<br>
>> >> parameter (lzl=3 + 2*(ldim-3))<br>
>> >> c<br>
>> >> parameter (lx2=lx1-2)<br>
>> >> parameter (ly2=ly1-2)<br>
>> >> parameter (lz2=1)<br>
>> >> parameter (lx3=lx2)<br>
>> >> parameter (ly3=ly2)<br>
>> >> parameter (lz3=lz2)<br>
>> >> c<br>
>> >> c parameter (lpelv=lelv,lpelt=lelt,lpert=3) ! perturbation<br>
>> >> c parameter (lpx1=lx1,lpy1=ly1,lpz1=lz1) ! array sizes<br>
>> >> c parameter (lpx2=lx2,lpy2=ly2,lpz2=lz2)<br>
>> >> c<br>
>> >> parameter (lpelv=1,lpelt=1,lpert=1) ! perturbation<br>
>> >> parameter (lpx1=1,lpy1=1,lpz1=1) ! array sizes<br>
>> >> parameter (lpx2=1,lpy2=1,lpz2=1)<br>
>> >> c<br>
>> >> c<br>
>> >> c parameter (lbelv=lelv,lbelt=lelt) ! MHD<br>
>> >> c parameter (lbx1=lx1,lby1=ly1,lbz1=lz1) ! array sizes<br>
>> >> c parameter (lbx2=lx2,lby2=ly2,lbz2=lz2)<br>
>> >> c<br>
>> >> parameter (lbelv=1,lbelt=1) ! MHD<br>
>> >> parameter (lbx1=1,lby1=1,lbz1=1) ! array sizes<br>
>> >> parameter (lbx2=1,lby2=1,lbz2=1)<br>
>> >> c<br>
>> >> C LX1M=LX1 when there are moving meshes; =1 otherwise<br>
>> >> parameter (lx1m=lx1,ly1m=ly1,lz1m=lz1)<br>
>> >> parameter (ldimt= 4) ! 3 passive scalars +<br>
>> >> T<br>
>> >> parameter (ldimt1=ldimt+1)<br>
>> >> parameter (ldimt3=ldimt+3)<br>
>> >> parameter (lp = 8)<br>
>> >> parameter (lelg = 300)<br>
>> >> c<br>
>> >> c Note: In the new code, LELGEC should be about sqrt(LELG)<br>
>> >> c<br>
>> >> PARAMETER (LELGEC = 1)<br>
>> >> PARAMETER (LXYZ2 = 1)<br>
>> >> PARAMETER (LXZ21 = 1)<br>
>> >> c<br>
>> >> PARAMETER (LMAXV=LX1*LY1*LZ1*LELV)<br>
>> >> PARAMETER (LMAXT=LX1*LY1*LZ1*LELT)<br>
>> >> PARAMETER (LMAXP=LX2*LY2*LZ2*LELV)<br>
>> >> PARAMETER (LXZ=LX1*LZ1)<br>
>> >> PARAMETER (LORDER=3)<br>
>> >> PARAMETER (MAXOBJ=4,MAXMBR=LELT*6,lhis=100)<br>
>> >> C<br>
>> >> C Common Block Dimensions<br>
>> >> C<br>
>> >> PARAMETER (LCTMP0 =2*LX1*LY1*LZ1*LELT)<br>
>> >> PARAMETER (LCTMP1 =4*LX1*LY1*LZ1*LELT)<br>
>> >> C<br>
>> >> C The parameter LVEC controls whether an additional 42 field arrays<br>
>> >> C are required for Steady State Solutions. If you are not using<br>
>> >> C Steady State, it is recommended that LVEC=1.<br>
>> >> C<br>
>> >> PARAMETER (LVEC=1)<br>
>> >> C<br>
>> >> C Uzawa projection array dimensions<br>
>> >> C<br>
>> >> parameter (mxprev = 80)<br>
>> >> parameter (lgmres = 40)<br>
>> >> C<br>
>> >> C Split projection array dimensions<br>
>> >> C<br>
>> >> parameter(lmvec = 1)<br>
>> >> parameter(lsvec = 1)<br>
>> >> parameter(lstore=lmvec*lsvec)<br>
>> >> c<br>
>> >> c NONCONFORMING STUFF<br>
>> >> c<br>
>> >> parameter (maxmor = lelt)<br>
>> >> C<br>
>> >> C Array dimensions<br>
>> >> C<br>
>> >> COMMON/DIMN/NELV,NELT,NX1,NY1,NZ1,NX2,NY2,NZ2<br>
>> >> $,NX3,NY3,NZ3,NDIM,NFIELD,NPERT,NID<br>
>> >> $,NXD,NYD,NZD<br>
>> >><br>
>> >> c automatically added by makenek<br>
>> >> parameter(lxo = lx1) ! max output grid size (lxo>=lx1)<br>
>> >><br>
>> >> c automatically added by makenek<br>
>> >> parameter(lpart = 1 ) ! max number of particles<br>
>> >><br>
>> >> c automatically added by makenek<br>
>> >> integer ax1,ay1,az1,ax2,ay2,az2<br>
>> >> parameter (ax1=lx1,ay1=ly1,az1=lz1,ax2=lx2,ay2=ly2,az2=lz2) !<br>
>> >> running<br>
>> >> averages<br>
>> >><br>
>> >> c automatically added by makenek<br>
>> >> parameter (lxs=1,lys=lxs,lzs=(lxs-1)*(ldim-2)+1) !New Pressure<br>
>> >> Preconditioner<br>
>> >><br>
>> >> c automatically added by makenek<br>
>> >> parameter (lfdm=0) ! == 1 for fast diagonalization method<br>
>> >><br>
>> >> Concerning Nek, yes I do get an output on the screen when lauching the<br>
>> >> calculation. Attached is the logfile for one example (2D cavity, only<br>
>> with<br>
>> >> 100 elements though but the problem is the same as I increase the<br>
>> >> number<br>
>> >> of<br>
>> >> elements).<br>
>> >><br>
>> >> Cheers,<br>
>> >><br>
>> >><br>
>> >> On 27 September 2011 17:40, <<a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a>> wrote:<br>
>> >><br>
>> >>> Hi,<br>
>> >>><br>
>> >>> Do you get any output printed to the screen when you launch Nek?<br>
>> >>> Please provide the output of the following command:<br>
>> >>> ulimit -a (in case you are using bash)<br>
>> >>> size nek5000<br>
>> >>> cat SIZE<br>
>> >>><br>
>> >>> Can you post the command you are using to launch Nek in parallel?<br>
>> >>><br>
>> >>> Cheers,<br>
>> >>> Stefan<br>
>> >>><br>
>> >>><br>
>> >>><br>
>> >>> On 9/27/11, <a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a><br>
>> >>> <<a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a>> wrote:<br>
>> >>> > Hi Neks,<br>
>> >>> ><br>
>> >>> > I do get the same error when I try to run nek on my laptop. Did you<br>
>> >>> > find<br>
>> >>> any<br>
>> >>> > way to solve the problem John?<br>
>> >>> ><br>
>> >>> > Regards,<br>
>> >>> ><br>
>> >>> > On 11 May 2011 21:25, <<a href="mailto:nek5000-users@lists.mcs.anl.gov">nek5000-users@lists.mcs.anl.gov</a>> wrote:<br>
>> >>> ><br>
>> >>> >> Hi,<br>
>> >>> >><br>
>> >>> >> I have lp=512 in SIZE, and I did compile with IFMPI=true. I<br>
>> >>> >> recompiled<br>
>> >>> >> yesterday because I thought this may be the case, but I was still<br>
>> >>> getting<br>
>> >>> >> the same error. When I first start a simulation and run top, it<br>
>> shows<br>
>> >>> >> four<br>
>> >>> >> processors running nek5000, but this changes to one processor after<br>
>> a<br>
>> >>> few<br>
>> >>> >> seconds.<br>
>> >>> >><br>
>> >>> >> Thanks,<br>
>> >>> >><br>
>> >>> >> John<br>
>> >>> >><br>
>> >>> >> _______________________________________________<br>
>> >>> >> Nek5000-users mailing list<br>
>> >>> >> <a href="mailto:Nek5000-users@lists.mcs.anl.gov">Nek5000-users@lists.mcs.anl.gov</a><br>
>> >>> >> <a href="https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users</a><br>
>> >>> >><br>
>> >>> >><br>
>> >>> ><br>
>> >>> ><br>
>> >>> > --<br>
>> >>> > Jean-Christophe<br>
>> >>> ><br>
>> >>> _______________________________________________<br>
>> >>> Nek5000-users mailing list<br>
>> >>> <a href="mailto:Nek5000-users@lists.mcs.anl.gov">Nek5000-users@lists.mcs.anl.gov</a><br>
>> >>> <a href="https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users</a><br>
>> >>><br>
>> >><br>
>> >><br>
>> >><br>
>> >> --<br>
>> >> Jean-Christophe<br>
>> >><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> > Jean-Christophe<br>
>> ><br>
>> _______________________________________________<br>
>> Nek5000-users mailing list<br>
>> <a href="mailto:Nek5000-users@lists.mcs.anl.gov">Nek5000-users@lists.mcs.anl.gov</a><br>
>> <a href="https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users</a><br>
>><br>
><br>
><br>
><br>
> --<br>
> Jean-Christophe<br>
><br>
_______________________________________________<br>
Nek5000-users mailing list<br>
<a href="mailto:Nek5000-users@lists.mcs.anl.gov">Nek5000-users@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Jean-Christophe<br>