[Nek5000-users] Running Simulations in Parallel with nekbmpi
nek5000-users at lists.mcs.anl.gov
nek5000-users at lists.mcs.anl.gov
Tue Oct 18 12:17:48 CDT 2011
Here it is:
#!/bin/bash
# Nek5000 build config file
# (c) 2008,2009,2010 UCHICAGO ARGONNE, LLC
# source path
SOURCE_ROOT="$HOME/Codes/nek5_svn/trunk/nek"
# Fortran compiler
F77="mpif77"
# C compiler
CC="mpicc"
# pre-processor symbol list
# (set PPLIST=? to get a list of available symbols)
#PPLIST="?"
# plug-in list
PLUGIN_LIST=""
# OPTIONAL SETTINGS
# -----------------
# enable MPI (default true)
#IFMPI="false"
# auxilliary files to compile
# NOTE: source files have to located in the same directory as makenek
# a makefile_usr.inc has to be provided containing the build rules
#USR="foo.o"
# linking flags
#USR_LFLAGS="-L/usr/lib -lfoo"
# generic compiler flags
#G="-g"
# optimization flags
#OPT_FLAGS_STD=""
#OPT_FLAGS_MAG=""
# enable AMG coarse grid solver (default XXT)
#IFAMG="true"
#IFAMG_DUMP="true"
# CVODE path
#CVODE_DIR=$HOME/cvode/lib
# MOAB/iMESH path
#MOAB_DIR="$HOME/moab"
###############################################################################
# DONT'T TOUCH WHAT FOLLOWS !!!
###############################################################################
# assign version tag
mver=1
# overwrite source path with optional 2nd argument
if [ -d $2 ] && [ $# -eq 2 ]; then
SOURCE_ROOT="$2"
echo "change source code directory to: ", $SOURCE_ROOT
fi
# do some checks and create makefile
source $SOURCE_ROOT/makenek.inc
# compile
make -j4 -f makefile 2>&1 | tee compiler.out
exit 0
On 18 October 2011 19:13, <nek5000-users at lists.mcs.anl.gov> wrote:
> Please post your makenek.
> -Stefan
>
> On 10/18/11, nek5000-users at lists.mcs.anl.gov
> <nek5000-users at lists.mcs.anl.gov> wrote:
> > Hi,
> >
> > Sorry for this very late response, it completely got out of my mind. Here
> > are the different outputs I get for these commands:
> >
> > jean-christophe at dyf-corto:~/Calculs/Cavite2D/BaseFlow$ ulimit -a
> > core file size (blocks, -c) 0
> > data seg size (kbytes, -d) unlimited
> > scheduling priority (-e) 0
> > file size (blocks, -f) unlimited
> > pending signals (-i) 31533
> > max locked memory (kbytes, -l) 64
> > max memory size (kbytes, -m) unlimited
> > open files (-n) 1024
> > pipe size (512 bytes, -p) 8
> > POSIX message queues (bytes, -q) 819200
> > real-time priority (-r) 0
> > stack size (kbytes, -s) 8192
> > cpu time (seconds, -t) unlimited
> > max user processes (-u) 31533
> > virtual memory (kbytes, -v) unlimited
> > file locks (-x) unlimited
> >
> > jean-christophe at dyf-corto:~/Calculs/Cavite2D/BaseFlow$ size nek5000
> > text data bss dec hex filename
> > 2232834 3484 185835044 188071362 b35bdc2 nek5000
> >
> > jean-christophe at dyf-corto:~/Calculs/Cavite2D/BaseFlow$ cat SIZE
> > C Dimension file to be included
> > C
> > C HCUBE array dimensions
> > C
> > parameter (ldim=2)
> > parameter (lx1=11,ly1=lx1,lz1=1,lelt=225,lelv=lelt)
> > parameter (lxd=17,lyd=lxd,lzd=1)
> > parameter (lelx=20,lely=60,lelz=1)
> > c
> > parameter (lzl=3 + 2*(ldim-3))
> > c
> > parameter (lx2=lx1-2)
> > parameter (ly2=ly1-2)
> > parameter (lz2=1)
> > parameter (lx3=lx2)
> > parameter (ly3=ly2)
> > parameter (lz3=lz2)
> > c
> > c parameter (lpelv=lelv,lpelt=lelt,lpert=3) ! perturbation
> > c parameter (lpx1=lx1,lpy1=ly1,lpz1=lz1) ! array sizes
> > c parameter (lpx2=lx2,lpy2=ly2,lpz2=lz2)
> > c
> > parameter (lpelv=1,lpelt=1,lpert=1) ! perturbation
> > parameter (lpx1=1,lpy1=1,lpz1=1) ! array sizes
> > parameter (lpx2=1,lpy2=1,lpz2=1)
> > c
> > c
> > c parameter (lbelv=lelv,lbelt=lelt) ! MHD
> > c parameter (lbx1=lx1,lby1=ly1,lbz1=lz1) ! array sizes
> > c parameter (lbx2=lx2,lby2=ly2,lbz2=lz2)
> > c
> > parameter (lbelv=1,lbelt=1) ! MHD
> > parameter (lbx1=1,lby1=1,lbz1=1) ! array sizes
> > parameter (lbx2=1,lby2=1,lbz2=1)
> > c
> > C LX1M=LX1 when there are moving meshes; =1 otherwise
> > parameter (lx1m=lx1,ly1m=ly1,lz1m=lz1)
> > parameter (ldimt= 4) ! 3 passive scalars + T
> > parameter (ldimt1=ldimt+1)
> > parameter (ldimt3=ldimt+3)
> > parameter (lp = 8)
> > parameter (lelg = 300)
> > c
> > c Note: In the new code, LELGEC should be about sqrt(LELG)
> > c
> > PARAMETER (LELGEC = 1)
> > PARAMETER (LXYZ2 = 1)
> > PARAMETER (LXZ21 = 1)
> > c
> > PARAMETER (LMAXV=LX1*LY1*LZ1*LELV)
> > PARAMETER (LMAXT=LX1*LY1*LZ1*LELT)
> > PARAMETER (LMAXP=LX2*LY2*LZ2*LELV)
> > PARAMETER (LXZ=LX1*LZ1)
> > PARAMETER (LORDER=3)
> > PARAMETER (MAXOBJ=4,MAXMBR=LELT*6,lhis=100)
> > C
> > C Common Block Dimensions
> > C
> > PARAMETER (LCTMP0 =2*LX1*LY1*LZ1*LELT)
> > PARAMETER (LCTMP1 =4*LX1*LY1*LZ1*LELT)
> > C
> > C The parameter LVEC controls whether an additional 42 field arrays
> > C are required for Steady State Solutions. If you are not using
> > C Steady State, it is recommended that LVEC=1.
> > C
> > PARAMETER (LVEC=1)
> > C
> > C Uzawa projection array dimensions
> > C
> > parameter (mxprev = 80)
> > parameter (lgmres = 40)
> > C
> > C Split projection array dimensions
> > C
> > parameter(lmvec = 1)
> > parameter(lsvec = 1)
> > parameter(lstore=lmvec*lsvec)
> > c
> > c NONCONFORMING STUFF
> > c
> > parameter (maxmor = lelt)
> > C
> > C Array dimensions
> > C
> > COMMON/DIMN/NELV,NELT,NX1,NY1,NZ1,NX2,NY2,NZ2
> > $,NX3,NY3,NZ3,NDIM,NFIELD,NPERT,NID
> > $,NXD,NYD,NZD
> >
> > c automatically added by makenek
> > parameter(lxo = lx1) ! max output grid size (lxo>=lx1)
> >
> > c automatically added by makenek
> > parameter(lpart = 1 ) ! max number of particles
> >
> > c automatically added by makenek
> > integer ax1,ay1,az1,ax2,ay2,az2
> > parameter (ax1=lx1,ay1=ly1,az1=lz1,ax2=lx2,ay2=ly2,az2=lz2) !
> running
> > averages
> >
> > c automatically added by makenek
> > parameter (lxs=1,lys=lxs,lzs=(lxs-1)*(ldim-2)+1) !New Pressure
> > Preconditioner
> >
> > c automatically added by makenek
> > parameter (lfdm=0) ! == 1 for fast diagonalization method
> >
> > Concerning Nek, yes I do get an output on the screen when lauching the
> > calculation. Attached is the logfile for one example (2D cavity, only
> with
> > 100 elements though but the problem is the same as I increase the number
> of
> > elements).
> >
> > Cheers,
> >
> > On 18 October 2011 18:03, Jean-Christophe Loiseau
> > <loiseau.jc at gmail.com>wrote:
> >
> >> Hi,
> >>
> >> Sorry for this very late response, it completely got out of my mind.
> Here
> >> are the different outputs I get for these commands:
> >>
> >> jean-christophe at dyf-corto:~/Calculs/Cavite2D/BaseFlow$ ulimit -a
> >> core file size (blocks, -c) 0
> >> data seg size (kbytes, -d) unlimited
> >> scheduling priority (-e) 0
> >> file size (blocks, -f) unlimited
> >> pending signals (-i) 31533
> >> max locked memory (kbytes, -l) 64
> >> max memory size (kbytes, -m) unlimited
> >> open files (-n) 1024
> >> pipe size (512 bytes, -p) 8
> >> POSIX message queues (bytes, -q) 819200
> >> real-time priority (-r) 0
> >> stack size (kbytes, -s) 8192
> >> cpu time (seconds, -t) unlimited
> >> max user processes (-u) 31533
> >> virtual memory (kbytes, -v) unlimited
> >> file locks (-x) unlimited
> >>
> >> jean-christophe at dyf-corto:~/Calculs/Cavite2D/BaseFlow$ size nek5000
> >> text data bss dec hex filename
> >> 2232834 3484 185835044 188071362 b35bdc2 nek5000
> >>
> >> jean-christophe at dyf-corto:~/Calculs/Cavite2D/BaseFlow$ cat SIZE
> >> C Dimension file to be included
> >> C
> >> C HCUBE array dimensions
> >> C
> >> parameter (ldim=2)
> >> parameter (lx1=11,ly1=lx1,lz1=1,lelt=225,lelv=lelt)
> >> parameter (lxd=17,lyd=lxd,lzd=1)
> >> parameter (lelx=20,lely=60,lelz=1)
> >> c
> >> parameter (lzl=3 + 2*(ldim-3))
> >> c
> >> parameter (lx2=lx1-2)
> >> parameter (ly2=ly1-2)
> >> parameter (lz2=1)
> >> parameter (lx3=lx2)
> >> parameter (ly3=ly2)
> >> parameter (lz3=lz2)
> >> c
> >> c parameter (lpelv=lelv,lpelt=lelt,lpert=3) ! perturbation
> >> c parameter (lpx1=lx1,lpy1=ly1,lpz1=lz1) ! array sizes
> >> c parameter (lpx2=lx2,lpy2=ly2,lpz2=lz2)
> >> c
> >> parameter (lpelv=1,lpelt=1,lpert=1) ! perturbation
> >> parameter (lpx1=1,lpy1=1,lpz1=1) ! array sizes
> >> parameter (lpx2=1,lpy2=1,lpz2=1)
> >> c
> >> c
> >> c parameter (lbelv=lelv,lbelt=lelt) ! MHD
> >> c parameter (lbx1=lx1,lby1=ly1,lbz1=lz1) ! array sizes
> >> c parameter (lbx2=lx2,lby2=ly2,lbz2=lz2)
> >> c
> >> parameter (lbelv=1,lbelt=1) ! MHD
> >> parameter (lbx1=1,lby1=1,lbz1=1) ! array sizes
> >> parameter (lbx2=1,lby2=1,lbz2=1)
> >> c
> >> C LX1M=LX1 when there are moving meshes; =1 otherwise
> >> parameter (lx1m=lx1,ly1m=ly1,lz1m=lz1)
> >> parameter (ldimt= 4) ! 3 passive scalars + T
> >> parameter (ldimt1=ldimt+1)
> >> parameter (ldimt3=ldimt+3)
> >> parameter (lp = 8)
> >> parameter (lelg = 300)
> >> c
> >> c Note: In the new code, LELGEC should be about sqrt(LELG)
> >> c
> >> PARAMETER (LELGEC = 1)
> >> PARAMETER (LXYZ2 = 1)
> >> PARAMETER (LXZ21 = 1)
> >> c
> >> PARAMETER (LMAXV=LX1*LY1*LZ1*LELV)
> >> PARAMETER (LMAXT=LX1*LY1*LZ1*LELT)
> >> PARAMETER (LMAXP=LX2*LY2*LZ2*LELV)
> >> PARAMETER (LXZ=LX1*LZ1)
> >> PARAMETER (LORDER=3)
> >> PARAMETER (MAXOBJ=4,MAXMBR=LELT*6,lhis=100)
> >> C
> >> C Common Block Dimensions
> >> C
> >> PARAMETER (LCTMP0 =2*LX1*LY1*LZ1*LELT)
> >> PARAMETER (LCTMP1 =4*LX1*LY1*LZ1*LELT)
> >> C
> >> C The parameter LVEC controls whether an additional 42 field arrays
> >> C are required for Steady State Solutions. If you are not using
> >> C Steady State, it is recommended that LVEC=1.
> >> C
> >> PARAMETER (LVEC=1)
> >> C
> >> C Uzawa projection array dimensions
> >> C
> >> parameter (mxprev = 80)
> >> parameter (lgmres = 40)
> >> C
> >> C Split projection array dimensions
> >> C
> >> parameter(lmvec = 1)
> >> parameter(lsvec = 1)
> >> parameter(lstore=lmvec*lsvec)
> >> c
> >> c NONCONFORMING STUFF
> >> c
> >> parameter (maxmor = lelt)
> >> C
> >> C Array dimensions
> >> C
> >> COMMON/DIMN/NELV,NELT,NX1,NY1,NZ1,NX2,NY2,NZ2
> >> $,NX3,NY3,NZ3,NDIM,NFIELD,NPERT,NID
> >> $,NXD,NYD,NZD
> >>
> >> c automatically added by makenek
> >> parameter(lxo = lx1) ! max output grid size (lxo>=lx1)
> >>
> >> c automatically added by makenek
> >> parameter(lpart = 1 ) ! max number of particles
> >>
> >> c automatically added by makenek
> >> integer ax1,ay1,az1,ax2,ay2,az2
> >> parameter (ax1=lx1,ay1=ly1,az1=lz1,ax2=lx2,ay2=ly2,az2=lz2) !
> >> running
> >> averages
> >>
> >> c automatically added by makenek
> >> parameter (lxs=1,lys=lxs,lzs=(lxs-1)*(ldim-2)+1) !New Pressure
> >> Preconditioner
> >>
> >> c automatically added by makenek
> >> parameter (lfdm=0) ! == 1 for fast diagonalization method
> >>
> >> Concerning Nek, yes I do get an output on the screen when lauching the
> >> calculation. Attached is the logfile for one example (2D cavity, only
> with
> >> 100 elements though but the problem is the same as I increase the number
> >> of
> >> elements).
> >>
> >> Cheers,
> >>
> >>
> >> On 27 September 2011 17:40, <nek5000-users at lists.mcs.anl.gov> wrote:
> >>
> >>> Hi,
> >>>
> >>> Do you get any output printed to the screen when you launch Nek?
> >>> Please provide the output of the following command:
> >>> ulimit -a (in case you are using bash)
> >>> size nek5000
> >>> cat SIZE
> >>>
> >>> Can you post the command you are using to launch Nek in parallel?
> >>>
> >>> Cheers,
> >>> Stefan
> >>>
> >>>
> >>>
> >>> On 9/27/11, nek5000-users at lists.mcs.anl.gov
> >>> <nek5000-users at lists.mcs.anl.gov> wrote:
> >>> > Hi Neks,
> >>> >
> >>> > I do get the same error when I try to run nek on my laptop. Did you
> >>> > find
> >>> any
> >>> > way to solve the problem John?
> >>> >
> >>> > Regards,
> >>> >
> >>> > On 11 May 2011 21:25, <nek5000-users at lists.mcs.anl.gov> wrote:
> >>> >
> >>> >> Hi,
> >>> >>
> >>> >> I have lp=512 in SIZE, and I did compile with IFMPI=true. I
> >>> >> recompiled
> >>> >> yesterday because I thought this may be the case, but I was still
> >>> getting
> >>> >> the same error. When I first start a simulation and run top, it
> shows
> >>> >> four
> >>> >> processors running nek5000, but this changes to one processor after
> a
> >>> few
> >>> >> seconds.
> >>> >>
> >>> >> Thanks,
> >>> >>
> >>> >> John
> >>> >>
> >>> >> _______________________________________________
> >>> >> Nek5000-users mailing list
> >>> >> Nek5000-users at lists.mcs.anl.gov
> >>> >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>> > --
> >>> > Jean-Christophe
> >>> >
> >>> _______________________________________________
> >>> Nek5000-users mailing list
> >>> Nek5000-users at lists.mcs.anl.gov
> >>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> >>>
> >>
> >>
> >>
> >> --
> >> Jean-Christophe
> >>
> >
> >
> >
> > --
> > Jean-Christophe
> >
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>
--
Jean-Christophe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/nek5000-users/attachments/20111018/262a4729/attachment.html>
More information about the Nek5000-users
mailing list