[Nek5000-users] turbJet compiling problem

Fri Oct 18 11:05:04 CDT 2013

Ok, sorry for this obvious thing. Recompiled for 128 processes and lelt=100.

However, when executed:

 read .rea file
 nelgt/nelgv/lelt:          10          10          80
 lx1  /lx2  /lx3 :           8           6           8

 mapping elements to processors
  ABORT: Too many processors (     128) for to few elements (      10).
  ABORTING IN MAPELPR.

call exitt: dying ...

backtrace(): obtained 9 stack frames.
[0x50f8c8]
[0x5ad1fd]
[0x4733da]
[0x43294c]
[0x401e8c]
[0x401e16]
[0x400714]
[0x76e794]
[0x4005e5]

Effectively there are 10 2D elements described in the *.rea files and my guess is that the 10000 3D elements are in the *.re2.
What am I missing?

On Oct 18, 2013, at 11:58 AM, <nek5000-users at lists.mcs.anl.gov>
 wrote:

>
> PS -- sorry; I was looking at turbChan, not turbJet.
>
> Katie is right -- turbJet is larger still.
>
> My suggestions below however would still apply.
>
> Paul
>
>
> On Fri, 18 Oct 2013, nek5000-users at lists.mcs.anl.gov wrote:
>
>>
>> My guess is simply the amount of memory required, which is 1.3 GB
>> per node (as set up in the example; one can easily lower this by
>> a factor of P when running on P processors; just set
>> lelt and lelv to be ~ 512 / P).
>>
>> Note that this example compiles here at ANL with 4 different
>> compilers, both in parallel an in serial, as do all the other
>> examples, so somehow it must have something to do with your
>> local environment settings.
>>
>> I just tried it on my workstation - no issues.
>>
>> What type of platform are you compiling on, etc.
>>
>> Paul
>>
>>
>> On Fri, 18 Oct 2013, nek5000-users at lists.mcs.anl.gov wrote:
>>
>>> Hi,
>>> What's inherently different with this example turbJet and most of the other's (they compiled correctly)?.
>>> Anyways, that's what I got after turning off MPI (IFMPI="false"):
>>> cc -c -Ktrap=fp -mcmodel=medium -O2 -DPTRSIZE8 -DLONGINT8 -DUNDERSCORE -DGLOBAL_LONG_LONG  -DPREFIX=jl_ /work/alexf/nek5_svn/trunk/nek/jl/xxt.c -o obj/jl_xxt.o
>>> cc -c -Ktrap=fp -mcmodel=medium -O2 -DPTRSIZE8 -DLONGINT8 -DUNDERSCORE -DGLOBAL_LONG_LONG  -DPREFIX=jl_ /work/alexf/nek5_svn/trunk/nek/jl/fcrs.c -o obj/jl_fcrs.o
>>> ftn -c -Ktrap=fp -mcmodel=medium -O2 -r8 -Mpreprocess -DPTRSIZE8 -DLONGINT8 -DUNDERSCORE -DGLOBAL_LONG_LONG -I/work/alexf/nek_examples/turbJet -I/work/alexf/nek5_svn/trunk/nek -I./ /work/alexf/nek_examples/turbJet/jet.f    -o obj/jet.o
>>> ftn -o nek5000 -Ktrap=fp -mcmodel=medium obj/jet.o obj/drive.o obj/drive1.o obj/drive2.o obj/plan4.o obj/bdry.o obj/coef.o obj/conduct.o obj/connect1.o obj/connect2.o obj/dssum.o obj/edgec.o obj/eigsolv.o obj/gauss.o obj/genxyz.o obj/navier1.o obj/makeq.o obj/navier0.o obj/navier2.o obj/navier3.o obj/navier4.o obj/prepost.o obj/speclib.o obj/map2.o obj/turb.o obj/mvmesh.o obj/ic.o obj/ssolv.o obj/planx.o obj/math.o obj/mxm_wrapper.o obj/hmholtz.o obj/gfdm_par.o obj/gfdm_op.o obj/gfdm_solve.o obj/subs1.o obj/subs2.o obj/genbox.o obj/gmres.o obj/hsmg.o obj/convect.o obj/induct.o obj/perturb.o obj/navier5.o obj/navier6.o obj/navier7.o obj/navier8.o obj/fast3d.o obj/fasts.o obj/calcz.o obj/byte.o obj/chelpers.o obj/byte_mpi.o obj/postpro.o obj/cvode_driver.o obj/nek_comm.o obj/init_plugin.o obj/setprop.o obj/qthermal.o obj/cvode_aux.o obj/makeq_aux.o obj/papi.o obj/ssygv.o obj/dsygv.o obj/nek_in_situ.o obj/mxm_std.o obj/blas.o obj/comm_mpi.o obj/mpi_dummy.o obj/singlmesh.o obj/
>> jl_gs.o obj/jl_sort.o obj/jl_sarray_transfer.o obj/jl_sarray_sort.o obj/jl_gs_local.o obj/jl_crystal.o obj/jl_comm.o obj/jl_tensor.o obj/jl_fail.o obj/jl_fcrystal.o obj/jl_findpts.o obj/jl_findpts_local.o obj/jl_obbox.o obj/jl_poly.o obj/jl_lob_bnd.o obj/jl_findpts_el_3.o obj/jl_findpts_el_2.o obj/jl_sparse_cholesky.o obj/jl_xxt.o obj/jl_fcrs.o
>>> /usr/lib64/libpthread.a(ptw-close.o): In function `close':
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/../sysdeps/unix/syscall-template.S:82: relocation truncated to fit: R_X86_64_PC32 against symbol `__pthread_multiple_threads' defined in COMMON section in /usr/lib64/libpthread.a(vars.o)
>>> /usr/lib64/libpthread.a(ptw-open.o): In function `open64':
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/../sysdeps/unix/syscall-template.S:82: relocation truncated to fit: R_X86_64_PC32 against symbol `__pthread_multiple_threads' defined in COMMON section in /usr/lib64/libpthread.a(vars.o)
>>> /usr/lib64/libpthread.a(pthread_create.o): In function `__nptl_setxid':
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:1066: relocation truncated to fit: R_X86_64_PC32 against symbol `__xidcmd' defined in COMMON section in /usr/lib64/libpthread.a(nptl-init.o)
>>> /usr/lib64/libpthread.a(pthread_create.o): In function `allocate_stack':
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:462: relocation truncated to fit: R_X86_64_PC32 against symbol `__static_tls_align_m1' defined in COMMON section in /usr/lib64/libpthread.a(nptl-init.o)
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:468: relocation truncated to fit: R_X86_64_PC32 against symbol `__static_tls_size' defined in COMMON section in /usr/lib64/libpthread.a(nptl-init.o)
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:549: relocation truncated to fit: R_X86_64_PC32 against symbol `__libc_multiple_threads_ptr' defined in COMMON section in /usr/lib64/libpthread.a(pthread_create.o)
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:549: relocation truncated to fit: R_X86_64_PC32 against symbol `__pthread_multiple_threads' defined in COMMON section in /usr/lib64/libpthread.a(vars.o)
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:715: relocation truncated to fit: R_X86_64_PC32 against symbol `__static_tls_size' defined in COMMON section in /usr/lib64/libpthread.a(nptl-init.o)
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:362: relocation truncated to fit: R_X86_64_PC32 against symbol `__static_tls_size' defined in COMMON section in /usr/lib64/libpthread.a(nptl-init.o)
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:368: relocation truncated to fit: R_X86_64_PC32 against symbol `__static_tls_align_m1' defined in COMMON section in /usr/lib64/libpthread.a(nptl-init.o)
>>> /usr/src/packages/BUILD/glibc-2.11.1/nptl/allocatestack.c:407: additional relocation overflows omitted from the output
>>> make: *** [nek5000] Error 2
>>> And this is the relevant part of my makenek file:
>>> nk/nek/mpi_dummy.f -o obj/mpi_dummy.o
>>> ftn -c -Ktrap=fp -mcmodel=medium -O2 -r8 -Mpreprocess -DPTRSIZE8 -DLONGINT8 -DUNDERSCORE -DGLOBAL_LONG_LONG -I/work/alexf/nek_examples/turbJet -I/work/alexf/nek5_svn/trunk/nek -I./ /work/alexf/nek5_svn/trunk/nek/singlmesh.f -o obj/singlmesh.o
>>> #!/bin/bash
>>> # Nek5000 build config file
>>> # (c) 2008,2009,2010 UCHICAGO ARGONNE, LLC
>>> # source path
>>> SOURCE_ROOT="/work/alexf/nek5_svn/trunk/nek"
>>> # Fortran compiler
>>> F77="ftn"
>>> # C compiler
>>> CC="cc"
>>> # pre-processor symbol list
>>> # (set PPLIST=? to get a list of available symbols)
>>> #PPLIST="?"
>>> # plug-in list
>>> PLUGIN_LIST=""
>>> # OPTIONAL SETTINGS
>>> # -----------------
>>> # enable MPI (default true)
>>> IFMPI="false"
>>> # auxilliary files to compile
>>> # NOTE: source files have to located in the same directory as makenek
>>> #       a makefile_usr.inc has to be provided containing the build rules
>>> #USR="foo.o"
>>> # linking flags
>>> #USR_LFLAGS="-L/usr/lib -lfoo"
>>> # generic compiler flags
>>> G="-Ktrap=fp -mcmodel=medium"
>>> On Oct 18, 2013, at 10:37 AM, <nek5000-users at lists.mcs.anl.gov>
>>> wrote:
>>>> Do you know what compiler was used for the mpi build ?
>>>> You might try turning off mpi in the makenek script.
>>>> Also, are you not using mpif77 / mpicc ?  (Obviously
>>>> if you set IFMPI to false then you don't want these
>>>> compilers.)
>>>> Paul
>>>> ----- Original Message -----
>>>> From: nek5000-users at lists.mcs.anl.gov
>>>> To: "<nek5000-users at lists.mcs.anl.gov>" <nek5000-users at lists.mcs.anl.gov>
>>>> Sent: Friday, October 18, 2013 9:22:23 AM
>>>> Subject: Re: [Nek5000-users] turbJet compiling problem
>>>> After adding G= -Ktrap=fp -mcmodel=medium, the compilation completed correctly for gfortran but for PGI I got a new error:
>>>> /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_poll.o): In function `MPID_nem_gni_process_remote_data_ack':
>>>> gni_poll.c:(.text+0x15b): relocation truncated to fit: R_X86_64_32S against symbol `MPID_nem_gni_send_recv_bufs' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0x216): relocation truncated to fit: R_X86_64_32S against symbol `MPID_nem_gni_send_bufs' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_poll.o): In function `MPID_nem_gni_progress_localCQ':
>>>> gni_poll.c:(.text+0xdde): relocation truncated to fit: R_X86_64_PC32 against symbol `MPID_nem_gni_local_nic_addrs' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0xe5b): relocation truncated to fit: R_X86_64_PC32 against symbol `MPID_nem_gni_local_nic_addrs' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0xe75): relocation truncated to fit: R_X86_64_PC32 against symbol `MPID_nem_gni_nic_hndls' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_poll.o): In function `MPID_nem_gni_check_localCQ':
>>>> gni_poll.c:(.text+0xf63): relocation truncated to fit: R_X86_64_PC32 against symbol `MPID_nem_gni_local_nic_addrs' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0xf8e): relocation truncated to fit: R_X86_64_PC32 against symbol `MPID_nem_gni_nic_hndls' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0x127f): relocation truncated to fit: R_X86_64_32S against symbol `MPID_nem_gni_queue_work_head' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0x1299): relocation truncated to fit: R_X86_64_PC32 against symbol `MPID_nem_gni_queue_work_head' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0x12a4): relocation truncated to fit: R_X86_64_PC32 against symbol `MPID_nem_gni_queue_work_head' defined in COMMON section in /opt/cray/mpt/5.5.4/gni/mpich2-pgi/119/lib/libmpich_pgi.a(gni_init.o)
>>>> gni_poll.c:(.text+0x1478): additional relocation overflows omitted from the output
>>>> make: *** [nek5000] Error 2
>>>> On Oct 17, 2013, at 12:31 PM, nek5000-users at lists.mcs.anl.gov wrote:
>>>>> Try modifying makenek to have the line:
>>>>> G= -Ktrap=fp -mcmodel=medium
>>>>> (the G= is there but is commented out)
>>>>> Then:
>>>>> makenek clean
>>>>> and remake...
>>>>> Paul
>>>>> On Thu, 17 Oct 2013, nek5000-users at lists.mcs.anl.gov wrote:
>>>>>> Hi neks,
>>>>>> I am trying to compile the out-of-the-box turbojet case in the examples.
>>>>>> This is the output I got for PGI:
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `__hpf_myprocnum':
>>>>>> initpar.c:(.text+0x6): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_lcpu' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `__hpf_ncpus':
>>>>>> initpar.c:(.text+0x16): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_tcpus' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `__hpf_getioproc':
>>>>>> initpar.c:(.text+0x26): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_ioproc' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `__hpf_is_ioproc':
>>>>>> initpar.c:(.text+0x36): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_lcpu' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> initpar.c:(.text+0x3e): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_ioproc' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `__hpf_abort':
>>>>>> initpar.c:(.text+0x7a): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_lcpu' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `__hpf_abortp':
>>>>>> initpar.c:(.text+0xe0): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_lcpu' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `__hpf_passargs':
>>>>>> initpar.c:(.text+0x4c2): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_lcpu' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> initpar.c:(.text+0x616): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_lcpu' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o): In function `pghpf_init':
>>>>>> initpar.c:(.text+0x7d8): relocation truncated to fit: R_X86_64_PC32 against symbol `__hpf_np2' defined in COMMON section in /opt/pgi/12.8.0/linux86-64/12.8/lib/libpgf90.a(initpar.o)
>>>>>> initpar.c:(.text+0x7e2): additional relocation overflows omitted from the output
>>>>>> make: *** [nek5000] Error 2
>>>>>> and this is for gfortran
>>>>>> obj/postpro.o: In function `intpts_':
>>>>>> postpro.f:(.text+0x1818): relocation truncated to fit: R_X86_64_32S against symbol `intp_r_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x1831): relocation truncated to fit: R_X86_64_32S against symbol `intp_r_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x1878): relocation truncated to fit: R_X86_64_32S against symbol `intp_i_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x1915): relocation truncated to fit: R_X86_64_32S against symbol `intp_r_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x1927): relocation truncated to fit: R_X86_64_32S against symbol `intp_i_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x1c7d): relocation truncated to fit: R_X86_64_32S against symbol `intp_r_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x1dc4): relocation truncated to fit: R_X86_64_32S against symbol `intp_r_' defined in COMMON section in obj/postpro.o
>>>>>> obj/postpro.o: In function `tens3d1_':
>>>>>> postpro.f:(.text+0x1ef0): relocation truncated to fit: R_X86_64_32S against symbol `ctensor_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x1ef8): relocation truncated to fit: R_X86_64_32S against symbol `ctensor_' defined in COMMON section in obj/postpro.o
>>>>>> obj/postpro.o: In function `build_1d_filt_':
>>>>>> postpro.f:(.text+0x2124): relocation truncated to fit: R_X86_64_32S against symbol `cfiltr_' defined in COMMON section in obj/postpro.o
>>>>>> postpro.f:(.text+0x2136): additional relocation overflows omitted from the output
>>>>>> collect2: ld returned 1 exit status
>>>>>> make: *** [nek5000] Error 1
>>>>>> Any help?. Thanks
>>>>>> ________________________________
>>>>>> CSI has been ranked one of America’s Best Colleges 2014<http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/regional-universities-north/top-public/spp+50/page+2> (North East Public) by US News and World Report.
>>>>>> _______________________________________________
>>>>> Nek5000-users mailing list
>>>>> Nek5000-users at lists.mcs.anl.gov
>>>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users_______________________________________________
>>>>> Nek5000-users mailing list
>>>>> Nek5000-users at lists.mcs.anl.gov
>>>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>>> ________________________________
>>>> CSI has been ranked one of America’s Best Colleges 2014<http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/regional-universities-north/top-public/spp+50/page+2> (North East Public) by US News and World Report.
>>>> _______________________________________________
>>>> Nek5000-users mailing list
>>>> Nek5000-users at lists.mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>>> _______________________________________________
>>>> Nek5000-users mailing list
>>>> Nek5000-users at lists.mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>> ________________________________
>>> CSI has been ranked one of America’s Best Colleges 2014<http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/regional-universities-north/top-public/spp+50/page+2> (North East Public) by US News and World Report.
>>> _______________________________________________
>>> Nek5000-users mailing list
>>> Nek5000-users at lists.mcs.anl.gov
>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

________________________________

CSI has been ranked one of America’s Best Colleges 2014<http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/regional-universities-north/top-public/spp+50/page+2> (North East Public) by US News and World Report.