[petsc-dev] KNC build?
Satish Balay
balay at mcs.anl.gov
Wed Nov 5 12:47:12 CST 2014
getPointerSizeFlag() is probably the wrong name here [perhaps it
should be: 'instruction set' or 'machine arch' or just 'misc'
Can you check if the following change fixes the issue?
Satish
-------
$ git diff |cat
diff --git a/config/BuildSystem/config/package.py b/config/BuildSystem/config/package.py
index 6080825..7f7cd3c 100644
--- a/config/BuildSystem/config/package.py
+++ b/config/BuildSystem/config/package.py
@@ -150,7 +150,7 @@ class Package(config.base.Configure):
return ''
def getPointerSizeFlag(self,cflags):
- for flag in ['-m32', '-m64', '-xarch=v9','-q64']:
+ for flag in ['-m32', '-m64', '-xarch=v9','-q64','-mmic']:
if cflags.find(flag) >=0: return flag
return ''
On Wed, 5 Nov 2014, Satish Balay wrote:
> x86_64-k1om-linux-ld: i386:x86-64 architecture of input file
> `/global/homes/m/madams/petsc_private/arch-knc-dbg/lib/libsuperlu_4.3.a(superlu_timer.o)'
> is incompatible with k1om output
>
>
> This is probably due to the hacky code required to build some of the
> externalpackages [to deal with NOOPT]
>
> superlu.py:
>
> g.write('NOOPTS = '+self.getSharedFlag(self.setCompilers.getCompilerFlags())+' '+self.getPointerSizeFlag(self.setCompilers. getCompilerFlags())+' '+self.getWindowsNonOptFlags(self.setCompilers.getCompilerFlags())+'\n')
>
> Well this needs a fix so that '-mmic' option can pass through [perhaps also '-mkl -fp-model precise']
>
> As a hack - you can try:
>
> g.write('NOOPTS = -mmic')
>
>
> SuperLU_DIST.py will also require similar fix.
>
> Satish
>
> On Wed, 5 Nov 2014, Mark Adams wrote:
>
> > Oh sorry, the error was at link time:
> >
> > 01:18 bint01 master ~/petsc_private/src/ksp/ksp/examples/tutorials$ make
> > PETSC_DIR=/global/homes/m/madams/petsc_private PETSC_ARCH=arch-knc-dbg ex1
> > mpiicc -o ex1.o -c -mmic -mkl -fp-model precise -g
> > -I/global/homes/m/madams/petsc_private/include
> > -I/global/homes/m/madams/petsc_private/arch-knc-dbg/include
> > -I/chos/global/u2/m/madams/petsc_private/arch-knc-dbg/include
> > -I/global/babbage/nsg/opt/intel/impi/4.1.3.048/intel64/include
> > `pwd`/ex1.c
> > mpiicc -mmic -mkl -fp-model precise -g -o ex1 ex1.o
> > -L/global/homes/m/madams/petsc_private/arch-knc-dbg/lib -lpetsc
> > -Wl,-rpath,/chos/global/u2/m/madams/petsc_private/arch-knc-dbg/lib
> > -L/chos/global/u2/m/madams/petsc_private/arch-knc-dbg/lib -lsuperlu_4.3
> > -lsuperlu_dist_3.3 -lparmetis -lmetis -lpthread
> > -L/global/babbage/nsg/opt/intel/impi/4.1.3.048/mic/lib
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/mkl/lib/mic
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/tbb/lib/mic
> > -L/chos/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic
> > -L/usr/linux-k1om-4.7/lib/gcc/x86_64-k1om-linux/4.7.0
> > -L/usr/linux-k1om-4.7/lib/gcc -L/usr/linux-k1om-4.7/linux-k1om/lib64
> > -L/usr/linux-k1om-4.7/linux-k1om/usr/lib64
> > -Wl,-rpath,/global/babbage/nsg/opt/intel/impi/4.1.3.048/mic/lib
> > -Wl,-rpath,/opt/intel/mpi-rt/4.1 -lifport -lifcore -lm
> > -Wl,-rpath,/chos/global/u2/m/madams/petsc_private/or -lmpigc4 -ldl
> > -L/global/babbage/nsg/opt/intel/impi/4.1.3.048/mic/lib -lmpigf -lmpi_dbg
> > -lmpigi -lrt -lpthread -L/global/babbage/nsg/opt/intel/impi/
> > 4.1.3.048/mic/lib
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/mkl/lib/mic
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/tbb/lib/mic
> > -L/chos/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic
> > -L/usr/linux-k1om-4.7/lib/gcc/x86_64-k1om-linux/4.7.0
> > -L/usr/linux-k1om-4.7/lib/gcc -L/usr/linux-k1om-4.7/linux-k1om/lib64
> > -L/usr/linux-k1om-4.7/linux-k1om/usr/lib64
> > -L/usr/linux-k1om-4.7/linux-k1om/lib64
> > -L/usr/linux-k1om-4.7/linux-k1om/usr/lib64 -lmkl_intel_lp64
> > -lmkl_intel_thread -lmkl_core -liomp5
> > -Wl,-rpath,/global/babbage/nsg/opt/intel/impi/4.1.3.048/mic/lib
> > -Wl,-rpath,/opt/intel/mpi-rt/4.1
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/mkl/lib/mic
> > -limf -lsvml -lirng -lipgo -ldecimal -lcilkrts -lstdc++ -lgcc_s -lirc
> > -lirc_s -L/global/babbage/nsg/opt/intel/impi/4.1.3.048/mic/lib
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/mkl/lib/mic
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/tbb/lib/mic
> > -L/chos/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic
> > -L/usr/linux-k1om-4.7/lib/gcc/x86_64-k1om-linux/4.7.0
> > -L/usr/linux-k1om-4.7/lib/gcc -L/usr/linux-k1om-4.7/linux-k1om/lib64
> > -L/usr/linux-k1om-4.7/linux-k1om/usr/lib64
> > -L/usr/linux-k1om-4.7/linux-k1om/lib64
> > -L/usr/linux-k1om-4.7/linux-k1om/usr/lib64
> > -L/global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/mkl/lib/mic
> > -ldl
> > x86_64-k1om-linux-ld: warning: libintlc.so.5, needed by
> > /global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic/libifport.so.5,
> > not found (try using -rpath or -rpath-link)
> > x86_64-k1om-linux-ld: i386:x86-64 architecture of input file
> > `/global/homes/m/madams/petsc_private/arch-knc-dbg/lib/libsuperlu_4.3.a(superlu_timer.o)'
> > is incompatible with k1om output
> > x86_64-k1om-linux-ld:
> > /global/homes/m/madams/petsc_private/arch-knc-dbg/lib/libsuperlu_4.3.a(superlu_timer.o)(.text+0xc):
> > reloc against `sysconf@@GLIBC_2.14': error 4
> > x86_64-k1om-linux-ld: final link failed: Nonrepresentable section on output
> > make: [ex1] Error 1 (ignored)
> > /bin/rm -f ex1.o
> >
> >
> > On Wed, Nov 5, 2014 at 12:29 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> >
> > > configure.log looks fine. Perhaps the errors are in test.log?
> > >
> > > There is stuff like:
> > >
> > > >>>>
> > > Possible ERROR while running linker:
> > > stderr:
> > > x86_64-k1om-linux-ld: warning: libintlc.so.5, needed by
> > > /global/babbage/nsg/opt/intel/composerxe/composer_xe_2015.0.090/compiler/lib/mic/libifport.so.5,
> > > not found (try using -rpath or -rpath-link)
> > > <<<<
> > >
> > > But perhaps thats a compiler issue [or an issue with autodetect stuff.]
> > >
> > > Satish
> > >
> > > On Wed, 5 Nov 2014, Mark Adams wrote:
> > >
> > > > I am thinking I need to get a manual build of SuperLU but let me know if
> > > > you have any suggestions.
> > > > Thanks,
> > > > Mark
> > > >
> > > > On Wed, Nov 5, 2014 at 11:59 AM, Satish Balay <balay at mcs.anl.gov> wrote:
> > > >
> > > > > No log attached - so I can't comment..
> > > > >
> > > > > satish
> > > > >
> > > > > On Wed, 5 Nov 2014, Mark Adams wrote:
> > > > >
> > > > > > Thanks, it is working now. Links fail with a SuperLU error. I'm
> > > > > guessing
> > > > > > I need to use a manually built SuperLU.
> > > > > > Mark
> > > > > >
> > > > > > On Tue, Nov 4, 2014 at 9:13 PM, Satish Balay <balay at mcs.anl.gov>
> > > wrote:
> > > > > >
> > > > > > > >>>>>>
> > > > > > > checking whether the C compiler works...configure: error: in
> > > > > > >
> > > > >
> > > `/chos/global/u2/m/madams/petsc_private/arch-knc-dbg64/externalpackages/hypre-2.9.1a/src':
> > > > > > > configure: error: cannot run C compiled programs.
> > > > > > > If you meant to cross compile, use `--host'.
> > > > > > > <<<<<
> > > > > > >
> > > > > > > we really don't have support for batch build of
> > > > > > > externalpacakges. [they usually work - which is great - but it
> > > won't
> > > > > > > work always].
> > > > > > >
> > > > > > > So it has to be installed manually.
> > > > > > >
> > > > > > > BTW: --with-cc="mpiicc " --with-cxx="mpiicpc " has extra
> > > > > > > 'spaces'. Perhaps its not causing problems yet. [cmake usually
> > > barfs
> > > > > > > with this type of usage]
> > > > > > >
> > > > > > > Satish
> > > > > > >
> > > > > > > On Tue, 4 Nov 2014, Mark Adams wrote:
> > > > > > >
> > > > > > > > Thanks, getting better ... I now get this error with hypre.
> > > hypre
> > > > > is
> > > > > > > not
> > > > > > > > critical here but if someone has ideas ...
> > > > > > > >
> > > > > > > > On Tue, Nov 4, 2014 at 7:43 PM, Satish Balay <balay at mcs.anl.gov>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > --CFLAGS=""-mmic -mkl -fp-model precise""
> > > > > > > > >
> > > > > > > > > remove the extra set of quotes from your ../arch-knc-dbg64.py.
> > > > > > > > > For eg: petsc/config/examples/arch-cray-xt6-pkgs-opt.py has:
> > > > > > > > >
> > > > > > > > > '--COPTFLAGS=-fast -mp',
> > > > > > > > >
> > > > > > > > > The equivalent shell would be:
> > > > > > > > >
> > > > > > > > > --COPTFLAGS="-fast -mp"
> > > > > > > > >
> > > > > > > > > Satish
> > > > > > > > >
> > > > > > > > > On Tue, 4 Nov 2014, Mark Adams wrote:
> > > > > > > > >
> > > > > > > > > > Thanks this helped. I am now getting METIS errors.
> > > > > > > > > >
> > > > > > > > > > On Tue, Nov 4, 2014 at 6:30 PM, Satish Balay <
> > > balay at mcs.anl.gov>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Ah - didn't check the configure comand closely enough..
> > > > > > > > > > >
> > > > > > > > > > > > --with-mpicc=/global/babbage/nsg/opt/intel/impi/
> > > > > > > > > > > 5.0.1.035/intel64/bin/mpicc
> > > > > > > > > > > > --with-mpicxx=/global/babbage/nsg/opt/intel/impi/
> > > > > > > > > > > 5.0.1.035/intel64/bin/mpicxx
> > > > > > > > > > > > --with-mpif90=/global/babbage/nsg/opt/intel/impi/
> > > > > > > > > > > 5.0.1.035/intel64/bin/mpif90
> > > > > > > > > > >
> > > > > > > > > > > petsc configure doesn't provide any such options..
> > > > > > > > > > >
> > > > > > > > > > > > --with-mpi-lib=/global/babbage/nsg/opt/intel/impi/
> > > > > > > > > 5.0.1.035/intel64/lib
> > > > > > > > > > >
> > > > > > > > > > > This would be the wrong way to provide MPI lib.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > The correct way to use Intel MPI is:
> > > > > > > > > > >
> > > > > > > > > > > --with-cc=mpiipc --with-cxx=mpiicpc --with-fc=mpiifort
> > > [i.e no
> > > > > > > > > > > --with-mpi-include or --with-mpi-lib options]
> > > > > > > > > > >
> > > > > > > > > > > But as Richard mentioned - the wrappers in intel impi-5 are
> > > > > > > > > > > broken. Alternative is to use impi-4 as suggested.
> > > > > > > > > > >
> > > > > > > > > > > Or something like the following (for compilers/mpi):
> > > > > > > > > > >
> > > > > > > > > > > ./configure --with-cc=icc --with-fc=ifort --with-cxx=0
> > > > > > > > > > > --with-mpi-include=/global/babbage/nsg/opt/intel/impi/
> > > > > > > > > > > 5.0.1.035/intel64/include
> > > > > > > > > > > --with-mpi-lib="-L/global/babbage/nsg/opt/intel/impi/
> > > > > > > > > > > 5.0.1.035/intel64/lib/release
> > > > > > > -L/global/babbage/nsg/opt/intel/impi/
> > > > > > > > > > > 5.0.1.035/intel64/lib -lmpifort -lmpi -lmpigi -ldl -lrt
> > > > > -lpthread"
> > > > > > > > > > >
> > > > > > > > > > > Satish
> > > > > > > > > > >
> > > > > > > > > > > On Tue, 4 Nov 2014, Richard Mills wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Mark,
> > > > > > > > > > > >
> > > > > > > > > > > > I noticed that you are using the wrong compiler wrappers.
> > > > > > > Instead of
> > > > > > > > > > > > 'mpicc', 'mpif90', and 'mpicxx' (which will invoke GNU
> > > > > > > compilers),
> > > > > > > > > you
> > > > > > > > > > > > probably want 'mpiicc', 'mpiifort', and 'mpiicpc'
> > > (invoke the
> > > > > > > Intel
> > > > > > > > > > > > compilers).
> > > > > > > > > > > >
> > > > > > > > > > > > You will probably also want to unload the IMPI 5.x and
> > > load
> > > > > the
> > > > > > > IMPI
> > > > > > > > > > > 4.1.3
> > > > > > > > > > > > module right now because of a stupid bug in the compiler
> > > > > wrappers
> > > > > > > > > (for
> > > > > > > > > > > > which I've filed a bug report--will be fixed in next IMPI
> > > > > > > release,
> > > > > > > > > > > sometime
> > > > > > > > > > > > before SC14).
> > > > > > > > > > > >
> > > > > > > > > > > > --Richard
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Nov 4, 2014 at 12:59 PM, Mark Adams <
> > > mfadams at lbl.gov
> > > > > >
> > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Thanks Victor,
> > > > > > > > > > > > >
> > > > > > > > > > > > > I am getting cc error. Any ideas?
> > > > > > > > > > > > > Mark
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Nov 4, 2014 at 12:38 PM, Victor Eijkhout <
> > > > > > > > > > > eijkhout at tacc.utexas.edu
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> > On Nov 4, 2014, at 11:32 AM, Mark Adams <
> > > > > mfadams at lbl.gov>
> > > > > > > > > wrote:
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> > Has anyone built PETSc on KNC (Babbage at NERSC to
> > > be
> > > > > > > specific)?
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> At TACC under intel 14:
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> CEE_OPTIONS="-mmic -mkl -fp-model precise"
> > > > > > > > > > > > >> ./configure --PETSC_ARCH=mic --with-fc=0
> > > --with-debug=0 \
> > > > > > > > > > > > >> --with-batch=1 --CPPFLAGS=-mmic \
> > > > > > > > > > > > >> --CFLAGS="${CEE_OPTIONS}"
> > > --CXXFLAGS="${CEE_OPTIONS}"
> > > > > > > > > > > > >> --FFLAGS="${CEE_OPTIONS}" \
> > > > > > > > > > > > >> --with-mpi=1 --known-mpi-shared-libraries=1 \
> > > > > > > > > > > > >> --with-mpi-include=${MPICH_HOME}/mic/include \
> > > > > > > > > > > > >> --with-mpi-lib=${MPICH_HOME}/mic/lib \
> > > > > > > > > > > > >> --with-mpicc=/opt/apps/intel13/impi/
> > > > > > > > > > > 4.1.0.030/intel64/bin/mpicc \
> > > > > > > > > > > > >> --with-mpicxx=/opt/apps/intel13/impi/
> > > > > > > > > > > 4.1.0.030/intel64/bin/mpicxx
> > > > > > > > > > > > >> \
> > > > > > > > > > > > >> --with-mpif90=/opt/apps/intel13/impi/
> > > > > > > > > > > 4.1.0.030/intel64/bin/mpif90
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> That first line is still giving me problems. Just nix
> > > the
> > > > > > > > > fp-model.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Victor.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
>
More information about the petsc-dev
mailing list