[petsc-dev] Possible error running C/C++ src/snes/examples/tutorials/ex19 with 1 MPI process

[Satish Balay]

On Wed, 12 Oct 2016, Antonio Trande wrote:

> >> Latest builds:
> >> http://copr-fe.cloud.fedoraproject.org/coprs/sagitter/petsc/build/464373/
> > 
> > I've mentioned this a few times - You haven't been able to run the sequntial test in valgrind?
> 
> It was ran in valgrind:
> https://copr-be.cloud.fedoraproject.org/results/sagitter/petsc/fedora-25-i386/00464297-petsc/build.log.gz

Hm - but thats an older build - and its not crashing.

The current builds that are crashing are not run with valgrind..
https://copr-be.cloud.fedoraproject.org/results/sagitter/petsc/fedora-25-i386/00464373-petsc/build.log.gz
https://copr-be.cloud.fedoraproject.org/results/sagitter/petsc/fedora-23-i386/00464373-petsc/build.log.gz
etc..

> > # you are missing 'CXX=g++' - but then --with-gnu-compilers=1 is guessing and compensating. I would remove --with-gnu-compilers --with-vendor-compilers - and just list all compilers [CC, FC, CXX].
> > 
> 
> g++ is not used, that's because i skipped it.
> I will remove --with-gnu-compilers --with-vendor-compilers.

I don't understand what you are saying here.

If you don't specify 'CC' - then PETSc configure is guessing and picking up something. [in this case g++ is correct].
To be consistant - its best if you secify all - CC,CXX,FC, CFLAGS,CXXFLAGS,FFLAGS etc.. [for all builds]

>>>>>>>>>
Compilers:
  C Compiler:         gcc -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables   -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables 
  C++ Compiler:       g++ -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables      
  Fortran Compiler:   gfortran -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables  
<<<<<<<<

> > '--with-blas-lapack-lib=-L/usr/lib -lopenblas'
> > 
> > # I would remove -L/usr/lib from everywhere..
> 
> This is another issue for me.
> Lib list of Sundials is
> 'libsundials_cvode.a','libsundials_nvecserial.a','libsundials_nvecparallel.a']
> 
> If i'm not wrong, linking 'libsundials_nvecparallel.so' will require
> 'libsundials_cvode.so' but they are located in different directories
> (/usr/lib/openmpi/lib and /usr/lib).
> 
> In this case, do i need using
> 
> --with-sundials-lib="-L/usr/lib -lsundials_cvode  -L/usr/lib/openmpi/lib
> -lsundials_nvecparallel"
> 
> ?

Don't really know the sundials libary depencency - they are listed in petsc as:
    self.liblist      = [['libsundials_cvode.a','libsundials_nvecserial.a','libsundials_nvecparallel.a']] #currently only support CVODE                                                            

However - even though they are in different dirs - you don't have to specify -L/usr/lib - as gcc with
automatically look at this location at the end.

For eg: - if you specify:

--with-sundials-lib="-L/usr/lib/openmpi/lib -lsundials_cvode.a -lsundials_nvecserial -lsundials_nvecparallel"

Then gcc will process the libraries in the following order:

1. look for -lsundials_cvode.a in /usr/lib/openmpi/lib
- if not found - look for it in /usr/lib

2. look for -lsundials_nvecserial in /usr/lib/openmpi/lib
- if not found look for it in /usr/lib

3. look for -lsundials_nvecparallel in /usr/lib/openmpi/lib
- if not found look for it in /usr/lib


When you have a long link line - it can get compilcated.

Using libraries: -L/builddir/build/BUILD/petsc-3.7.4/buildopenmpi_dir/i386/lib -L/builddir/build/BUILD/petsc-3.7.4/buildopenmpi_dir/i386/lib  -lpetsc -L/usr/lib/openmpi/lib -lmumps_common -ldmumps -lpord -lHYPRE -lscalapack -lsundials_nvecparallel -L/usr/lib -lsundials_cvode -lopenblas -lptscotch -lscotch -lptscotcherr -lscotcherr -lhdf5 -lX11 -lcgns -lhdf5 -lm -ldl 

Here - because -L/builddir/build/BUILD/petsc-3.7.4/buildopenmpi_dir/i386/lib is listed before -L/usr/lib - nothing bad happens..

but if somehow the order gets swapped:

-L/usr/lib -lopenblas -L/builddir/build/BUILD/petsc-3.7.4/buildopenmpi_dir/i386/lib -lpetsc -lmumps_commonetc//

1. gcc looks for -lopenblas in /usr/lib
2. gcc looks for -lpetsc in /usr/lib [before looking at /builddir/build/BUILD/petsc-3.7.4/buildopenmpi_dir/i386/lib]
   and finds the wrong one in /usr/lib

So its best to avoid listing /usr/lib [which is also /usr/lib64 on 64bit builds]

Satish