[petsc-users] Error while building PETSc with MATLAB

Satish Balay balay at mcs.anl.gov
Sat Sep 2 11:42:08 CDT 2023


Please don't cc: both petsc-users and petsc-maint [reverting thread to petsc-users only]

I'm not sure what is happening here. Can you send the corresponding configure.log, make.log [compressed]?

Here is my attempt to reproduce (with petsc-3.13) on Ubuntu-20.04, with Matlab-R2021a - and that works:

balay at compute-386-07:/scratch/balay/petsc$ git branch
  main
  release
* release-3.13
balay at compute-386-07:/scratch/balay/petsc$ module load matlab/R2021a
balay at compute-386-07:/scratch/balay/petsc$ which matlab
/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/bin/matlab
balay at compute-386-07:/scratch/balay/petsc$ ./configure --with-mpi-dir=/nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/ COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 PETSC_ARCH=linux-opt --with-debugging=0 --with-x=0 --with-matlab-dir=/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a --with-blaslapack-dir=/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a --known-64-bit-blas-indices=1 --with-matlab-engine=1
<snip>
Compilers:                                                                                                                                                                                                                                    
  C Compiler:         /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/bin/mpicc  -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -O3 
    Version: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
  C++ Compiler:       /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/bin/mpicxx  -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -O3  -fPIC  
    Version: g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
  Fortran Compiler:   /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/bin/mpif90  -fPIC -Wall -ffree-line-length-0 -Wno-unused-dummy-argument -O3  
    Version: GNU Fortran (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Linkers:
  Shared linker:   /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/bin/mpicc  -shared  -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -O3
  Dynamic linker:   /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/bin/mpicc  -shared  -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -O3
  Libraries linked against:   -lquadmath -lstdc++ -ldl 
make:
  Version:  4.2
  /usr/bin/make
BlasLapack:
  Library:  /nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/bin/glnxa64/mkl.so -Wl,-rpath,/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/sys/os/glnxa64 -L/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/sys/os/glnxa64 -liomp5 -lpthread
  uses OpenMP; use export OMP_NUM_THREADS=<p> or -omp_num_threads <p> to control the number of threads
  uses 8 byte integers
MPI:
  Version:  4
  Includes: -I/nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/include
  Mpiexec: /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/bin/mpiexec
  MPICH_NUMVERSION: 40002300
pthread:
  Library:  -lpthread
cmake:
  Version:  3.16.3
  /usr/bin/cmake
regex:
Matlab:
  Includes: -I/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/extern/include
  /nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a
MatlabEngine:
  Library:  -Wl,-rpath,/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/sys/os/glnxa64:/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/bin/glnxa64:/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/extern/lib/glnxa64 -L/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/bin/glnxa64 -L/nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/extern/lib/glnxa64 -leng -lmex -lmx -lmat -lut -licudata -licui18n -licuuc
sowing:
  Version:  1.1.25
  /scratch/balay/petsc/linux-opt/bin/bfort
valgrind:
  Language used to compile PETSc: C
PETSc:
  PETSC_ARCH: linux-opt
  PETSC_DIR: /scratch/balay/petsc
  Scalar type: real
  Precision: double
  Support for __float128
  Integer size: 4 bytes
  shared libraries: enabled
  Memory alignment from malloc(): 16 bytes
xxx=========================================================================xxx
 Configure stage complete. Now build PETSc libraries with:
   make PETSC_DIR=/scratch/balay/petsc PETSC_ARCH=linux-opt all
xxx=========================================================================xxx
balay at compute-386-07:/scratch/balay/petsc$ make                                                                                                                                                                                               

<snip>
          FC linux-opt/obj/tao/f90-mod/petsctaomod.o
     CLINKER linux-opt/lib/libpetsc.so.3.13.6
BEGINNING TO COMPILE MATLAB INTERFACE
Building with 'gcc'.
/scratch/balay/petsc/src/sys/classes/viewer/impls/socket/matlab/bread.c: In function ‘PetscBinaryWrite’:
/scratch/balay/petsc/src/sys/classes/viewer/impls/socket/matlab/bread.c:167:6: warning: ‘err’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  167 |   if (err < 0) PETSC_MEX_ERROR("Error writing to socket\n");
      |      ^

MEX completed successfully.
Building with 'gcc'.
MEX completed successfully.
Building with 'gcc'.
MEX completed successfully.
Building with 'gcc'.
/scratch/balay/petsc/src/sys/classes/viewer/impls/socket/matlab/bread.c: In function ‘PetscBinaryWrite’:
/scratch/balay/petsc/src/sys/classes/viewer/impls/socket/matlab/bread.c:167:6: warning: ‘err’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  167 |   if (err < 0) PETSC_MEX_ERROR("Error writing to socket\n");
      |      ^

MEX completed successfully.
=========================================
Now to check if the libraries are working do:
make PETSC_DIR=/scratch/balay/petsc PETSC_ARCH=linux-opt check
=========================================
balay at compute-386-07:/scratch/balay/petsc$ make test query=requires queryval=matlab
Using MAKEFLAGS: -- queryval=matlab query=requires
          CC linux-opt/tests/ksp/ksp/tutorials/ex72.o
     CLINKER linux-opt/tests/ksp/ksp/tutorials/ex72
        TEST linux-opt/tests/counts/ksp_ksp_tutorials-ex72_12.counts
not ok ksp_ksp_tutorials-ex72_12 # Error code: 1
#	../ex72: /nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/sys/os/glnxa64/libgfortran.so.5: version `GFORTRAN_9' not found (required by /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/lib/libmpifort.so.12)
 ok ksp_ksp_tutorials-ex72_12 # SKIP Command failed so no diff
          CC linux-opt/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test.o
     CLINKER linux-opt/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test
        TEST linux-opt/tests/counts/tao_leastsquares_tutorials_matlab-matlab_ls_test.counts
not ok tao_leastsquares_tutorials_matlab-matlab_ls_test # Error code: 1
#	../matlab_ls_test: /nfs/gce/software/custom/linux-ubuntu20.04-x86_64/matlab/R2021a/sys/os/glnxa64/libgfortran.so.5: version `GFORTRAN_9' not found (required by /nfs/gce/projects/petsc/soft/u20.04/mpich-4.0.2/lib/libmpifort.so.12)
 ok tao_leastsquares_tutorials_matlab-matlab_ls_test # SKIP Command failed so no diff
<snip>
balay at compute-386-07:/scratch/balay/petsc$ LD_PRELOAD=/lib/x86_64-linux-gnu/libgfortran.so.5 make test query=requires queryval=matlab
Using MAKEFLAGS: -- queryval=matlab query=requires
        TEST linux-opt/tests/counts/ksp_ksp_tutorials-ex72_12.counts
 ok ksp_ksp_tutorials-ex72_12
 ok diff-ksp_ksp_tutorials-ex72_12
        TEST linux-opt/tests/counts/tao_leastsquares_tutorials_matlab-matlab_ls_test.counts
 ok tao_leastsquares_tutorials_matlab-matlab_ls_test
 ok diff-tao_leastsquares_tutorials_matlab-matlab_ls_test

# -------------
#   Summary    
# -------------
# success 4/4 tests (100.0%)
# failed 0/4 tests (0.0%)
# todo 0/4 tests (0.0%)
# skip 0/4 tests (0.0%)
#
# Wall clock time for tests: 27 sec
# Approximate CPU time (not incl. build time): 15.190000000000001 sec
#
# Timing summary (actual test time / total CPU time): 
#   tao_leastsquares_tutorials_matlab-matlab_ls_test: 6.47 sec / 7.54 sec
#   ksp_ksp_tutorials-ex72_12: 6.46 sec / 7.65 sec
balay at compute-386-07:/scratch/balay/petsc$ 





On Sat, 2 Sep 2023, INTURU SRINIVAS 20PHD0548 wrote:

> Based on the suggestions given by Sathish, I have tried to configure petsc
> 3.13.4 with Matlab-R2020b. But, it was not successful. After running make
> -j 10 test command I have got the following error message.
> 
> not ok tao_leastsquares_tutorials_matlab-matlab_ls_test # Error code: 124
> # Running problem 5
> # [0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> # [0]PETSC ERROR: Error in external library
> # [0]PETSC ERROR: Unable to start MATLAB engine on
> # [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
> # [0]PETSC ERROR: Petsc Release Version 3.13.4, Aug 01, 2020
> # [0]PETSC ERROR: ../matlab_ls_test on a linux-opt named MB108SMEC028 by
> vit Sat Sep  2 12:27:22 2023
> # [0]PETSC ERROR: Configure options
> --with-mpi-dir=/home/vit/sfw/linux/openmpi/4.1.4 COPTFLAGS=-O3
> CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 PETSC_ARCH=linux-opt --with-debugging=0
> --with-x=0 --with-matlab-dir=/usr/local/MATLAB/R2020b
> --with-matlab-engine=1 --with-blaslapack-dir=/usr/local/MATLAB/R2020b
> --known-64-bit-blas-indices=1
> # [0]PETSC ERROR: #1 PetscMatlabEngineCreate() line 67 in
> /home/vit/sfw/petsc/3.13.4/src/sys/classes/matlabengine/matlab.c
> # [0]PETSC ERROR: #2 main() line 126 in
> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c
> # [0]PETSC ERROR: PETSc Option Table entries:
> # [0]PETSC ERROR: -prob_id 5
> # [0]PETSC ERROR: -tao_smonitor
> # [0]PETSC ERROR: ----------------End of Error Message -------send entire
> error message to petsc-maint at mcs.anl.gov----------
> # --------------------------------------------------------------------------
> # MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_SELF
> # with errorcode 126076.
> #
> # NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
> # You may or may not see output from other processes, depending on
> # exactly when Open MPI kills them.
> # --------------------------------------------------------------------------
> 
> I request you to help me to sort out this error.
> 
> Thanks
> Srinivas
> 
> 
> On Fri, Sep 1, 2023 at 11:42 PM Amneet Bhalla <mail2amneet at gmail.com> wrote:
> 
> > I think there should be two hyphens in
> > -with-blaslapack-dir=/usr/local/MATLAB/R2020b
> >
> > i.e.:
> >
> > --with-blaslapack-dir=/usr/local/MATLAB/R2020b
> >
> > On Fri, Sep 1, 2023 at 10:25 AM INTURU SRINIVAS 20PHD0548 via petsc-users <
> > petsc-users at mcs.anl.gov> wrote:
> >
> >> Thank you Sathish.I will try this
> >>
> >> On Fri, Sep 1, 2023, 22:53 Satish Balay <balay at mcs.anl.gov> wrote:
> >>
> >>> yes [and remove fblaslapack. don't know if hypre will work here].
> >>>
> >>> i.e:
> >>>
> >>> ./configure --with-mpi-dir=/home/vit/sfw/linux/openmpi/4.1.4
> >>> COPTFLAGS=-O3 CXXOPTFLAGS=-O3 FOPTFLAGS=-O3 PETSC_ARCH=linux-opt
> >>> --with-debugging=0 --with-x=0 \
> >>> --with-matlab-dir=/usr/local/MATLAB/R2020b --with-matlab-engine=1
> >>> -with-blaslapack-dir=/usr/local/MATLAB/R2020b --known-64-bit-blas-indices=1
> >>>
> >>> Satish
> >>>
> >>> On Fri, 1 Sep 2023, INTURU SRINIVAS 20PHD0548 via petsc-users wrote:
> >>>
> >>> > Hi Satish,
> >>> >
> >>> > -with-blaslapack-dir=/path/to/matlab_dir
> >>> > --known-64-bit-blas-indices=1
> >>> >
> >>> > Is this what you are suggesting?
> >>> >
> >>> > On Fri, Sep 1, 2023, 20:42 Satish Balay <balay at mcs.anl.gov> wrote:
> >>> >
> >>> > > Also:
> >>> > >
> >>> > >     '-known-64-bit-blas-indices=1',
> >>> > >
> >>> > > Note: most externalpackages might not work in this mode.
> >>> > >
> >>> > > [we can't really over come such dependency/conflicts across packages]
> >>> > >
> >>> > > Satish
> >>> > >
> >>> > > On Fri, 1 Sep 2023, Satish Balay via petsc-users wrote:
> >>> > >
> >>> > > > Here is the matlab test that runs in CI
> >>> > > >
> >>> > > > https://gitlab.com/petsc/petsc/-/jobs/4904566768
> >>> > > >
> >>> > > > config/examples/arch-ci-linux-matlab-ilp64.py
> >>> > > >
> >>> > > > # Note: regular BLAS [with 32-bit integers] conflict with
> >>> > > > # MATLAB BLAS - hence requiring -known-64-bit-blas-indices=1
> >>> > > >
> >>> > > > Ah - so you need to use the ilp64 blas/lapack with matlab  - to
> >>> have a
> >>> > > compatible build
> >>> > > >
> >>> > > >     '--with-blaslapack-dir='+matlab_dir,
> >>> > > >
> >>> > > > Satish
> >>> > > >
> >>> > > >
> >>> > > > On Fri, 1 Sep 2023, INTURU SRINIVAS 20PHD0548 via petsc-users
> >>> wrote:
> >>> > > >
> >>> > > > > Hi Amneet,
> >>> > > > >
> >>> > > > > Without libmesh, even for PETSc with MATLAB is not working. It is
> >>> > > showing
> >>> > > > > error for both 3.13.4 and 3.17.5 versions.
> >>> > > > >
> >>> > > > > I am trying to install IBAMR in HPC cluster with libmesh and
> >>> this is
> >>> > > for
> >>> > > > > general usage not only for WEC. This I tried without linking
> >>> Matlab
> >>> > > with
> >>> > > > > PETSc. Here also I got the error while configuring libmesh 1.6.2
> >>> that
> >>> > > PETSc
> >>> > > > > was not found but --enable-petsc-required is specified.
> >>> > > > >
> >>> > > > > The reason for specifically asking for libmesh with PETSc and
> >>> Matlab
> >>> > > is in
> >>> > > > > the cfd-mpc-wecs application for complicated geometries it is
> >>> > > mentioned to
> >>> > > > > build cfd-mpc-wecs with libmesh construct.
> >>> > > > >
> >>> > > > > For the past 10 days I am trying these various things and ended
> >>> up with
> >>> > > > > errors.
> >>> > > > > I want to solve this and proceed further to do my research work.
> >>> > > > >
> >>> > > > > Please help me to solve this as soon as possible.
> >>> > > > >
> >>> > > > >
> >>> > > > > Thanks and Regards
> >>> > > > > Srinivas
> >>> > > > >
> >>> > > > > On Fri, Sep 1, 2023, 19:48 Amneet Bhalla <mail2amneet at gmail.com>
> >>> > > wrote:
> >>> > > > >
> >>> > > > > > Hi Srinivas,
> >>> > > > > >
> >>> > > > > > As discussed earlier you don’t need libMesh for the WEC
> >>> application.
> >>> > > You
> >>> > > > > > can just work with PETSc 3.17.5 that builds with Matlab.
> >>> > > > > >
> >>> > > > > > Do you have a specific reason for wanting to build libMesh?
> >>> > > > > >
> >>> > > > > > Thanks,
> >>> > > > > > —Amneet
> >>> > > > > >
> >>> > > > > > On Thu, Aug 31, 2023 at 10:16 PM INTURU SRINIVAS 20PHD0548 via
> >>> > > petsc-users
> >>> > > > > > <petsc-users at mcs.anl.gov> wrote:
> >>> > > > > >
> >>> > > > > >> Hi,
> >>> > > > > >>
> >>> > > > > >> I ran "make all" for petsc 3.13.4 by removing all occurrences
> >>> of
> >>> > > "-lut
> >>> > > > > >> -licudata -licui18n -licuuc". When I ran "make test" got the
> >>> > > following
> >>> > > > > >> errors
> >>> > > > > >> in petsc/3.13.4/linux-debug:
> >>> > > > > >> TEST
> >>> > > > > >>
> >>> > >
> >>> linux-debug/tests/counts/tao_leastsquares_tutorials_matlab-matlab_ls_test.counts
> >>> > > > > >> not ok tao_leastsquares_tutorials_matlab-matlab_ls_test #
> >>> Error
> >>> > > code: 124
> >>> > > > > >> # Running problem 5
> >>> > > > > >> # [0]PETSC ERROR: --------------------- Error Message
> >>> > > > > >> --------------------------------------------------------------
> >>> > > > > >> # [0]PETSC ERROR: Error in external library
> >>> > > > > >> # [0]PETSC ERROR: Unable to start MATLAB engine on
> >>> > > > > >> # [0]PETSC ERROR: See
> >>> > > > > >> https://www.mcs.anl.gov/petsc/documentation/faq.html for
> >>> trouble
> >>> > > > > >> shooting.
> >>> > > > > >> # [0]PETSC ERROR: Petsc Release Version 3.13.4, Aug 01, 2020
> >>> > > > > >> # [0]PETSC ERROR: ../matlab_ls_test on a linux-debug named
> >>> > > MB108SMEC028
> >>> > > > > >> by vit Fri Sep  1 10:01:11 2023
> >>> > > > > >> # [0]PETSC ERROR: Configure options
> >>> > > > > >> --CC=/home/vit/sfw/linux/openmpi/4.1.4/bin/mpicc
> >>> > > > > >> --CXX=/home/vit/sfw/linux/openmpi/4.1.4/bin/mpicxx
> >>> > > > > >> --FC=/home/vit/sfw/linux/openmpi/4.1.4/bin/mpif90
> >>> --with-debugging=1
> >>> > > > > >> --download-hypre=1 --download-fblaslapack=1 --with-x=0
> >>> > > > > >> --with-matlab-dir=/usr/local/MATLAB/R2020b/
> >>> --with-matlab-engine=1
> >>> > > > > >>
> >>> --with-matlab-engine-dir=/usr/local/MATLAB/R2020b/extern/engines/
> >>> > > > > >> # [0]PETSC ERROR: #1 PetscMatlabEngineCreate() line 67 in
> >>> > > > > >>
> >>> /home/vit/sfw/petsc/3.13.4/src/sys/classes/matlabengine/matlab.c
> >>> > > > > >> # [0]PETSC ERROR: #2 main() line 126 in
> >>> > > > > >>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c
> >>> > > > > >> # [0]PETSC ERROR: PETSc Option Table entries:
> >>> > > > > >> # [0]PETSC ERROR: -prob_id 5
> >>> > > > > >> # [0]PETSC ERROR: -tao_smonitor
> >>> > > > > >> # [0]PETSC ERROR: ----------------End of Error Message
> >>> -------send
> >>> > > entire
> >>> > > > > >> error message to petsc-maint at mcs.anl.gov----------
> >>> > > > > >> #
> >>> > > > > >>
> >>> > >
> >>> --------------------------------------------------------------------------
> >>> > > > > >> # MPI_ABORT was invoked on rank 0 in communicator
> >>> MPI_COMM_SELF
> >>> > > > > >> # with errorcode 126076.
> >>> > > > > >> #
> >>> > > > > >> # NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI
> >>> > > processes.
> >>> > > > > >> # You may or may not see output from other processes,
> >>> depending on
> >>> > > > > >> # exactly when Open MPI kills them.
> >>> > > > > >> #
> >>> > > > > >>
> >>> > >
> >>> --------------------------------------------------------------------------
> >>> > > > > >>  ok tao_leastsquares_tutorials_matlab-matlab_ls_test # SKIP
> >>> Command
> >>> > > > > >> failed so no diff
> >>> > > > > >>
> >>> > > > > >> in petsc/3.13.4/linux-opt
> >>> > > > > >>  TEST
> >>> > > > > >>
> >>> > >
> >>> linux-opt/tests/counts/tao_leastsquares_tutorials_matlab-matlab_ls_test.counts
> >>> > > > > >> not ok tao_leastsquares_tutorials_matlab-matlab_ls_test #
> >>> Error
> >>> > > code: 124
> >>> > > > > >> # Running problem 5
> >>> > > > > >> # [0]PETSC ERROR: --------------------- Error Message
> >>> > > > > >> --------------------------------------------------------------
> >>> > > > > >> # [0]PETSC ERROR: Error in external library
> >>> > > > > >> # [0]PETSC ERROR: Unable to start MATLAB engine on
> >>> > > > > >> # [0]PETSC ERROR: See
> >>> > > > > >> https://www.mcs.anl.gov/petsc/documentation/faq.html for
> >>> trouble
> >>> > > > > >> shooting.
> >>> > > > > >> # [0]PETSC ERROR: Petsc Release Version 3.13.4, Aug 01, 2020
> >>> > > > > >> # [0]PETSC ERROR: ../matlab_ls_test on a linux-opt named
> >>> > > MB108SMEC028 by
> >>> > > > > >> vit Fri Sep  1 10:34:12 2023
> >>> > > > > >> # [0]PETSC ERROR: Configure options
> >>> > > > > >> --CC=/home/vit/sfw/linux/openmpi/4.1.4/bin/mpicc
> >>> > > > > >> --CXX=/home/vit/sfw/linux/openmpi/4.1.4/bin/mpicxx
> >>> > > > > >> --FC=/home/vit/sfw/linux/openmpi/4.1.4/bin/mpif90
> >>> --COPTFLAGS=-O3
> >>> > > > > >> --CXXOPTFLAGS=-O3 --FOPTFLAGS=-O3 --PETSC_ARCH=linux-opt
> >>> > > --with-debugging=0
> >>> > > > > >> --download-hypre=1 --with-x=0 --download-fblaslapack=1
> >>> > > > > >> --with-matlab-dir=/usr/local/MATLAB/R2020b/
> >>> --with-matlab-engine=1
> >>> > > > > >>
> >>> --with-matlab-engine-dir=/usr/local/MATLAB/R2020b/extern/engines/
> >>> > > > > >> # [0]PETSC ERROR: #1 PetscMatlabEngineCreate() line 67 in
> >>> > > > > >>
> >>> /home/vit/sfw/petsc/3.13.4/src/sys/classes/matlabengine/matlab.c
> >>> > > > > >> # [0]PETSC ERROR: #2 main() line 126 in
> >>> > > > > >>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c
> >>> > > > > >> # [0]PETSC ERROR: PETSc Option Table entries:
> >>> > > > > >> # [0]PETSC ERROR: -prob_id 5
> >>> > > > > >> # [0]PETSC ERROR: -tao_smonitor
> >>> > > > > >> # [0]PETSC ERROR: ----------------End of Error Message
> >>> -------send
> >>> > > entire
> >>> > > > > >> error message to petsc-maint at mcs.anl.gov----------
> >>> > > > > >> #
> >>> > > > > >>
> >>> > >
> >>> --------------------------------------------------------------------------
> >>> > > > > >> # MPI_ABORT was invoked on rank 0 in communicator
> >>> MPI_COMM_SELF
> >>> > > > > >> # with errorcode 126076.
> >>> > > > > >> #
> >>> > > > > >> # NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI
> >>> > > processes.
> >>> > > > > >> # You may or may not see output from other processes,
> >>> depending on
> >>> > > > > >> # exactly when Open MPI kills them.
> >>> > > > > >> #
> >>> > > > > >>
> >>> > >
> >>> --------------------------------------------------------------------------
> >>> > > > > >>  ok tao_leastsquares_tutorials_matlab-matlab_ls_test # SKIP
> >>> Command
> >>> > > > > >> failed so no diff.
> >>> > > > > >>
> >>> > > > > >> The same errors occurred for petsc 3.17.5 version. I am
> >>> sharing the
> >>> > > > > >> config.log and make.log for both petsc 3.13.4 and petsc
> >>> 3.17.5.
> >>> > > Please find
> >>> > > > > >> the attachment and do the needful.
> >>> > > > > >>
> >>> > > > > >> Thanks and regards
> >>> > > > > >> Srinivas
> >>> > > > > >>
> >>> > > > > >> On Tue, Aug 29, 2023 at 10:56 PM Satish Balay <
> >>> balay at mcs.anl.gov>
> >>> > > wrote:
> >>> > > > > >>
> >>> > > > > >>> Well - you sent in libmesh log not petsc's
> >>> configure.log/make.log
> >>> > > for
> >>> > > > > >>> petsc-3.17
> >>> > > > > >>>
> >>> > > > > >>> Anyway - with petsc-3.13 - you have:
> >>> > > > > >>>
> >>> > > > > >>> >>>>
> >>> > > > > >>>
> >>> > > > > >>> Matlab:
> >>> > > > > >>>   Includes: -I/usr/local/MATLAB/R2020b/extern/include
> >>> > > > > >>>   /usr/local/MATLAB/R2020b
> >>> > > > > >>> MatlabEngine:
> >>> > > > > >>>   Library:
> >>> > > > > >>>
> >>> > >
> >>> -Wl,-rpath,/usr/local/MATLAB/R2020b/sys/os/glnxa64:/usr/local/MATLAB/R2020b/bin/glnxa64:/usr/local/MATLAB/R2020b/extern/lib/glnxa64
> >>> > > > > >>> -L/usr/local/MATLAB/R2020b/bin/glnxa64
> >>> > > > > >>> -L/usr/local/MATLAB/R2020b/extern/lib/glnxa64 -leng -lmex
> >>> -lmx
> >>> > > -lmat -lut
> >>> > > > > >>> -licudata -licui18n -licuuc
> >>> > > > > >>>   Language used to compile PETSc: C
> >>> > > > > >>> <<<<<
> >>> > > > > >>>
> >>> > > > > >>> With petsc-3.19 (and matlab-R2022a) - we are seeing:
> >>> > > > > >>>
> >>> > > > > >>> https://gitlab.com/petsc/petsc/-/jobs/4904566768
> >>> > > > > >>>
> >>> > > > > >>> >>>
> >>> > > > > >>> Matlab:
> >>> > > > > >>>   Includes:
> >>> > > > > >>>
> >>> > >
> >>> -I/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/matlab/R2022a/extern/include
> >>> > > > > >>>   Libraries:
> >>> > > > > >>>
> >>> > >
> >>> -Wl,-rpath,/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/matlab/R2022a/bin/glnxa64
> >>> > > > > >>>
> >>> > >
> >>> -L/nfs/gce/software/custom/linux-ubuntu22.04-x86_64/matlab/R2022a/bin/glnxa64
> >>> > > > > >>> -leng -lmex -lmx -lmat
> >>> > > > > >>>   Executable:
> >>> > > > > >>>
> >>> /nfs/gce/software/custom/linux-ubuntu22.04-x86_64/matlab/R2022a
> >>> > > > > >>>   mex:
> >>> > > > > >>>
> >>> > >
> >>> /nfs/gce/software/custom/linux-ubuntu22.04-x86_64/matlab/R2022a/bin/mex
> >>> > > > > >>>   matlab:
> >>> > > > > >>>
> >>> > >
> >>> /nfs/gce/software/custom/linux-ubuntu22.04-x86_64/matlab/R2022a/bin/matlab
> >>> > > > > >>> -glnxa64
> >>> > > > > >>> <<<
> >>> > > > > >>>
> >>> > > > > >>> I.e "-lut -licudata -licui18n -licuuc" are not preset here.
> >>> This
> >>> > > might
> >>> > > > > >>> be a change wrt newer matlab versions.
> >>> > > > > >>>
> >>> > > > > >>> You can:
> >>> > > > > >>>
> >>> > > > > >>> - edit
> >>> > > > > >>>
> >>> /home/vit/sfw/petsc/3.13.4/linux-opt/lib/petsc/conf/petscvariables
> >>> > > and
> >>> > > > > >>> remove all occurrences of "-lut -licudata -licui18n -licuuc"
> >>> > > > > >>> - now run 'make all' in '/home/vit/sfw/petsc/3.13.4'
> >>> > > > > >>>
> >>> > > > > >>> And see if the build works now.
> >>> > > > > >>>
> >>> > > > > >>> Satish
> >>> > > > > >>>
> >>> > > > > >>> On Tue, 29 Aug 2023, INTURU SRINIVAS 20PHD0548 via
> >>> petsc-users
> >>> > > wrote:
> >>> > > > > >>>
> >>> > > > > >>> > I am sharing the log files while building petsc3.13.4 with
> >>> > > matlab and
> >>> > > > > >>> also
> >>> > > > > >>> > the log file while building libmesh with petsc3.17.5 and
> >>> matlab.
> >>> > > > > >>> Building
> >>> > > > > >>> > petsc 3.17.5 with matlab was done successfully. But
> >>> libmesh is
> >>> > > not
> >>> > > > > >>> able to
> >>> > > > > >>> > find the petsc
> >>> > > > > >>> > Please find the attachments.
> >>> > > > > >>> >
> >>> > > > > >>> > On Tue, Aug 29, 2023 at 7:31 PM Satish Balay <
> >>> balay at mcs.anl.gov>
> >>> > > > > >>> wrote:
> >>> > > > > >>> >
> >>> > > > > >>> > > Send configure.log, make.log from both petsc-3.13 and
> >>> 3.17 [or
> >>> > > 3.19].
> >>> > > > > >>> > >
> >>> > > > > >>> > > [you can gzip them to make the logs friendly to mailing
> >>> list -
> >>> > > or
> >>> > > > > >>> send
> >>> > > > > >>> > > them to petsc-maint]
> >>> > > > > >>> > >
> >>> > > > > >>> > > And does test suite work with 3.17? [or 3.19?]
> >>> > > > > >>> > >
> >>> > > > > >>> > > Satish
> >>> > > > > >>> > >
> >>> > > > > >>> > > On Tue, 29 Aug 2023, INTURU SRINIVAS 20PHD0548 via
> >>> petsc-users
> >>> > > wrote:
> >>> > > > > >>> > >
> >>> > > > > >>> > > > I am sharing the make.log file while building
> >>> petsc-3.13.4
> >>> > > with
> >>> > > > > >>> Matlab.
> >>> > > > > >>> > > > Please find the attachment and do the needful.
> >>> > > > > >>> > > >
> >>> > > > > >>> > > > On Tue, Aug 29, 2023 at 10:19 AM INTURU SRINIVAS
> >>> 20PHD0548 <
> >>> > > > > >>> > > > inturu.srinivas2020 at vitstudent.ac.in> wrote:
> >>> > > > > >>> > > >
> >>> > > > > >>> > > > > I tried with petsc-3.17.5. During building of
> >>> libmesh, the
> >>> > > error
> >>> > > > > >>> shows
> >>> > > > > >>> > > > > petsc was not found
> >>> > > > > >>> > > > >
> >>> > > > > >>> > > > > On Mon, Aug 28, 2023 at 9:43 PM Satish Balay <
> >>> > > balay at mcs.anl.gov>
> >>> > > > > >>> > > wrote:
> >>> > > > > >>> > > > >
> >>> > > > > >>> > > > >> https://ibamr.github.io/linux says petsc-3.17
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > > > >> Here you are using 3.13
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > > > >> Can you retry with petsc-3.17.5?
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > > > >> Satish
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > > > >> On Mon, 28 Aug 2023, INTURU SRINIVAS 20PHD0548 via
> >>> > > petsc-users
> >>> > > > > >>> wrote:
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > > > >> > Hello,
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> > I want to build PETSc with MATLAB for working on
> >>> the
> >>> > > > > >>> simulation
> >>> > > > > >>> > > using
> >>> > > > > >>> > > > >> IBAMR
> >>> > > > > >>> > > > >> > open software. While building the PETSc, using the
> >>> > > following
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> > export PETSC_DIR=$PWD
> >>> > > > > >>> > > > >> > export PETSC_ARCH=linux-debug
> >>> > > > > >>> > > > >> > ./configure \
> >>> > > > > >>> > > > >> >   --CC=$HOME/sfw/linux/openmpi/4.1.4/bin/mpicc \
> >>> > > > > >>> > > > >> >   --CXX=$HOME/sfw/linux/openmpi/4.1.4/bin/mpicxx \
> >>> > > > > >>> > > > >> >   --FC=$HOME/sfw/linux/openmpi/4.1.4/bin/mpif90 \
> >>> > > > > >>> > > > >> >   --with-debugging=1 \
> >>> > > > > >>> > > > >> >   --download-hypre=1 \
> >>> > > > > >>> > > > >> >   --download-fblaslapack=1 \
> >>> > > > > >>> > > > >> >   --with-x=0 \
> >>> > > > > >>> > > > >> >   --with-matlab-dir=/usr/local/MATLAB/R2020b/
> >>> > > > > >>> > > > >> >   --with-matlab-engine=1
> >>> > > > > >>> > > > >> >
> >>> > > > > >>>
> >>> --with-matlab-engine-dir=/usr/local/MATLAB/R2020b/extern/engines/
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> > make -j4
> >>> > > > > >>> > > > >> > make -j4 test
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> > I got the following error
> >>> > > > > >>> > > > >> > CLINKER
> >>> > > > > >>> > > > >>
> >>> > > > > >>>
> >>> linux-debug/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> linux-debug/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test.o: in
> >>> > > > > >>> > > > >> > function `EvaluateResidual':
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:32:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEnginePut'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:33:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineEvaluate'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:35:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGet'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> linux-debug/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test.o: in
> >>> > > > > >>> > > > >> > function `EvaluateJacobian':
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:46:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEnginePut'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:47:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineEvaluate'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:49:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGet'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> linux-debug/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test.o: in
> >>> > > > > >>> > > > >> > function `TaoPounders':
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:75:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGet'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> linux-debug/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test.o: in
> >>> > > > > >>> > > > >> > function `main':
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:126:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineCreate'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:127:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineEvaluate'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:139:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineEvaluate'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:140:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGetArray'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:142:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGetArray'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:144:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGetArray'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:146:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGetArray'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:148:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineGetArray'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:154:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineEvaluate'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:157:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineEvaluate'
> >>> > > > > >>> > > > >> > /usr/bin/ld:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > >
> >>> > > > > >>>
> >>> > >
> >>> /home/vit/sfw/petsc/3.13.4/src/tao/leastsquares/tutorials/matlab/matlab_ls_test.c:158:
> >>> > > > > >>> > > > >> > undefined reference to `PetscMatlabEngineDestroy'
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> > collect2: error: ld returned 1 exit status
> >>> > > > > >>> > > > >> > make: *** [gmakefile.test:185:
> >>> > > > > >>> > > > >> >
> >>> > > > > >>>
> >>> linux-debug/tests/tao/leastsquares/tutorials/matlab/matlab_ls_test]
> >>> > > > > >>> > > > >> Error 1
> >>> > > > > >>> > > > >> > make: *** Waiting for unfinished jobs....
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> > Please help me to solve this issue
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> > Thank you
> >>> > > > > >>> > > > >> > Srinivas
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >> >
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > > > >>
> >>> > > > > >>> > > >
> >>> > > > > >>> > > >
> >>> > > > > >>> > >
> >>> > > > > >>> >
> >>> > > > > >>> >
> >>> > > > > >>>
> >>> > > > > >>
> >>> > > > > >>
> >>> > > > > >> *Disclaimer:This message was sent from Vellore Institute of
> >>> > > Technology.
> >>> > > > > >> The contents of this email may contain legally protected
> >>> > > confidential or
> >>> > > > > >> privileged information of “Vellore Institute of Technology”.
> >>> If
> >>> > > you are
> >>> > > > > >> not the intended recipient, you should not disseminate,
> >>> distribute
> >>> > > or copy
> >>> > > > > >> this e-mail. Please notify the sender immediately and destroy
> >>> all
> >>> > > copies of
> >>> > > > > >> this message and any attachments. If you have received this
> >>> email
> >>> > > in error,
> >>> > > > > >> please promptly notify the sender by reply email and delete
> >>> the
> >>> > > original
> >>> > > > > >> email and any backup copies without reading them.*
> >>> > > > > >>
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>>
> >>
> >>
> >> *Disclaimer:This message was sent from Vellore Institute of Technology.
> >> The contents of this email may contain legally protected confidential or
> >> privileged information of “Vellore Institute of Technology”.  If you are
> >> not the intended recipient, you should not disseminate, distribute or copy
> >> this e-mail. Please notify the sender immediately and destroy all copies of
> >> this message and any attachments. If you have received this email in error,
> >> please promptly notify the sender by reply email and delete the original
> >> email and any backup copies without reading them.*
> >>
> >
> >
> > --
> > --Amneet
> >
> >
> >
> >
> 
> 


More information about the petsc-users mailing list