[petsc-dev] Cannot locate file: share/petsc/datafiles/matrices/small

Satish Balay balay at mcs.anl.gov
Tue Sep 14 07:44:27 CDT 2021 

cc: Antonio

On Tue, 14 Sep 2021, Matthew Knepley wrote:

> Okay, we have to send this to OpenMPI. Volunteers?
> 
> Maybe we should note this in the FAQ, or installation, so we remember how
> to fix it if someone else asks?
> 
>   Thanks,
> 
>      Matt
> 
> On Tue, Sep 14, 2021 at 2:35 AM Stefano Zampini <stefano.zampini at gmail.com>
> wrote:
> 
> > I can reproduce it even with OpenMPI 4.1.1 on a different machine (Ubuntu
> > 18 + AMD milan + clang from AOCC)  and it is definitely an OpenMPI bug in
> > the vader BTL If I use tcp, everything runs smooth
> >
> > zampins at kanary:~/Devel/petsc$ cat
> > /home/zampins/local/etc/openmpi-mca-params.conf | grep btl
> > btl = tcp,self
> > zampins at kanary:~/Devel/petsc$ make -f gmakefile.test
> > vec_is_sf_tutorials-ex1_4
> > Using MAKEFLAGS:
> >         TEST arch-debug/tests/counts/vec_is_sf_tutorials-ex1_4.counts
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic
> >  ok
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create
> >  ok
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic
> >  ok
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate
> >
> >
> > zampins at kanary:~/Devel/petsc$ cat
> > /home/zampins/local/etc/openmpi-mca-params.conf | grep btl
> > btl = vader,tcp,self
> > zampins at kanary:~/Devel/petsc$ make -f gmakefile.test
> > vec_is_sf_tutorials-ex1_4
> > Using MAKEFLAGS:
> >         TEST arch-debug/tests/counts/vec_is_sf_tutorials-ex1_4.counts
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create
> > not ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-create
> > # Error code: 1
> > # 43,46c43,46
> > # < [0] 0: 4001 2000 2002 3002 4002
> > # < [1] 0: 1001 3000
> > # < [2] 0: 2001 4000
> > # < [3] 0: 3001 1000
> > # ---
> > # > [0] 0: 2002 2146435072 2 2146435072 38736240
> > # > [1] 0: 3000 2146435072
> > # > [2] 0: 2001 2146435072
> > # > [3] 0: 3001 2146435072
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic
> > not ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-dynamic
> > # Error code: 1
> > # 43,46c43,46
> > # < [0] 0: 4001 2000 2002 3002 4002
> > # < [1] 0: 1001 3000
> > # < [2] 0: 2001 4000
> > # < [3] 0: 3001 1000
> > # ---
> > # > [0] 0: 2002 2146435072 2 2146435072 0
> > # > [1] 0: 3000 2146435072
> > # > [2] 0: 2001 2146435072
> > # > [3] 0: 3001 2146435072
> >  ok
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-fence_sf_window_flavor-allocate
> > # retrying
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create
> > not ok
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-create #
> > Error code: 98
> > # [1]PETSC ERROR: --------------------- Error Message
> > --------------------------------------------------------------
> > # [1]PETSC ERROR: General MPI error
> > # [1]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank
> > # [1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> > # [1]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b
> >  GIT Date: 2021-09-13 14:01:22 +0000
> > # [1]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by
> > zampins Tue Sep 14 09:31:42 2021
> > # [1]PETSC ERROR: [2]PETSC ERROR: --------------------- Error Message
> > --------------------------------------------------------------
> > # [2]PETSC ERROR: General MPI error
> > # [2]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank
> > # [2]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> > # [2]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b
> >  GIT Date: 2021-09-13 14:01:22 +0000
> > # [2]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by
> > zampins Tue Sep 14 09:31:42 2021
> > # [2]PETSC ERROR: Configure options
> > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14
> > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1
> > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0
> > --with-hip-dir=/opt/rocm --with-hip=1
> > --with-hypre-dir=/home/zampins/local-petsc
> > --with-kokkos-dir=/home/zampins/local-petsc
> > --with-kokkos-kernels-dir=/home/zampins/local-petsc
> > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include
> > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]"
> > HIPFLAGS=--amdgpu-target=gfx908
> > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug
> > # [2]PETSC ERROR: #1 PetscSFGetGroups() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166
> > # [2]PETSC ERROR: #2 PetscSFSetUp_Window() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708
> > # [2]PETSC ERROR: #3 PetscSFSetUp() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318
> > # [2]PETSC ERROR: #4 main() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172
> > # [2]PETSC ERROR: PETSc Option Table entries:
> > # [2]PETSC ERROR: -sf_type window
> > # [2]PETSC ERROR: -sf_window_flavor create
> > # [2]PETSC ERROR: -sf_window_sync active
> > # [2]PETSC ERROR: -test_gather
> > # [2]PETSC ERROR: ----------------End of Error Message -------send entire
> > error message to petsc-maint at mcs.anl.gov----------
> > # [3]PETSC ERROR: --------------------- Error Message
> > --------------------------------------------------------------
> > # [3]PETSC ERROR: General MPI error
> > # [3]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank
> > # [3]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> > # [3]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b
> >  GIT Date: 2021-09-13 14:01:22 +0000
> > # [3]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by
> > zampins Tue Sep 14 09:31:42 2021
> > # [3]PETSC ERROR: Configure options
> > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14
> > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1
> > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0
> > --with-hip-dir=/opt/rocm --with-hip=1
> > --with-hypre-dir=/home/zampins/local-petsc
> > --with-kokkos-dir=/home/zampins/local-petsc
> > --with-kokkos-kernels-dir=/home/zampins/local-petsc
> > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include
> > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]"
> > HIPFLAGS=--amdgpu-target=gfx908
> > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug
> > # [3]PETSC ERROR: #1 PetscSFGetGroups() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166
> > # [3]PETSC ERROR: #2 PetscSFSetUp_Window() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708
> > # [3]PETSC ERROR: #3 PetscSFSetUp() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318
> > # [3]PETSC ERROR: #4 main() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172
> > # [3]PETSC ERROR: PETSc Option Table entries:
> > # [3]PETSC ERROR: -sf_type window
> > # [3]PETSC ERROR: -sf_window_flavor create
> > # [3]PETSC ERROR: Configure options
> > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14
> > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1
> > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0
> > --with-hip-dir=/opt/rocm --with-hip=1
> > --with-hypre-dir=/home/zampins/local-petsc
> > --with-kokkos-dir=/home/zampins/local-petsc
> > --with-kokkos-kernels-dir=/home/zampins/local-petsc
> > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include
> > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]"
> > HIPFLAGS=--amdgpu-target=gfx908
> > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug
> > # [1]PETSC ERROR: #1 PetscSFGetGroups() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166
> > # [1]PETSC ERROR: #2 PetscSFSetUp_Window() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708
> > # [1]PETSC ERROR: #3 PetscSFSetUp() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318
> > # [1]PETSC ERROR: #4 main() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172
> > # [1]PETSC ERROR: PETSc Option Table entries:
> > # [1]PETSC ERROR: -sf_type window
> > # [1]PETSC ERROR: -sf_window_flavor create
> > # [1]PETSC ERROR: -sf_window_sync active
> > # [1]PETSC ERROR: -test_gather
> > # [1]PETSC ERROR: ----------------End of Error Message -------send entire
> > error message to petsc-maint at mcs.anl.gov----------
> > # -sf_window_sync active
> > # [3]PETSC ERROR: -test_gather
> > # [3]PETSC ERROR: ----------------End of Error Message -------send entire
> > error message to petsc-maint at mcs.anl.gov----------
> > #
> > --------------------------------------------------------------------------
> > # MPI_ABORT was invoked on rank 2 in communicator MPI_COMM_WORLD
> > # with errorcode 98.
> > #
> > # NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
> > # You may or may not see output from other processes, depending on
> > # exactly when Open MPI kills them.
> > #
> > --------------------------------------------------------------------------
> > # [kanary.kaust.edu.sa:115527] 2 more processes have sent help message
> > help-mpi-api.txt / mpi-abort
> > # [kanary.kaust.edu.sa:115527] Set MCA parameter
> > "orte_base_help_aggregate" to 0 to see all help / error messages
> >  ok vec_is_sf_tutorials-ex1_4 # SKIP Command failed so no diff
> > # retrying
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic
> > not ok
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-dynamic #
> > Error code: 98
> > # [1]PETSC ERROR: --------------------- Error Message
> > --------------------------------------------------------------
> > # [1]PETSC ERROR: General MPI error
> > # [1]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank
> > # [1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> > # [1]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b
> >  GIT Date: 2021-09-13 14:01:22 +0000
> > # [1]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by
> > zampins Tue Sep 14 09:31:45 2021
> > # [1]PETSC ERROR: Configure options
> > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14
> > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1
> > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0
> > --with-hip-dir=/opt/rocm --with-hip=1
> > --with-hypre-dir=/home/zampins/local-petsc
> > --with-kokkos-dir=/home/zampins/local-petsc
> > --with-kokkos-kernels-dir=/home/zampins/local-petsc
> > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include
> > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]"
> > HIPFLAGS=--amdgpu-target=gfx908
> > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug
> > # [1]PETSC ERROR: #1 PetscSFGetGroups() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166
> > # [1]PETSC ERROR: #2 PetscSFSetUp_Window() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708
> > # [1]PETSC ERROR: #3 PetscSFSetUp() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318
> > # [1]PETSC ERROR: #4 main() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172
> > # [1]PETSC ERROR: PETSc Option Table entries:
> > # [1]PETSC ERROR: -sf_type window
> > # [1]PETSC ERROR: [2]PETSC ERROR: --------------------- Error Message
> > --------------------------------------------------------------
> > # [2]PETSC ERROR: General MPI error
> > # [2]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank
> > # [2]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> > # [2]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b
> >  GIT Date: 2021-09-13 14:01:22 +0000
> > # [2]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by
> > zampins Tue Sep 14 09:31:45 2021
> > # [2]PETSC ERROR: Configure options
> > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14
> > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1
> > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0
> > --with-hip-dir=/opt/rocm --with-hip=1
> > --with-hypre-dir=/home/zampins/local-petsc
> > --with-kokkos-dir=/home/zampins/local-petsc
> > --with-kokkos-kernels-dir=/home/zampins/local-petsc
> > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include
> > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]"
> > HIPFLAGS=--amdgpu-target=gfx908
> > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug
> > # [2]PETSC ERROR: #1 PetscSFGetGroups() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166
> > # [2]PETSC ERROR: #2 PetscSFSetUp_Window() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708
> > # [2]PETSC ERROR: #3 PetscSFSetUp() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318
> > # [2]PETSC ERROR: #4 main() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172
> > # [2]PETSC ERROR: PETSc Option Table entries:
> > # [2]PETSC ERROR: -sf_type window
> > # [2]PETSC ERROR: -sf_window_flavor dynamic
> > # [2]PETSC ERROR: -sf_window_sync active
> > # [2]PETSC ERROR: -test_gather
> > # [2]PETSC ERROR: ----------------End of Error Message -------send entire
> > error message to petsc-maint at mcs.anl.gov----------
> > # [3]PETSC ERROR: --------------------- Error Message
> > --------------------------------------------------------------
> > # [3]PETSC ERROR: General MPI error
> > # [3]PETSC ERROR: MPI error 6 MPI_ERR_RANK: invalid rank
> > # [3]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
> > # [3]PETSC ERROR: Petsc Development GIT revision: v3.15.4-783-g168bb9f76b
> >  GIT Date: 2021-09-13 14:01:22 +0000
> > # [3]PETSC ERROR: ../ex1 on a arch-debug named kanary.kaust.edu.sa by
> > zampins Tue Sep 14 09:31:45 2021
> > # [3]PETSC ERROR: Configure options
> > --with-cc=/home/zampins/local/bin/mpicc --with-cxx-dialect=c++14
> > --with-cxx=/home/zampins/local/bin/mpicxx --with-debugging=1
> > --with-fc=/home/zampins/local/bin/mpifort --with-fortran-bindings=0
> > --with-hip-dir=/opt/rocm --with-hip=1
> > --with-hypre-dir=/home/zampins/local-petsc
> > --with-kokkos-dir=/home/zampins/local-petsc
> > --with-kokkos-kernels-dir=/home/zampins/local-petsc
> > --with-blaslapack-include=/home/zampins/local-aocl/aocc/3.0-6/include
> > --with-blaslapack-lib="[/home/zampins/local-aocl/aocc/3.0-6/lib/liblapacke.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libflame.a,/home/zampins/local-aocl/aocc/3.0-6/lib/libblis-mt.a]"
> > HIPFLAGS=--amdgpu-target=gfx908
> > HIPPPFLAGS=-I/home/zampins/local-petsc/include PETSC_ARCH=arch-debug
> > # [3]PETSC ERROR: #1 PetscSFGetGroups() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:1166
> > # [3]PETSC ERROR: #2 PetscSFSetUp_Window() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/impls/window/sfwindow.c:708
> > # [3]PETSC ERROR: #3 PetscSFSetUp() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/interface/sf.c:318
> > # [3]PETSC ERROR: #4 main() at
> > /home/zampins/Devel/petsc/src/vec/is/sf/tutorials/ex1.c:172
> > # [3]PETSC ERROR: PETSc Option Table entries:
> > # [3]PETSC ERROR: -sf_type window
> > # [3]PETSC ERROR: -sf_window_flavor dynamic
> > # [3]PETSC ERROR: -sf_window_sync active
> > # [3]PETSC ERROR: -test_gather
> > # [3]PETSC ERROR: ----------------End of Error Message -------send entire
> > error message to petsc-maint at mcs.anl.gov----------
> > # -sf_window_flavor dynamic
> > # [1]PETSC ERROR: -sf_window_sync active
> > # [1]PETSC ERROR: -test_gather
> > # [1]PETSC ERROR: ----------------End of Error Message -------send entire
> > error message to petsc-maint at mcs.anl.gov----------
> > #
> > --------------------------------------------------------------------------
> > # MPI_ABORT was invoked on rank 3 in communicator MPI_COMM_WORLD
> > # with errorcode 98.
> > #
> > # NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
> > # You may or may not see output from other processes, depending on
> > # exactly when Open MPI kills them.
> > #
> > --------------------------------------------------------------------------
> > # [kanary.kaust.edu.sa:115572] 2 more processes have sent help message
> > help-mpi-api.txt / mpi-abort
> > # [kanary.kaust.edu.sa:115572] Set MCA parameter
> > "orte_base_help_aggregate" to 0 to see all help / error messages
> >  ok vec_is_sf_tutorials-ex1_4 # SKIP Command failed so no diff
> >  ok
> > vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-active_sf_window_flavor-allocate
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create
> > not ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-create
> > # Error code: 1
> > # 43,46c43,46
> > # < [0] 0: 4001 2000 2002 3002 4002
> > # < [1] 0: 1001 3000
> > # < [2] 0: 2001 4000
> > # < [3] 0: 3001 1000
> > # ---
> > # > [0] 0: 4002 2146435072 2 2146435072 34619728
> > # > [1] 0: 3000 2146435072
> > # > [2] 0: 4000 2146435072
> > # > [3] 0: 3001 2146435072
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic
> > not ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-dynamic
> > # Error code: 1
> > # 43,46c43,46
> > # < [0] 0: 4001 2000 2002 3002 4002
> > # < [1] 0: 1001 3000
> > # < [2] 0: 2001 4000
> > # < [3] 0: 3001 1000
> > # ---
> > # > [0] 0: 4002 2146435072 2 2146435072 0
> > # > [1] 0: 3000 2146435072
> > # > [2] 0: 4000 2146435072
> > # > [3] 0: 3001 2146435072
> >  ok vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate
> >  ok
> > diff-vec_is_sf_tutorials-ex1_4+sf_window_sync-lock_sf_window_flavor-allocate
> >
> >
> >
> > Il giorno mar 14 set 2021 alle ore 07:44 Stefano Zampini <
> > stefano.zampini at gmail.com> ha scritto:
> >
> >> I'll see if I can reproduce
> >>
> >> Il Mar 14 Set 2021, 06:58 Junchao Zhang <junchao.zhang at gmail.com> ha
> >> scritto:
> >>
> >>> Hi, Stefano,
> >>>    Ping you again to see if you want to resolve this problem before
> >>> petsc-3.16
> >>>
> >>> --Junchao Zhang
> >>>
> >>>
> >>> On Sun, Sep 12, 2021 at 3:06 PM Antonio T. sagitter <
> >>> sagitter at fedoraproject.org> wrote:
> >>>
> >>>> Unfortunately, it's not possible. I must use the OpenMPI provided by
> >>>> Fedora build-system (these rpm builds of PETSc are for Fedora's
> >>>> repositories), downloading external software is not permitted.
> >>>>
> >>>> On 9/12/21 21:10, Pierre Jolivet wrote:
> >>>> >
> >>>> >> On 12 Sep 2021, at 8:56 PM, Matthew Knepley <knepley at gmail.com
> >>>> >> <mailto:knepley at gmail.com>> wrote:
> >>>> >>
> >>>> >> On Sun, Sep 12, 2021 at 2:49 PM Antonio T. sagitter
> >>>> >> <sagitter at fedoraproject.org <mailto:sagitter at fedoraproject.org>>
> >>>> wrote:
> >>>> >>
> >>>> >>     Those attached are configure.log/make.log from a MPI build in
> >>>> >>     Fedora 34
> >>>> >>     x86_64 where the error below occurred.
> >>>> >>
> >>>> >>
> >>>> >> This is OpenMPI 4.1.0. Is that the only MPI you build? My first
> >>>> >> inclination is that this is an MPI implementation bug.
> >>>> >>
> >>>> >> Junchao, do we have an OpenMPI build in the CI?
> >>>> >
> >>>> > config/examples/arch-ci-linux-cuda-double-64idx.py:
> >>>> >   '--download-openmpi=1',
> >>>> > config/examples/arch-ci-linux-pkgs-dbg-ftn-interfaces.py:
> >>>> >   '--download-openmpi=1',
> >>>> > config/examples/arch-ci-linux-pkgs-opt.py:  '--download-openmpi=1',
> >>>> >
> >>>> > config/BuildSystem/config/packages/OpenMPI.py uses version 4.1.0 as
> >>>> well.
> >>>> > I’m not sure PETSc is to blame here Antonio. You may want to try to
> >>>> > ditch the OpenMPI shipped by your packet manager and try
> >>>> > --download-openmpi as well, just for a quick sanity check.
> >>>> >
> >>>> > Thanks,
> >>>> > Pierre
> >>>> >
> >>>>
> >>>> --
> >>>> ---
> >>>> Antonio Trande
> >>>> Fedora Project
> >>>> mailto: sagitter at fedoraproject.org
> >>>> GPG key: 0x29FBC85D7A51CC2F
> >>>> GPG key server: https://keyserver1.pgp.com/
> >>>>
> >>>
> >
> > --
> > Stefano
> >
> 
> 
>

More information about the petsc-dev mailing list