[petsc-users] DMDA Error

[Smith, Barry F.]

   Are you increasing your problem size with the number of ranks or same size problem? 

    It could also be out of memory issues.

    No error message is printed; which is not standard. It should print first a message why it failed. 

    Are you sure all the libraries were rebuilt. 

    Run with -malloc_debug it will go slow but check for memory corruption, if possible use valgrind


    Barry


> On Jan 16, 2020, at 2:52 AM, Anthony Jourdon <jourdon_anthony at hotmail.fr> wrote:
> 
> Dear Petsc developer,
> 
> I need assistance with an error.
> 
> I run a code that uses the DMDA related functions. I'm using petsc-3.8.4.
> 
> This code used to run very well on a super computer with the OS SLES11.
> Petsc was built using an intel mpi 5.1.3.223 module and intel mkl version 2016.0.2.181 
> The code was running with no problem on 1024 and more mpi ranks.
> 
> Recently, the OS of the computer has been updated to RHEL7
> I rebuilt Petsc using new available versions of intel mpi (2019U5) and mkl (2019.0.5.281) which are the same versions for compilers and mkl.
> Since then I tested to run the exact same code on 8, 16, 24, 48, 512 and 1024 mpi ranks.
> Until 1024 mpi ranks no problem, but for 1024 an error related to DMDA appeared. I snip the first lines of the error stack here and the full error stack is attached.
> 
> [534]PETSC ERROR: #1 PetscGatherMessageLengths() line 120 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/sys/utils/mpimesg.c
> [534]PETSC ERROR: #2 VecScatterCreate_PtoS() line 2288 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/vec/vec/utils/vpscat.c
> [534]PETSC ERROR: #3 VecScatterCreate() line 1462 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/vec/vec/utils/vscat.c
> [534]PETSC ERROR: #4 DMSetUp_DA_3D() line 1042 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/dm/impls/da/da3.c
> [534]PETSC ERROR: #5 DMSetUp_DA() line 25 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/dm/impls/da/dareg.c
> [534]PETSC ERROR: #6 DMSetUp() line 720 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/dm/interface/dm.c
>  
> Thank you for your time,
> Sincerly,
> 
> Anthony Jourdon
> <DMDAError_1024_O0.err>