[petsc-users] DMDA Error

[Anthony Jourdon]

Dear Petsc developer,


I need assistance with an error.


I run a code that uses the DMDA related functions. I'm using petsc-3.8.4.


This code used to run very well on a super computer with the OS SLES11.

Petsc was built using an intel mpi 5.1.3.223 module and intel mkl version 2016.0.2.181

The code was running with no problem on 1024 and more mpi ranks.


Recently, the OS of the computer has been updated to RHEL7

I rebuilt Petsc using new available versions of intel mpi (2019U5) and mkl (2019.0.5.281) which are the same versions for compilers and mkl.

Since then I tested to run the exact same code on 8, 16, 24, 48, 512 and 1024 mpi ranks.

Until 1024 mpi ranks no problem, but for 1024 an error related to DMDA appeared. I snip the first lines of the error stack here and the full error stack is attached.


[534]PETSC ERROR: #1 PetscGatherMessageLengths() line 120 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/sys/utils/mpimesg.c

[534]PETSC ERROR: #2 VecScatterCreate_PtoS() line 2288 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/vec/vec/utils/vpscat.c

[534]PETSC ERROR: #3 VecScatterCreate() line 1462 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/vec/vec/utils/vscat.c

[534]PETSC ERROR: #4 DMSetUp_DA_3D() line 1042 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/dm/impls/da/da3.c

[534]PETSC ERROR: #5 DMSetUp_DA() line 25 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/dm/impls/da/dareg.c

[534]PETSC ERROR: #6 DMSetUp() line 720 in /scratch2/dlp/appli_local/SCR/OROGEN/petsc3.8.4_MPI/petsc-3.8.4/src/dm/interface/dm.c



Thank you for your time,

Sincerly,


Anthony Jourdon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200116/2918c9b6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DMDAError_1024_O0.err
Type: application/octet-stream
Size: 1008621 bytes
Desc: DMDAError_1024_O0.err
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20200116/2918c9b6/attachment-0001.obj>