[petsc-dev] DMView_Plex crash with sequential DM and numproc>1
Matthew Knepley
knepley at gmail.com
Sat Apr 12 17:32:25 CDT 2014
On Fri, Apr 11, 2014 at 12:35 PM, Matthew Knepley <knepley at gmail.com> wrote:
> On Fri, Apr 11, 2014 at 12:11 PM, Andrés Alessandro León Baldelli <
> a.leon.baldelli at gmail.com> wrote:
>
>> Hi,
>> I am trying to use the new Plex to HDF5 interface. From what I
>> understand, it only works in sequential mode.
>> I would like to write a sequential DM, created e.g. from an exodusII mesh
>> via DMPlexCreateExodusFromFile.
>> What puzzles me is that I only can do it if I run on a single processor,
>> since it crashes as soon its communicator contains more than one proc.
>>
>> My understanding is that, if numproc>1, the crash is due to the call to
>> ISView in src/dm/impls/plex/plex.c:785 which fails because its two
>> arguments do not share the same communicator.
>>
>> Could you help me understand a little more what is going on, what I am
>> doing wrong and what could help solving this issue?
>>
>
> Use 'next', not 'master'
>
I just pushed an update to next that should allow parallel reading and
writing to HDF5. I test using
-dm_view hdf5:sol.h5 -vec_view hdf5:sol.h5::append
and then DMLoad()/VecLoad(). It of course needs a lot more testing, but it
works on my small case
right now.
Matt
> Matt
>
>
>> I attach the error message, a minimal code to reproduce the crash and a
>> test mesh.
>>
>> Best,
>>
>> ---------- begin output----------
>> geothermal:dmcomplex kumiori$ mpirun -n 2 ./testHDF5Plex -i
>> data/TwoTri.gen
>> cubit(/Users/blaise/Development/DMComplex/data/junkTri.gen): 08/31/2013:
>> 19:01:0 in 2 dimensions:
>> 0-cells: 6 0
>> 2-cells: 4 0
>> Labels:
>> depth: 2 strata of sizes (6, 4)
>> Cell Sets: 1 strata of sizes (4)
>> [0]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [0]PETSC ERROR: Arguments must have same communicators
>> [0]PETSC ERROR: Different communicators in the two objects: Argument # 1
>> and 2 flag 3
>> [0]PETSC ERROR: See http://
>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
>> [0]PETSC ERROR: Petsc Development GIT revision: v3.4.4-3957-g24a0f54 GIT
>> Date: 2014-04-02 15:35:24 -0500
>> [0]PETSC ERROR: ./testHDF5Plex on a Darwin-llvm50-g named
>> geothermal.lsu.edu by kumiori Fri Apr 11 12:06:44 2014
>> [0]PETSC ERROR: Configure options --CFLAGS=-Wno-unused --download-chaco=1
>> --download-exodusii=1 --download-hdf5=1 --download-metis=1
>> --download-mpich=1 --download-netcdf=1 --download-parmetis=1
>> --download-sowing=1 --download-triangle=1 --download-yaml=1
>> --with-cmake=cmake --with-debugging=1 --with-pic --with-shared-libraries=1
>> --with-x11=1 PETSC_ARCH=Darwin-llvm50-g
>> [0]PETSC ERROR: #1 ISView() line 756 in
>> /opt/HPC/petsc/src/vec/is/is/interface/index.c
>> [0]PETSC ERROR: #2 DMPlexView_HDF5() line 785 in
>> /opt/HPC/petsc/src/dm/impls/plex/plex.c
>> [0]PETSC ERROR: #3 DMView_Plex() line 812 in
>> /opt/HPC/petsc/src/dm/impls/plex/plex.c
>> [0]PETSC ERROR: #4 main() line 61 in
>> /Users/kumiori/Code/dmcomplex/testHDF5Plex.c
>> [0]PETSC ERROR: ----------------End of Error Message -------send entire
>> error message to petsc-maint at mcs.anl.gov----------
>> application called MPI_Abort(MPI_COMM_WORLD, 80) - process 0
>> [cli_0]: aborting job:
>> application called MPI_Abort(MPI_COMM_WORLD, 80) - process 0
>> HDF5-DIAG: Error detected in HDF5 (1.8.10-patch1) MPI-process 1:
>> #000: H5F.c line 2058 in H5Fclose(): decrementing file ID failed
>> major: Object atom
>> minor: Unable to close file
>> #001: H5I.c line 1479 in H5I_dec_app_ref(): can't decrement ID ref count
>> major: Object atom
>> minor: Unable to decrement reference count
>> #002: H5F.c line 1835 in H5F_close(): can't close file
>> major: File accessability
>> minor: Unable to close file
>> #003: H5F.c line 1997 in H5F_try_close(): problems closing file
>> major: File accessability
>> minor: Unable to close file
>> #004: H5F.c line 1142 in H5F_dest(): low level truncate failed
>> major: File accessability
>> minor: Write failed
>> #005: H5FD.c line 1897 in H5FD_truncate(): driver truncate request
>> failed
>> major: Virtual File Layer
>> minor: Can't update object
>> #006: H5FDmpio.c line 1984 in H5FD_mpio_truncate(): MPI_File_set_size
>> failed
>> major: Internal error (too specific to document in detail)
>> minor: Some MPI function failed
>> #007: H5FDmpio.c line 1984 in H5FD_mpio_truncate(): Invalid argument,
>> error stack:
>> MPI_FILE_SET_SIZE(74): Inconsistent arguments to collective routine
>> major: Internal error (too specific to document in detail)
>> minor: MPI Error String
>> [0]PETSC ERROR:
>> ------------------------------------------------------------------------
>> [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
>> probably memory access out of range
>> [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
>> [0]PETSC ERROR: or see
>> http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind[0]PETSCERROR: or try
>> http://valgrind.org on GNU/linux and Apple Mac OS X to find memory
>> corruption errors
>> [0]PETSC ERROR: likely location of problem given in stack below
>> [0]PETSC ERROR: --------------------- Stack Frames
>> ------------------------------------
>> [0]PETSC ERROR: Note: The EXACT line numbers in the stack are not
>> available,
>> [0]PETSC ERROR: INSTEAD the line number of the start of the function
>> [0]PETSC ERROR: is given.
>> [0]PETSC ERROR: [0] PetscError line 363 /opt/HPC/petsc/src/sys/error/err.c
>> [0]PETSC ERROR: [0] ISView line 750
>> /opt/HPC/petsc/src/vec/is/is/interface/index.c
>> [0]PETSC ERROR: [0] DMPlexView_HDF5 line 656
>> /opt/HPC/petsc/src/dm/impls/plex/plex.c
>> [0]PETSC ERROR: [0] DMView_Plex line 803
>> /opt/HPC/petsc/src/dm/impls/plex/plex.c
>> [0]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [0]PETSC ERROR: Signal received
>> [0]PETSC ERROR: See http://
>> http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
>> [0]PETSC ERROR: Petsc Development GIT revision: v3.4.4-3957-g24a0f54 GIT
>> Date: 2014-04-02 15:35:24 -0500
>> [0]PETSC ERROR: ./testHDF5Plex on a Darwin-llvm50-g named
>> geothermal.lsu.edu by kumiori Fri Apr 11 12:06:44 2014
>> [0]PETSC ERROR: Configure options --CFLAGS=-Wno-unused --download-chaco=1
>> --download-exodusii=1 --download-hdf5=1 --download-metis=1
>> --download-mpich=1 --download-netcdf=1 --download-parmetis=1
>> --download-sowing=1 --download-triangle=1 --download-yaml=1
>> --with-cmake=cmake --with-debugging=1 --with-pic --with-shared-libraries=1
>> --with-x11=1 PETSC_ARCH=Darwin-llvm50-g
>> [0]PETSC ERROR: #5 User provided function() line 0 in unknown file
>> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
>> [cli_0]: aborting job:
>> application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
>>
>>
>> ===================================================================================
>> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
>> = EXIT CODE: 59
>> = CLEANING UP REMAINING PROCESSES
>> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
>>
>> ===================================================================================
>> geothermal:dmcomplex kumiori$
>>
>> ---------- end output----------
>>
>> ----------begin C source code testHDF5plex.c --------------
>> /* usage: testHDF5plex -i meshfilename
>> */
>> #include <petscdmplex.h>
>> #include <petscsf.h>
>> #include <exodusII.h>
>> #include <petsc-private/dmimpl.h> /*I "petscdm.h" I*/
>> #include <petscsf.h>
>> #include <petscviewerhdf5.h>
>> #include <petscdmplex.h>
>>
>> #undef __FUNCT__
>> #define __FUNCT__ "main"
>> int main(int argc,char **argv)
>> {
>> PetscErrorCode ierr;
>> DM dm;
>> char ifilename[PETSC_MAX_PATH_LEN];
>> PetscBool flg;
>> int CPU_word_size = 0,IO_word_size = 0,exoid;
>> float version;
>> PetscMPIInt numproc,rank;
>> PetscViewer hdf5Viewer;
>> PetscInt dim;
>> PetscInt numFields = 2;
>> PetscInt numComp[2] = {1,1};
>> PetscInt numDof[6] = {1, 0, 0,
>> 0, 0, 1}; /*{Vertex, Edge, Cell}
>> */
>> PetscInt bcFields[1] = {0}, numBC=0;
>> IS bcPoints[1] = {NULL};
>> PetscSection section;
>> MPI_Comm comm;
>>
>> ierr = PetscInitialize(&argc, &argv, (char*)0,
>> help);CHKERRQ(ierr);
>> comm = PETSC_COMM_WORLD;
>>
>> ierr = PetscOptionsBegin(PETSC_COMM_WORLD,"","testHDF5Plex Test
>> Options","none");CHKERRQ(ierr);
>> ierr = PetscOptionsEnd();
>> ierr = PetscOptionsGetString(PETSC_NULL,"-i",ifilename,sizeof
>> ifilename,&flg);CHKERRQ(ierr);
>>
>> ierr = MPI_Comm_size(PETSC_COMM_WORLD,&numproc);
>> ierr = MPI_Comm_rank(PETSC_COMM_WORLD,&rank);
>>
>> ierr = DMPlexCreateExodusFromFile(PETSC_COMM_WORLD, ifilename,
>> PETSC_FALSE, &dm);CHKERRQ(ierr);
>> ierr = DMPlexGetDimension(dm, &dim);CHKERRQ(ierr);
>> ierr = DMPlexCreateSection(dm, dim, numFields, numComp, numDof,
>> numBC, bcFields, bcPoints, NULL, §ion);CHKERRQ(ierr);
>> ierr = DMSetDefaultSection(dm, section);CHKERRQ(ierr);
>> ierr = PetscViewerHDF5Open(PetscObjectComm((PetscObject) dm),
>> "out/seqDM.h5", FILE_MODE_WRITE, &hdf5Viewer); CHKERRQ(ierr);
>> // Crashes if numproc > 1,
>> ierr = DMView_Plex(dm, hdf5Viewer); CHKERRQ(ierr);
>> PetscViewerDestroy(&hdf5Viewer);
>>
>> ierr = DMDestroy(&dm);CHKERRQ(ierr);
>> ierr = PetscFinalize();
>> return 0;
>> }
>>
>> ----------end C source code testHDF5plex.c --------------
>>
>> ---------- test mesh
>>
>> Andrés A. León Baldelli
>> --
>> LSU Center for Computation & Technology
>> 2099 Louisiana Digital Media Center
>>
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
--
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20140412/3e0325d0/attachment.html>
More information about the petsc-dev
mailing list