[petsc-users] VecView to hdf5 broken for large (complex) vectors

[Matthew Knepley]

On Tue, Apr 16, 2019 at 10:34 AM Sajid Ali via petsc-users <
petsc-users at mcs.anl.gov> wrote:

> Hi PETSc developers,
>
> I’m trying to write a large vector created with VecCreateMPI (size
> 32768x32768) concurrently from 4 nodes (+32 tasks per node, total 128
> mpi-ranks) and I see the following (indicative) error : [Full error log is
> here : https://file.io/CdjUfe]
>
> HDF5-DIAG: Error detected in HDF5 (1.10.5) MPI-process 52:
>   #000: H5D.c line 145 in H5Dcreate2(): unable to create dataset
>     major: Dataset
>     minor: Unable to initialize object
>   #001: H5Dint.c line 329 in H5D__create_named(): unable to create and link to dataset
>     major: Dataset
>     minor: Unable to initialize object
>   #002: H5L.c line 1557 in H5L_link_object(): unable to create new link to object
>     major: Links
>     minor: Unable to initialize object
>   #003: H5L.c line 1798 in H5L__create_real(): can't insert link
>     major: Links
>     minor: Unable to insert object
>   #004: H5Gtraverse.c line 851 in H5G_traverse(): internal path traversal failed
>     major: Symbol table
> HDF5-DIAG: Error detected in HDF5 (1.10.5) MPI-process 59:
>   #000: H5D.c line 145 in H5Dcreate2(): unable to create dataset
>     major: Dataset
>     minor: Unable to initialize object
>   #001: H5Dint.c line 329 in H5D__create_named(): unable to create and link to dataset
>     major: Dataset
>     minor: Unable to initialize object
>   #002: H5L.c line 1557 in H5L_link_object(): unable to create new link to object
>     major: Links
>     minor: Unable to initialize object
>   #003: H5L.c line 1798 in H5L__create_real(): can't insert link
>     major: Links
>     minor: Unable to insert object
>   #004: H5Gtraverse.c line 851 in H5G_traverse(): internal path traversal failed
>     major: Symbol table
>     minor: Object not found
>   #005: H5Gtraverse.c line 627 in H5G__traverse_real(): traversal operator failed
>     major: Symbol table
>     minor: Callback failed
>   #006: H5L.c line 1604 in H5L__link_cb(): unable to create object
>     major: Links
>     minor: Unable to initialize object
>   #007: H5Oint.c line 2453 in H5O_obj_create(): unable to open object
>     major: Object header
>     minor: Can't open object
>   #008: H5Doh.c line 300 in H5O__dset_create(): unable to create dataset
>     minor: Object not found
>   #005: H5Gtraverse.c line 627 in H5G__traverse_real(): traversal operator failed
>     major: Symbol table
>     minor: Callback failed
>   #006: H5L.c line 1604 in H5L__link_cb(): unable to create object
>     major: Links
>     minor: Unable to initialize object
>   #007: H5Oint.c line 2453 in H5O_obj_create(): unable to open object
>     major: Object header
>     minor: Can't open object
>   #008: H5Doh.c line 300 in H5O__dset_create(): unable to create dataset
>     major: Dataset
>     minor: Unable to initialize object
>   #009: H5Dint.c line 1274 in H5D__create(): unable to construct layout information
>     major: Dataset
>     minor: Unable to initialize object
>   #010: H5Dchunk.c line 872 in H5D__chunk_construct(): unable to set chunk sizes
>     major: Dataset
>     minor: Bad value
>   #011: H5Dchunk.c line 831 in H5D__chunk_set_sizes(): chunk size must be < 4GB
>     major: Dataset
>     minor: Unable to initialize object
>     major: Dataset
>     minor: Unable to initialize object
>   #009: H5Dint.c line 1274 in H5D__create(): unable to construct layout information
>     major: Dataset
>     minor: Unable to initialize object
>   #010: H5Dchunk.c line 872 in H5D__chunk_construct(): unable to set chunk sizes
>     major: Dataset
>     minor: Bad value
>   #011: H5Dchunk.c line 831 in H5D__chunk_set_sizes(): chunk size must be < 4GB
>     major: Dataset
>     minor: Unable to initialize object
> .......
>
> I spoke to Barry last evening who said that this is a known error that was
> fixed for DMDA vecs but is broken for non-dmda vecs.
>
> Could this be fixed ?
>
Barry, what is broken here?


https://bitbucket.org/petsc/petsc/src/8d0323b9f315b69b7b31c5e966afbeb37eb6eb24/src/vec/vec/impls/mpi/pdvec.c#lines-757

Go in the debugger and see what chunksize is being set here, since it
should be less than 4GB, unless HDF5_INT_MAX is somehow wrong
in your installation.

  Thanks,

     Matt

> Thank You,
> Sajid Ali
> Applied Physics
> Northwestern University
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190416/f9e77492/attachment.html>