[petsc-users] Error in creating compressed data using HDF5

[Matthew Knepley]

On Tue, Sep 3, 2019 at 1:57 AM Danyang Su via petsc-users <
petsc-users at mcs.anl.gov> wrote:

> Unfortunately, the master branch with hdf-1.10.5 returns similar error.
>

It looks like its complaining about the group_id. Are you sure its correct?

   Matt


> Danyang
>
> On 2019-09-02 5:31 p.m., Danyang Su wrote:
> > Hi Barry,
> >
> > Yes, I have already included zlib and szlib during the configuration.
> > I will try the dev version to see if it works.
> >
> > Thanks,
> >
> > Danyang
> >
> > On 2019-09-02 12:45 p.m., Smith, Barry F. wrote:
> >>    You could try the master branch of PETSc that uses a much more
> >> recent branch of hdf5
> >>
> >>    When you did the --download-hdf5 did you also do --download-zlib
> >> and --download-szlib (though I would hope hdf5 would give you a very
> >> useful error message that they need to be installed instead of the
> >> vague error message they do provide.)
> >>
> >>     Barry
> >>
> >>
> >>> On Sep 2, 2019, at 2:24 PM, Danyang Su via petsc-users
> >>> <petsc-users at mcs.anl.gov> wrote:
> >>>
> >>> Dear All,
> >>>
> >>> Not sure if this is the right place to ask hdf5 question. I
> >>> installed hdf5 through PETSc configuration --download-hdf5=yes. The
> >>> code runs without problem except the function to create compressed
> >>> data (red part shown below).
> >>>
> >>>      !c create local memory space and hyperslab
> >>>      call h5screate_simple_f(hdf5_ndim, hdf5_dsize,
> >>> memspace,           &
> >>>                              hdf5_ierr)
> >>>      call h5sselect_hyperslab_f(memspace,
> >>> H5S_SELECT_SET_F,             &
> >>>                                 hdf5_offset, hdf5_count,
> >>> hdf5_ierr,     &
> >>>                                 hdf5_stride, hdf5_block)
> >>>
> >>>      !c create the global file space and hyperslab
> >>>      call h5screate_simple_f(hdf5_ndim,hdf5_gdsize,filespace, &
> >>>                              hdf5_ierr)
> >>>      call h5sselect_hyperslab_f(filespace,
> >>> H5S_SELECT_SET_F,            &
> >>>                                 hdf5_goffset, hdf5_count,
> >>> hdf5_ierr,    &
> >>>                                 hdf5_stride, hdf5_block)
> >>>
> >>>      !c create a data chunking property
> >>>      call h5pcreate_f(H5P_DATASET_CREATE_F, chunk_id, hdf5_ierr)
> >>>      call h5pset_chunk_f(chunk_id, hdf5_ndim, hdf5_csize, hdf5_ierr)
> >>>
> >>>      !c create compressed data, dataset must be chunked for compression
> >>>      !c the following cause crash in hdf5 library, check when new
> >>>      !c hdf5 version is available
> >>>
> >>>      ! Set ZLIB / DEFLATE Compression using compression level 6.
> >>>      ! To use SZIP Compression comment out these lines.
> >>>      !call h5pset_deflate_f(chunk_id, 6, hdf5_ierr)
> >>>
> >>>      ! Uncomment these lines to set SZIP Compression
> >>>      !szip_options_mask = H5_SZIP_NN_OM_F
> >>>      !szip_pixels_per_block = 16
> >>>      !call H5Pset_szip_f(chunk_id,
> >>> szip_options_mask,                    &
> >>>      !                   szip_pixels_per_block, hdf5_ierr)
> >>>
> >>>      !c create the dataset id
> >>>      call h5dcreate_f(group_id, dataname,
> >>> H5T_NATIVE_INTEGER,           &
> >>>                       filespace, dset_id,
> >>> hdf5_ierr,                    &
> >>>                       dcpl_id=chunk_id)
> >>>
> >>>      !c create a data transfer property
> >>>      call h5pcreate_f(H5P_DATASET_XFER_F, xlist_id, hdf5_ierr)
> >>>      call h5pset_dxpl_mpio_f(xlist_id,
> >>> H5FD_MPIO_COLLECTIVE_F,          &
> >>>                              hdf5_ierr)
> >>>
> >>>      !c write the dataset collectively
> >>>      call h5dwrite_f(dset_id, H5T_NATIVE_INTEGER, dataset,
> >>> hdf5_dsize,  &
> >>>                      hdf5_ierr,
> >>> file_space_id=filespace,                &
> >>>                      mem_space_id=memspace, xfer_prp = xlist_id)
> >>>
> >>>      call h5dclose_f(dset_id, hdf5_ierr)
> >>>
> >>>      !c close resources
> >>>      call h5sclose_f(filespace, hdf5_ierr)
> >>>      call h5sclose_f(memspace, hdf5_ierr)
> >>>      call h5pclose_f(chunk_id, hdf5_ierr)
> >>>      call h5pclose_f(xlist_id, hdf5_ierr)
> >>>
> >>>
> >>>
> >>> Both h5pset_deflate_f and H5Pset_szip_f crashes the code with error
> >>> information as shown below. If I comment out h5pset_deflate_f and
> >>> H5Pset_szip_f, then everything works fine.
> >>>
> >>> HDF5-DIAG: Error detected in HDF5 (1.8.18) MPI-process 0:
> >>>    #000: H5D.c line 194 in H5Dcreate2(): unable to create dataset
> >>>      major: Dataset
> >>>      minor: Unable to initialize object
> >>>    #001: H5Dint.c line 455 in H5D__create_named(): unable to create
> >>> and link to dataset
> >>>      major: Dataset
> >>>      minor: Unable to initialize object
> >>>    #002: H5L.c line 1638 in H5L_link_object(): unable to create new
> >>> link to object
> >>>      major: Links
> >>>      minor: Unable to initialize object
> >>>    #003: H5L.c line 1882 in H5L_create_real(): can't insert link
> >>>      major: Symbol table
> >>>      minor: Unable to insert object
> >>>    #004: H5Gtraverse.c line 861 in H5G_traverse(): internal path
> >>> traversal failed
> >>>      major: Symbol table
> >>>      minor: Object not found
> >>>
> >>> Does anyone encounter this kind of error before?
> >>>
> >>> Kind regards,
> >>>
> >>> Danyang
> >>>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190903/412807f7/attachment.html>