[petsc-users] Error in creating compressed data using HDF5

[Danyang Su]

Unfortunately, the master branch with hdf-1.10.5 returns similar error.

Danyang

On 2019-09-02 5:31 p.m., Danyang Su wrote:
> Hi Barry,
>
> Yes, I have already included zlib and szlib during the configuration. 
> I will try the dev version to see if it works.
>
> Thanks,
>
> Danyang
>
> On 2019-09-02 12:45 p.m., Smith, Barry F. wrote:
>>    You could try the master branch of PETSc that uses a much more 
>> recent branch of hdf5
>>
>>    When you did the --download-hdf5 did you also do --download-zlib 
>> and --download-szlib (though I would hope hdf5 would give you a very 
>> useful error message that they need to be installed instead of the 
>> vague error message they do provide.)
>>
>>     Barry
>>
>>
>>> On Sep 2, 2019, at 2:24 PM, Danyang Su via petsc-users 
>>> <petsc-users at mcs.anl.gov> wrote:
>>>
>>> Dear All,
>>>
>>> Not sure if this is the right place to ask hdf5 question. I 
>>> installed hdf5 through PETSc configuration --download-hdf5=yes. The 
>>> code runs without problem except the function to create compressed 
>>> data (red part shown below).
>>>
>>>      !c create local memory space and hyperslab
>>>      call h5screate_simple_f(hdf5_ndim, hdf5_dsize, 
>>> memspace,           &
>>>                              hdf5_ierr)
>>>      call h5sselect_hyperslab_f(memspace, 
>>> H5S_SELECT_SET_F,             &
>>>                                 hdf5_offset, hdf5_count, 
>>> hdf5_ierr,     &
>>>                                 hdf5_stride, hdf5_block)
>>>
>>>      !c create the global file space and hyperslab
>>>      call h5screate_simple_f(hdf5_ndim,hdf5_gdsize,filespace, &
>>>                              hdf5_ierr)
>>>      call h5sselect_hyperslab_f(filespace, 
>>> H5S_SELECT_SET_F,            &
>>>                                 hdf5_goffset, hdf5_count, 
>>> hdf5_ierr,    &
>>>                                 hdf5_stride, hdf5_block)
>>>
>>>      !c create a data chunking property
>>>      call h5pcreate_f(H5P_DATASET_CREATE_F, chunk_id, hdf5_ierr)
>>>      call h5pset_chunk_f(chunk_id, hdf5_ndim, hdf5_csize, hdf5_ierr)
>>>
>>>      !c create compressed data, dataset must be chunked for compression
>>>      !c the following cause crash in hdf5 library, check when new
>>>      !c hdf5 version is available
>>>
>>>      ! Set ZLIB / DEFLATE Compression using compression level 6.
>>>      ! To use SZIP Compression comment out these lines.
>>>      !call h5pset_deflate_f(chunk_id, 6, hdf5_ierr)
>>>
>>>      ! Uncomment these lines to set SZIP Compression
>>>      !szip_options_mask = H5_SZIP_NN_OM_F
>>>      !szip_pixels_per_block = 16
>>>      !call H5Pset_szip_f(chunk_id, 
>>> szip_options_mask,                    &
>>>      !                   szip_pixels_per_block, hdf5_ierr)
>>>
>>>      !c create the dataset id
>>>      call h5dcreate_f(group_id, dataname, 
>>> H5T_NATIVE_INTEGER,           &
>>>                       filespace, dset_id, 
>>> hdf5_ierr,                    &
>>>                       dcpl_id=chunk_id)
>>>
>>>      !c create a data transfer property
>>>      call h5pcreate_f(H5P_DATASET_XFER_F, xlist_id, hdf5_ierr)
>>>      call h5pset_dxpl_mpio_f(xlist_id, 
>>> H5FD_MPIO_COLLECTIVE_F,          &
>>>                              hdf5_ierr)
>>>
>>>      !c write the dataset collectively
>>>      call h5dwrite_f(dset_id, H5T_NATIVE_INTEGER, dataset, 
>>> hdf5_dsize,  &
>>>                      hdf5_ierr, 
>>> file_space_id=filespace,                &
>>>                      mem_space_id=memspace, xfer_prp = xlist_id)
>>>
>>>      call h5dclose_f(dset_id, hdf5_ierr)
>>>
>>>      !c close resources
>>>      call h5sclose_f(filespace, hdf5_ierr)
>>>      call h5sclose_f(memspace, hdf5_ierr)
>>>      call h5pclose_f(chunk_id, hdf5_ierr)
>>>      call h5pclose_f(xlist_id, hdf5_ierr)
>>>
>>>
>>>
>>> Both h5pset_deflate_f and H5Pset_szip_f crashes the code with error 
>>> information as shown below. If I comment out h5pset_deflate_f and 
>>> H5Pset_szip_f, then everything works fine.
>>>
>>> HDF5-DIAG: Error detected in HDF5 (1.8.18) MPI-process 0:
>>>    #000: H5D.c line 194 in H5Dcreate2(): unable to create dataset
>>>      major: Dataset
>>>      minor: Unable to initialize object
>>>    #001: H5Dint.c line 455 in H5D__create_named(): unable to create 
>>> and link to dataset
>>>      major: Dataset
>>>      minor: Unable to initialize object
>>>    #002: H5L.c line 1638 in H5L_link_object(): unable to create new 
>>> link to object
>>>      major: Links
>>>      minor: Unable to initialize object
>>>    #003: H5L.c line 1882 in H5L_create_real(): can't insert link
>>>      major: Symbol table
>>>      minor: Unable to insert object
>>>    #004: H5Gtraverse.c line 861 in H5G_traverse(): internal path 
>>> traversal failed
>>>      major: Symbol table
>>>      minor: Object not found
>>>
>>> Does anyone encounter this kind of error before?
>>>
>>> Kind regards,
>>>
>>> Danyang
>>>