[petsc-users] HDF5 corruption

[Adrian Croucher]

hi Jed,

It looked to me like a call to h5f_flush() is all that is required.

Some people said there would be a performance hit (maybe ~ 10% slower), 
which would be the trade-off for increased reliability. So if this were 
made available via PetscViewerFlush(), I'd probably make it optional in 
my code so the user could decide for themselves if it was worth it for them.

Do you think flushing would be a better option than closing/opening the 
file between writes?

Regards, Adrian

On 10/8/21 9:36 AM, Jed Brown wrote:
> Adrian Croucher <a.croucher at auckland.ac.nz> writes:
>
> > hi,
> >
> > One of the users of my PETSc-based code has reported that HDF5 output
> > files can be corrupted and unusable if e.g. the run is killed. I've 
> just
> > done a bit of reading about this and it appears to be a known issue 
> with
> > HDF5.
> >
> > Some people suggest flushing the HDF5 file periodically to help prevent
> > data loss. I had a look at PetscViewerFlush() but it doesn't seem to be
> > implemented for the HDF5 viewer- is that correct?
>
> Correct, but I think we can and should implement it. In your research 
> just now, were there subtleties beyond this call?
>
> https://portal.hdfgroup.org/display/HDF5/H5F_FLUSH 
> <https://portal.hdfgroup.org/display/HDF5/H5F_FLUSH>

-- 
Dr Adrian Croucher
Senior Research Fellow
Department of Engineering Science
University of Auckland, New Zealand
email: a.croucher at auckland.ac.nz
tel: +64 (0)9 923 4611

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211008/294c8dd0/attachment.html>