[petsc-users] HDF5 corruption

[Hapla Vaclav]

> On 8 Oct 2021, at 00:48, Jed Brown <jed at jedbrown.org> wrote:
> 
> Adrian Croucher <a.croucher at auckland.ac.nz> writes:
> 
>> hi Jed,
>> 
>> It looked to me like a call to h5f_flush() is all that is required.
>> 
>> Some people said there would be a performance hit (maybe ~ 10% slower), 
>> which would be the trade-off for increased reliability. So if this were 
>> made available via PetscViewerFlush(), I'd probably make it optional in 
>> my code so the user could decide for themselves if it was worth it for them.
>> 
>> Do you think flushing would be a better option than closing/opening the 
>> file between writes?
> 
> Yes, less costly at scale (metadata like opening files can be expensive on parallel file systems), and simpler to manage from your code.

I have just come across this a couple of days ago. I think PetscViewerFlush() [no-op for HDF5 currently] should call H5Fflush() for sure. I can do it now.

I agree with Jed that closing/opening can have significant overhead on large number of processes due to metadata processing.

Thanks,

Vaclav