[petsc-users] VecLoad from HDF5 file
Hapla Vaclav
vaclav.hapla at erdw.ethz.ch
Thu Apr 8 03:01:42 CDT 2021
On 8 Apr 2021, at 02:42, Barry Smith <bsmith at petsc.dev<mailto:bsmith at petsc.dev>> wrote:
Matt,
Is there anyway to provide this "extra meta-data" inside the generated HDF file? Then when a fresh viewer opens the file it uses this meta-data to know that the file contains "time-steps" and allows processing of them? The simplest thing would be to have the viewer generate an error if the file has time-steps but the user does not "request a (or more) time-steps" instead of getting confused about the vector sizes.
Yeah, I think we could use HDF5 attributes for this. I'm now dealing with them so I will take a look once I finish my current MR.
Oh, and I think there's even an old issue about this
https://gitlab.com/petsc/petsc/-/issues/425
- time to resolve it.
Thanks,
Vaclav
Barry
On Apr 7, 2021, at 6:32 PM, Matthew Knepley <knepley at gmail.com<mailto:knepley at gmail.com>> wrote:
On Wed, Apr 7, 2021 at 4:10 PM Thibault Bridel-Bertomeu <thibault.bridelbertomeu at gmail.com<mailto:thibault.bridelbertomeu at gmail.com>> wrote:
Hello Vaclav,
Thank you for your quick answer !!
OK so, if I need to push the group, I added :
call PetscViewerHDF5PushGroup(hdf5Viewer, "/fields", ierr); CHKERRA(ierr)
right after the call to PetscViewerFileSetName.
The result is the same, it produces the following error :
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Unexpected data in file
[0]PETSC ERROR: Global size of array in file is 105, not 25300 as expected
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.14.4-671-g707297fd510 GIT Date: 2021-02-24 22:50:05 +0000
[0]PETSC ERROR: ../../../bin/eulerian3D on a named macbook-pro-de-thibault.home by tbridel Wed Apr 7 22:05:14 2021
[0]PETSC ERROR: Configure options --with-clean=1 --prefix=/Users/tbridel/Documents/1-CODES/04-PETSC/build_uns3D --with-make-np=2 --with-windows-graphics=0 --with-debugging=0 --download-fblaslapack --download-mpich-shared=0 --with-x=0 --with-pthread=0 --with-valgrind=0 --PETSC_ARCH=macosx_uns3D --with-fc=/usr/local/bin/mpifort --with-cc=/usr/local/bin/mpicc --with-cxx=/usr/local/bin/mpic++ --with-openmp=0 --download-hypre=yes --download-sowing=yes --download-metis=yes --download-parmetis=yes --download-triangle=yes --download-tetgen=yes --download-ctetgen=yes --download-p4est=yes --download-zlib=yes --download-c2html=yes --download-eigen=yes --download-pragmatic=yes --with-hdf5-dir=/usr/local/opt/hdf5-mpi --with-cmake-dir=/usr/local/opt/cmake --with-libtoolize=/usr/local/bin/glibtoolize --with-autoreconf=/usr/local/bin/autoreconf
[0]PETSC ERROR: #1 PetscViewerHDF5ReadSizes_Private() line 114 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/vec/is/utils/hdf5io.c
[0]PETSC ERROR: #2 PetscViewerHDF5Load() line 208 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/vec/is/utils/hdf5io.c
[0]PETSC ERROR: #3 VecLoad_HDF5() line 132 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/vec/vec/utils/vecio.c
[0]PETSC ERROR: #4 VecLoad_Default() line 257 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/vec/vec/utils/vecio.c
[0]PETSC ERROR: #5 VecLoad_Plex_Local() line 474 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/dm/impls/plex/plex.c
[0]PETSC ERROR: #6 VecLoad_Plex_HDF5_Internal() line 295 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/dm/impls/plex/plexhdf5.c
[0]PETSC ERROR: #7 VecLoad_Plex() line 496 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/dm/impls/plex/plex.c
[0]PETSC ERROR: #8 VecLoad() line 953 in /Users/tbridel/Documents/1-CODES/04-PETSC/src/vec/vec/interface/vector.c
[0]PETSC ERROR: #9 User provided function() line 0 in User file
Do you know where it could come from ?
I think I understand this. PETSc tries to be clever to allow you to store timesteps. It gives the HDF5 array an extra dimension. Somehow
the Viewer has to know this. The TS does this automatically, so you have an array
GROUP "fields" {
DATASET "Solution" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 21, 5060, 5 ) / ( H5S_UNLIMITED, 5060, 5 ) }
}
}
which has 21 timesteps. However, when you create a brand new Viewer, it does not know, and mistakenly thinks there is a single vector
of length 21 * 5 = 105. You can tell your reader which timestep you want to extract using
https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Viewer/PetscViewerHDF5SetTimestep.html
Thanks,
Matt
As for what I am doing exactly : i am using the DMPlex with a PetscFV to solve the fluid mechanics Euler equations in 3D. The PetscFV linked to the DS of the DMPlex might be why there is a /fields before the /Solution maybe .. ?
Cheers and thank you again for your help !!
Thibault
Le mer. 7 avr. 2021 à 10:07, Hapla Vaclav <vaclav.hapla at erdw.ethz.ch<mailto:vaclav.hapla at erdw.ethz.ch>> a écrit :
Dear Thibault
On 7 Apr 2021, at 08:18, Thibault Bridel-Bertomeu <thibault.bridelbertomeu at gmail.com<mailto:thibault.bridelbertomeu at gmail.com>> wrote:
Dear all,
I have been facing a problem with VecLoad recently, even though it seems to me I did exactly like in the examples/tutorials.
Basically, a program writes a vector with the HDF5 writer like this :
call DMCreateGlobalVector(dm, sol, ierr); CHKERRA(ierr)
call VecZeroEntries(X, ierr); CHKERRA(ierr)
call PetscObjectSetName(X, "Solution", ierr); CHKERRA(ierr)
< do something with X to fill it up with relevant data >
call PetscViewerCreate(PETSC_COMM_WORLD, hdf5Viewer, ierr); CHKERRA(ierr)
call PetscViewerSetType(hdf5Viewer, PETSCVIEWERHDF5, ierr); CHKERRA(ierr)
call PetscViewerFileSetMode(hdf5Viewer, FILE_MODE_WRITE, ierr); CHKERRA(ierr);
write(filename,'(A,I5.5,A)') "restart_", stepnum, ".h5"
call PetscViewerFileSetName(hdf5Viewer, trim(filename), ierr); CHKERRA(ierr)
call VecView(X, hdf5Viewer, ierr); CHKERRA(ierr)
call PetscViewerDestroy(hdf5Viewer, ierr); CHKERRA(ierr)
and the same program (but with different start-up options, say) re-reads such a file like this :
call DMCreateGlobalVector(dm, sol, ierr); CHKERRA(ierr)
call VecZeroEntries(sol, ierr); CHKERRA(ierr)
call PetscObjectSetName(sol, "Solution", ierr); CHKERRA(ierr)
call PetscViewerCreate(PETSC_COMM_WORLD, hdf5Viewer, ierr); CHKERRA(ierr)
call PetscViewerSetType(hdf5Viewer, PETSCVIEWERHDF5, ierr); CHKERRA(ierr)
call PetscViewerFileSetMode(hdf5Viewer, FILE_MODE_READ, ierr); CHKERRA(ierr)
call PetscViewerFileSetName(hdf5Viewer, trim(restartname), ierr); CHKERRA(ierr)
call VecLoad(sol, hdf5Viewer, ierr); CHKERRA(ierr)
call PetscViewerDestroy(hdf5Viewer, ierr); CHKERRA(ierr)
Such a dataset can be found under this link : https://drive.google.com/file/d/1owLAx5vknNhj61_5ieAwnWOR9cmkTseL/view?usp=sharing
I'm just looking at the HDF5 file. The structure is like this
> $PETSC_DIR/$PETSC_ARCH/bin/h5dump -H restart_00020.h5
HDF5 "restart_00020.h5" {
GROUP "/" {
GROUP "cell_fields" {
DATASET "Solution_FV solver" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 21, 3884, 5 ) / ( H5S_UNLIMITED, 3884, 5 ) }
ATTRIBUTE "vector_field_type" {
DATATYPE H5T_STRING {
STRSIZE 7;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}
DATASPACE SCALAR
}
}
}
GROUP "fields" {
DATASET "Solution" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 21, 5060, 5 ) / ( H5S_UNLIMITED, 5060, 5 ) }
}
}
DATASET "time" {
DATATYPE H5T_IEEE_F64LE
DATASPACE SIMPLE { ( 21, 1 ) / ( H5S_UNLIMITED, 1 ) }
}
}
}
I would like the reader to read the /fields/Solution group basically, but I am not even sure it tries to do that.
Anyhow, I got an error, saying that the size found in the file (105) does not match the expected size (25300). If I look at the shape of /fields/Solution it is given as (21, 5030, 5). First, it is weird, cause the 21 seems to be 1 + current iteration number ... but anyways we find the 5 variables and the 5030 cells. Only the reader seems to do 21 * 5 when it should be doing 5030 * 5 ...
I tried adding 'PetscViewerHDF5PushGroup(hdf5Viewer, "/fields/Solution", ierr)' to force it to read that group, but it does not change anything.
You definitely need to push the group, unless it's the root group "/". There is no way the reader would guess the correct group if it's not the root one [and I don't think it would be a good idea to implement such searching].
If you tried adding
PetscViewerHDF5PushGroup(hdf5Viewer, "/fields/Solution", ierr)
you likely pushed a wrong group. If the Vec name was set to "Solution" like in your snippet [using PetscObjectSetName()], the absolute dataset name to look up would be "/fields/Solution/Solution".
But in your file, there's just a dataset "/fields/Solution", so its parent group is just "/fields". So please try pushing this.
I would gladly try to reproduce your case - perhaps the error handling should be improved so that it would guide you into the right direction. But it would be helpful to know exactly what you're doing - the snippet with VecView() above should produce "/Solution" dataset but in the file you're sending, there's "/fields/Solution".
Note also you don't need to do VecZeroEntries() before loading because VecLoad() fully rewrites the Vec data in memory anyway.
I would appreciate it if anyone could give me pointers on this issue ...
Thank you very much in advance !!
Thibault
Thanks,
Vaclav
--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
-- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20210408/d807b9de/attachment-0001.html>
More information about the petsc-users
mailing list