[petsc-users] How to read/write a HDF5 file using petsc4py ?

Tue Dec 7 07:58:24 CST 2021

On Tue, Dec 7, 2021 at 8:26 AM Quentin Chevalier <
quentin.chevalier at polytechnique.edu> wrote:

> Ok my bad, that log corresponded to a tentative --download-hdf5. This
> log corresponds to the commands given above and has --with-hdf5 in its
> options.
>

Okay, this configure was successful and found HDF5

> The whole process still results in the same error.
>

Now send me the complete error output with this PETSc.

  Thanks,

     Matt

> Quentin
>
>
>
> Quentin CHEVALIER – IA parcours recherche
>
> LadHyX - Ecole polytechnique
>
> __________
>
>
>
> On Tue, 7 Dec 2021 at 13:59, Matthew Knepley <knepley at gmail.com> wrote:
> >
> > On Tue, Dec 7, 2021 at 3:55 AM Quentin Chevalier <
> quentin.chevalier at polytechnique.edu> wrote:
> >>
> >> Hello Matthew,
> >>
> >> That would indeed make sense.
> >>
> >> Full log is attached, I grepped hdf5 in there and didn't find anything
> alarming.
> >
> >
> > At the top of this log:
> >
> > Configure Options: --configModules=PETSc.Configure
> --optionsModule=config.compilerOptions PETSC_ARCH=linux-gnu-complex-64
> --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-make-np=2
> --with-64-bit-indices=yes --with-debugging=no --with-fortran-bindings=no
> --with-shared-libraries --download-hypre --download-mumps
> --download-ptscotch --download-scalapack --download-suitesparse
> --download-superlu_dist --with-scalar-type=complex
> >
> >
> > So the HDF5 option is not being specified.
> >
> >   Thanks,
> >
> >      Matt
> >
> >> Cheers,
> >>
> >> Quentin
> >>
> >>
> >>
> >>
> >> Quentin CHEVALIER – IA parcours recherche
> >>
> >> LadHyX - Ecole polytechnique
> >>
> >> __________
> >>
> >>
> >>
> >> On Mon, 6 Dec 2021 at 21:39, Matthew Knepley <knepley at gmail.com> wrote:
> >>>
> >>> On Mon, Dec 6, 2021 at 3:27 PM Quentin Chevalier <
> quentin.chevalier at polytechnique.edu> wrote:
> >>>>
> >>>> Fine. MWE is unchanged :
> >>>> * Run this docker container
> >>>> * Do : python3 -c "from petsc4py import PETSc;
> PETSc.Viewer().createHDF5('dummy.h5')"
> >>>>
> >>>> Updated attempt at a fix :
> >>>> * cd /usr/local/petsc/
> >>>> * ./configure PETSC_ARCH= linux-gnu-real-32
> PETSC_DIR=/usr/local/petsc --with-hdf5 --force
> >>>
> >>>
> >>> Did it find HDF5? If not, it will shut it off. You need to send us
> >>>
> >>>   $PETSC_DIR/configure.log
> >>>
> >>> so we can see what happened in the configure run.
> >>>
> >>>   Thanks,
> >>>
> >>>       Matt
> >>>
> >>>>
> >>>> * make PETSC_DIR=/usr/local/petsc PETSC-ARCH= linux-gnu-real-32 all
> >>>>
> >>>> Still no joy. The same error remains.
> >>>>
> >>>> Quentin
> >>>>
> >>>>
> >>>>
> >>>> On Mon, 6 Dec 2021 at 20:04, Pierre Jolivet <pierre at joliv.et> wrote:
> >>>> >
> >>>> >
> >>>> >
> >>>> > On 6 Dec 2021, at 7:42 PM, Quentin Chevalier <
> quentin.chevalier at polytechnique.edu> wrote:
> >>>> >
> >>>> > The PETSC_DIR exactly corresponds to the previous one, so I guess
> that rules option b) out, except if a specific option is required to
> overwrite a previous installation of PETSc. As for a), well I thought
> reconfiguring pretty direct, you're welcome to give me a hint as to what
> could be wrong in the following process.
> >>>> >
> >>>> > Steps to reproduce this behaviour are as follows :
> >>>> > * Run this docker container
> >>>> > * Do : python3 -c "from petsc4py import PETSc;
> PETSc.Viewer().createHDF5('dummy.h5')"
> >>>> >
> >>>> > After you get the error Unknown PetscViewer type, feel free to try :
> >>>> >
> >>>> > * cd /usr/local/petsc/
> >>>> > * ./configure --with-hfd5
> >>>> >
> >>>> >
> >>>> > It’s hdf5, not hfd5.
> >>>> > It’s PETSC_ARCH, not PETSC-ARCH.
> >>>> > PETSC_ARCH is missing from your configure line.
> >>>> >
> >>>> > Thanks,
> >>>> > Pierre
> >>>> >
> >>>> > * make PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 all
> >>>> >
> >>>> > Then repeat the MWE and observe absolutely no behavioural change
> whatsoever. I'm afraid I don't know PETSc well enough to be surprised by
> that.
> >>>> >
> >>>> > Quentin
> >>>> >
> >>>> >
> >>>> >
> >>>> > Quentin CHEVALIER – IA parcours recherche
> >>>> >
> >>>> > LadHyX - Ecole polytechnique
> >>>> >
> >>>> > __________
> >>>> >
> >>>> >
> >>>> >
> >>>> > On Mon, 6 Dec 2021 at 19:24, Matthew Knepley <knepley at gmail.com>
> wrote:
> >>>> >>
> >>>> >> On Mon, Dec 6, 2021 at 1:22 PM Quentin Chevalier <
> quentin.chevalier at polytechnique.edu> wrote:
> >>>> >>>
> >>>> >>> It failed all of the tests included in `make
> >>>> >>> PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 check`,
> with
> >>>> >>> the error `/usr/bin/bash: line 1: cd: src/snes/tutorials: No such
> file
> >>>> >>> or directory`
> >>>> >>>
> >>>> >>> I am therefore fairly confident this a "file absence" problem,
> and not
> >>>> >>> a compilation problem.
> >>>> >>>
> >>>> >>> I repeat that there was no error at compilation stage. The final
> stage
> >>>> >>> did present `gmake[3]: Nothing to be done for 'libs'.` but that's
> all.
> >>>> >>>
> >>>> >>> Again, running `./configure --with-hdf5` followed by a `make
> >>>> >>> PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 all` does
> not
> >>>> >>> change the problem. I get the same error at the same position as
> >>>> >>> before.
> >>>> >>
> >>>> >>
> >>>> >> If you reconfigured and rebuilt, it is impossible to get the same
> error, so
> >>>> >>
> >>>> >>   a) You did not reconfigure
> >>>> >>
> >>>> >>   b) Your new build is somewhere else on the machine
> >>>> >>
> >>>> >>   Thanks,
> >>>> >>
> >>>> >>      Matt
> >>>> >>
> >>>> >>>
> >>>> >>> I will comment I am running on OpenSUSE.
> >>>> >>>
> >>>> >>> Quentin
> >>>> >>>
> >>>> >>> On Mon, 6 Dec 2021 at 19:09, Matthew Knepley <knepley at gmail.com>
> wrote:
> >>>> >>> >
> >>>> >>> > On Mon, Dec 6, 2021 at 1:08 PM Quentin Chevalier <
> quentin.chevalier at polytechnique.edu> wrote:
> >>>> >>> >>
> >>>> >>> >> Hello Matthew and thanks for your quick response.
> >>>> >>> >>
> >>>> >>> >> I'm afraid I did try to snoop around the container and rerun
> PETSc's
> >>>> >>> >> configure with the --with-hdf5 option, to absolutely no avail.
> >>>> >>> >>
> >>>> >>> >> I didn't see any errors during config or make, but it failed
> the tests
> >>>> >>> >> (which aren't included in the minimal container I suppose)
> >>>> >>> >
> >>>> >>> >
> >>>> >>> > Failed which tests? What was the error?
> >>>> >>> >
> >>>> >>> >   Thanks,
> >>>> >>> >
> >>>> >>> >     Matt
> >>>> >>> >
> >>>> >>> >>
> >>>> >>> >> Quentin
> >>>> >>> >>
> >>>> >>> >>
> >>>> >>> >>
> >>>> >>> >> Quentin CHEVALIER – IA parcours recherche
> >>>> >>> >>
> >>>> >>> >> LadHyX - Ecole polytechnique
> >>>> >>> >>
> >>>> >>> >> __________
> >>>> >>> >>
> >>>> >>> >>
> >>>> >>> >>
> >>>> >>> >> On Mon, 6 Dec 2021 at 19:02, Matthew Knepley <
> knepley at gmail.com> wrote:
> >>>> >>> >> >
> >>>> >>> >> > On Mon, Dec 6, 2021 at 11:28 AM Quentin Chevalier <
> quentin.chevalier at polytechnique.edu> wrote:
> >>>> >>> >> >>
> >>>> >>> >> >> Hello PETSc users,
> >>>> >>> >> >>
> >>>> >>> >> >> This email is a duplicata of this gitlab issue, sorry for
> any inconvenience caused.
> >>>> >>> >> >>
> >>>> >>> >> >> I want to compute a PETSc vector in real mode, than perform
> calculations with it in complex mode. I want as much of this process to be
> parallel as possible. Right now, I compile PETSc in real mode, compute my
> vector and save it to a file, then switch to complex mode, read it, and
> move on.
> >>>> >>> >> >>
> >>>> >>> >> >> This creates unexpected behaviour using MPIIO, so on
> Lisandro Dalcinl's advice I'm moving to HDF5 format. My code is as follows
> (taking inspiration from petsc4py doc, a bitbucket example and another one,
> all top Google results for 'petsc hdf5') :
> >>>> >>> >> >>>
> >>>> >>> >> >>> viewer = PETSc.Viewer().createHDF5(file_name, 'r',
> COMM_WORLD)
> >>>> >>> >> >>> q.load(viewer)
> >>>> >>> >> >>> q.ghostUpdate(addv=PETSc.InsertMode.INSERT,
> mode=PETSc.ScatterMode.FORWARD)
> >>>> >>> >> >>
> >>>> >>> >> >>
> >>>> >>> >> >> This crashes my code. I obtain traceback :
> >>>> >>> >> >>>
> >>>> >>> >> >>>   File "/home/shared/code.py", line 121, in Load
> >>>> >>> >> >>>     viewer = PETSc.Viewer().createHDF5(file_name, 'r',
> COMM_WORLD)
> >>>> >>> >> >>>   File "PETSc/Viewer.pyx", line 182, in
> petsc4py.PETSc.Viewer.createHDF5
> >>>> >>> >> >>> petsc4py.PETSc.Error: error code 86
> >>>> >>> >> >>> [0] PetscViewerSetType() at
> /usr/local/petsc/src/sys/classes/viewer/interface/viewreg.c:442
> >>>> >>> >> >>> [0] Unknown type. Check for miss-spelling or missing
> package: https://petsc.org/release/install/install/#external-packages
> >>>> >>> >> >>> [0] Unknown PetscViewer type given: hdf5
> >>>> >>> >> >
> >>>> >>> >> > This means that PETSc has not been configured with HDF5
> (--with-hdf5 or --download-hdf5), so the container should be updated.
> >>>> >>> >> >
> >>>> >>> >> >   THanks,
> >>>> >>> >> >
> >>>> >>> >> >     Matt
> >>>> >>> >> >
> >>>> >>> >> >>
> >>>> >>> >> >> I have petsc4py 3.16 from this docker container (list of
> dependencies include PETSc and petsc4py).
> >>>> >>> >> >>
> >>>> >>> >> >> I'm pretty sure this is not intended behaviour. Any insight
> as to how to fix this issue (I tried running ./configure --with-hdf5 to no
> avail) or more generally to perform this jiggling between real and complex
> would be much appreciated,
> >>>> >>> >> >>
> >>>> >>> >> >> Kind regards.
> >>>> >>> >> >>
> >>>> >>> >> >> Quentin
> >>>> >>> >> >
> >>>> >>> >> >
> >>>> >>> >> >
> >>>> >>> >> > --
> >>>> >>> >> > What most experimenters take for granted before they begin
> their experiments is infinitely more interesting than any results to which
> their experiments lead.
> >>>> >>> >> > -- Norbert Wiener
> >>>> >>> >> >
> >>>> >>> >> > https://www.cse.buffalo.edu/~knepley/
> >>>> >>> >
> >>>> >>> >
> >>>> >>> >
> >>>> >>> > --
> >>>> >>> > What most experimenters take for granted before they begin
> their experiments is infinitely more interesting than any results to which
> their experiments lead.
> >>>> >>> > -- Norbert Wiener
> >>>> >>> >
> >>>> >>> > https://www.cse.buffalo.edu/~knepley/
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> >>>> >> -- Norbert Wiener
> >>>> >>
> >>>> >> https://www.cse.buffalo.edu/~knepley/
> >>>> >
> >>>> > <image003.jpg>
> >>>> >
> >>>> >
> >>>
> >>>
> >>>
> >>> --
> >>> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> >>> -- Norbert Wiener
> >>>
> >>> https://www.cse.buffalo.edu/~knepley/
> >
> >
> >
> > --
> > What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> > -- Norbert Wiener
> >
> > https://www.cse.buffalo.edu/~knepley/
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211207/e407bb9b/attachment.html>