[petsc-users] How to read/write a HDF5 file using petsc4py ?

Tue Dec 7 06:58:46 CST 2021

On Tue, Dec 7, 2021 at 3:55 AM Quentin Chevalier <
quentin.chevalier at polytechnique.edu> wrote:

> Hello Matthew,
>
> That would indeed make sense.
>
> Full log is attached, I grepped hdf5 in there and didn't find anything
> alarming.
>

At the top of this log:

Configure Options: --configModules=PETSc.Configure
--optionsModule=config.compilerOptions PETSC_ARCH=linux-gnu-complex-64
--COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-make-np=2
--with-64-bit-indices=yes --with-debugging=no --with-fortran-bindings=no
--with-shared-libraries --download-hypre --download-mumps
--download-ptscotch --download-scalapack --download-suitesparse
--download-superlu_dist --with-scalar-type=complex

So the HDF5 option is not being specified.

  Thanks,

     Matt

Cheers,
>
> Quentin
>
>
>
>
> [image: cid:image003.jpg at 01D690CB.3B3FDC10]
>
> Quentin CHEVALIER – IA parcours recherche
>
> LadHyX - Ecole polytechnique
>
> __________
>
>
> On Mon, 6 Dec 2021 at 21:39, Matthew Knepley <knepley at gmail.com> wrote:
>
>> On Mon, Dec 6, 2021 at 3:27 PM Quentin Chevalier <
>> quentin.chevalier at polytechnique.edu> wrote:
>>
>>> Fine. MWE is unchanged :
>>> * Run this docker container <https://hub.docker.com/r/dolfinx/dolfinx>
>>> * Do : python3 -c "from petsc4py import PETSc;
>>> PETSc.Viewer().createHDF5('dummy.h5')"
>>>
>>> Updated attempt at a fix :
>>> * cd /usr/local/petsc/
>>> * ./configure PETSC_ARCH= linux-gnu-real-32  PETSC_DIR=/usr/local/petsc
>>> --with-hdf5 --force
>>>
>>
>> Did it find HDF5? If not, it will shut it off. You need to send us
>>
>>   $PETSC_DIR/configure.log
>>
>> so we can see what happened in the configure run.
>>
>>   Thanks,
>>
>>       Matt
>>
>>
>>> * make PETSC_DIR=/usr/local/petsc PETSC-ARCH= linux-gnu-real-32 all
>>>
>>> Still no joy. The same error remains.
>>>
>>> Quentin
>>>
>>>
>>>
>>> On Mon, 6 Dec 2021 at 20:04, Pierre Jolivet <pierre at joliv.et> wrote:
>>> >
>>> >
>>> >
>>> > On 6 Dec 2021, at 7:42 PM, Quentin Chevalier <
>>> quentin.chevalier at polytechnique.edu> wrote:
>>> >
>>> > The PETSC_DIR exactly corresponds to the previous one, so I guess that
>>> rules option b) out, except if a specific option is required to overwrite a
>>> previous installation of PETSc. As for a), well I thought reconfiguring
>>> pretty direct, you're welcome to give me a hint as to what could be wrong
>>> in the following process.
>>> >
>>> > Steps to reproduce this behaviour are as follows :
>>> > * Run this docker container
>>> > * Do : python3 -c "from petsc4py import PETSc;
>>> PETSc.Viewer().createHDF5('dummy.h5')"
>>> >
>>> > After you get the error Unknown PetscViewer type, feel free to try :
>>> >
>>> > * cd /usr/local/petsc/
>>> > * ./configure --with-hfd5
>>> >
>>> >
>>> > It’s hdf5, not hfd5.
>>> > It’s PETSC_ARCH, not PETSC-ARCH.
>>> > PETSC_ARCH is missing from your configure line.
>>> >
>>> > Thanks,
>>> > Pierre
>>> >
>>> > * make PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 all
>>> >
>>> > Then repeat the MWE and observe absolutely no behavioural change
>>> whatsoever. I'm afraid I don't know PETSc well enough to be surprised by
>>> that.
>>> >
>>> > Quentin
>>> >
>>> >
>>> >
>>> > Quentin CHEVALIER – IA parcours recherche
>>> >
>>> > LadHyX - Ecole polytechnique
>>> >
>>> > __________
>>> >
>>> >
>>> >
>>> > On Mon, 6 Dec 2021 at 19:24, Matthew Knepley <knepley at gmail.com>
>>> wrote:
>>> >>
>>> >> On Mon, Dec 6, 2021 at 1:22 PM Quentin Chevalier <
>>> quentin.chevalier at polytechnique.edu> wrote:
>>> >>>
>>> >>> It failed all of the tests included in `make
>>> >>> PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 check`, with
>>> >>> the error `/usr/bin/bash: line 1: cd: src/snes/tutorials: No such
>>> file
>>> >>> or directory`
>>> >>>
>>> >>> I am therefore fairly confident this a "file absence" problem, and
>>> not
>>> >>> a compilation problem.
>>> >>>
>>> >>> I repeat that there was no error at compilation stage. The final
>>> stage
>>> >>> did present `gmake[3]: Nothing to be done for 'libs'.` but that's
>>> all.
>>> >>>
>>> >>> Again, running `./configure --with-hdf5` followed by a `make
>>> >>> PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 all` does not
>>> >>> change the problem. I get the same error at the same position as
>>> >>> before.
>>> >>
>>> >>
>>> >> If you reconfigured and rebuilt, it is impossible to get the same
>>> error, so
>>> >>
>>> >>   a) You did not reconfigure
>>> >>
>>> >>   b) Your new build is somewhere else on the machine
>>> >>
>>> >>   Thanks,
>>> >>
>>> >>      Matt
>>> >>
>>> >>>
>>> >>> I will comment I am running on OpenSUSE.
>>> >>>
>>> >>> Quentin
>>> >>>
>>> >>> On Mon, 6 Dec 2021 at 19:09, Matthew Knepley <knepley at gmail.com>
>>> wrote:
>>> >>> >
>>> >>> > On Mon, Dec 6, 2021 at 1:08 PM Quentin Chevalier <
>>> quentin.chevalier at polytechnique.edu> wrote:
>>> >>> >>
>>> >>> >> Hello Matthew and thanks for your quick response.
>>> >>> >>
>>> >>> >> I'm afraid I did try to snoop around the container and rerun
>>> PETSc's
>>> >>> >> configure with the --with-hdf5 option, to absolutely no avail.
>>> >>> >>
>>> >>> >> I didn't see any errors during config or make, but it failed the
>>> tests
>>> >>> >> (which aren't included in the minimal container I suppose)
>>> >>> >
>>> >>> >
>>> >>> > Failed which tests? What was the error?
>>> >>> >
>>> >>> >   Thanks,
>>> >>> >
>>> >>> >     Matt
>>> >>> >
>>> >>> >>
>>> >>> >> Quentin
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> Quentin CHEVALIER – IA parcours recherche
>>> >>> >>
>>> >>> >> LadHyX - Ecole polytechnique
>>> >>> >>
>>> >>> >> __________
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> On Mon, 6 Dec 2021 at 19:02, Matthew Knepley <knepley at gmail.com>
>>> wrote:
>>> >>> >> >
>>> >>> >> > On Mon, Dec 6, 2021 at 11:28 AM Quentin Chevalier <
>>> quentin.chevalier at polytechnique.edu> wrote:
>>> >>> >> >>
>>> >>> >> >> Hello PETSc users,
>>> >>> >> >>
>>> >>> >> >> This email is a duplicata of this gitlab issue, sorry for any
>>> inconvenience caused.
>>> >>> >> >>
>>> >>> >> >> I want to compute a PETSc vector in real mode, than perform
>>> calculations with it in complex mode. I want as much of this process to be
>>> parallel as possible. Right now, I compile PETSc in real mode, compute my
>>> vector and save it to a file, then switch to complex mode, read it, and
>>> move on.
>>> >>> >> >>
>>> >>> >> >> This creates unexpected behaviour using MPIIO, so on Lisandro
>>> Dalcinl's advice I'm moving to HDF5 format. My code is as follows (taking
>>> inspiration from petsc4py doc, a bitbucket example and another one, all top
>>> Google results for 'petsc hdf5') :
>>> >>> >> >>>
>>> >>> >> >>> viewer = PETSc.Viewer().createHDF5(file_name, 'r', COMM_WORLD)
>>> >>> >> >>> q.load(viewer)
>>> >>> >> >>> q.ghostUpdate(addv=PETSc.InsertMode.INSERT,
>>> mode=PETSc.ScatterMode.FORWARD)
>>> >>> >> >>
>>> >>> >> >>
>>> >>> >> >> This crashes my code. I obtain traceback :
>>> >>> >> >>>
>>> >>> >> >>>   File "/home/shared/code.py", line 121, in Load
>>> >>> >> >>>     viewer = PETSc.Viewer().createHDF5(file_name, 'r',
>>> COMM_WORLD)
>>> >>> >> >>>   File "PETSc/Viewer.pyx", line 182, in
>>> petsc4py.PETSc.Viewer.createHDF5
>>> >>> >> >>> petsc4py.PETSc.Error: error code 86
>>> >>> >> >>> [0] PetscViewerSetType() at
>>> /usr/local/petsc/src/sys/classes/viewer/interface/viewreg.c:442
>>> >>> >> >>> [0] Unknown type. Check for miss-spelling or missing package:
>>> https://petsc.org/release/install/install/#external-packages
>>> >>> >> >>> [0] Unknown PetscViewer type given: hdf5
>>> >>> >> >
>>> >>> >> > This means that PETSc has not been configured with HDF5
>>> (--with-hdf5 or --download-hdf5), so the container should be updated.
>>> >>> >> >
>>> >>> >> >   THanks,
>>> >>> >> >
>>> >>> >> >     Matt
>>> >>> >> >
>>> >>> >> >>
>>> >>> >> >> I have petsc4py 3.16 from this docker container (list of
>>> dependencies include PETSc and petsc4py).
>>> >>> >> >>
>>> >>> >> >> I'm pretty sure this is not intended behaviour. Any insight as
>>> to how to fix this issue (I tried running ./configure --with-hdf5 to no
>>> avail) or more generally to perform this jiggling between real and complex
>>> would be much appreciated,
>>> >>> >> >>
>>> >>> >> >> Kind regards.
>>> >>> >> >>
>>> >>> >> >> Quentin
>>> >>> >> >
>>> >>> >> >
>>> >>> >> >
>>> >>> >> > --
>>> >>> >> > What most experimenters take for granted before they begin
>>> their experiments is infinitely more interesting than any results to which
>>> their experiments lead.
>>> >>> >> > -- Norbert Wiener
>>> >>> >> >
>>> >>> >> > https://www.cse.buffalo.edu/~knepley/
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > --
>>> >>> > What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> >>> > -- Norbert Wiener
>>> >>> >
>>> >>> > https://www.cse.buffalo.edu/~knepley/
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> >> -- Norbert Wiener
>>> >>
>>> >> https://www.cse.buffalo.edu/~knepley/
>>> >
>>> > <image003.jpg>
>>> >
>>> >
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>> https://www.cse.buffalo.edu/~knepley/
>> <http://www.cse.buffalo.edu/~knepley/>
>>
>

-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211207/9841dbf1/attachment-0001.html>