[petsc-users] How to read/write a HDF5 file using petsc4py ?

Quentin Chevalier quentin.chevalier at polytechnique.edu
Tue Dec 7 07:26:30 CST 2021 

Ok my bad, that log corresponded to a tentative --download-hdf5. This
log corresponds to the commands given above and has --with-hdf5 in its
options.

The whole process still results in the same error.

Quentin



Quentin CHEVALIER – IA parcours recherche

LadHyX - Ecole polytechnique

__________



On Tue, 7 Dec 2021 at 13:59, Matthew Knepley <knepley at gmail.com> wrote:
>
> On Tue, Dec 7, 2021 at 3:55 AM Quentin Chevalier <quentin.chevalier at polytechnique.edu> wrote:
>>
>> Hello Matthew,
>>
>> That would indeed make sense.
>>
>> Full log is attached, I grepped hdf5 in there and didn't find anything alarming.
>
>
> At the top of this log:
>
> Configure Options: --configModules=PETSc.Configure --optionsModule=config.compilerOptions PETSC_ARCH=linux-gnu-complex-64 --COPTFLAGS=-O2 --CXXOPTFLAGS=-O2 --FOPTFLAGS=-O2 --with-make-np=2 --with-64-bit-indices=yes --with-debugging=no --with-fortran-bindings=no --with-shared-libraries --download-hypre --download-mumps --download-ptscotch --download-scalapack --download-suitesparse --download-superlu_dist --with-scalar-type=complex
>
>
> So the HDF5 option is not being specified.
>
>   Thanks,
>
>      Matt
>
>> Cheers,
>>
>> Quentin
>>
>>
>>
>>
>> Quentin CHEVALIER – IA parcours recherche
>>
>> LadHyX - Ecole polytechnique
>>
>> __________
>>
>>
>>
>> On Mon, 6 Dec 2021 at 21:39, Matthew Knepley <knepley at gmail.com> wrote:
>>>
>>> On Mon, Dec 6, 2021 at 3:27 PM Quentin Chevalier <quentin.chevalier at polytechnique.edu> wrote:
>>>>
>>>> Fine. MWE is unchanged :
>>>> * Run this docker container
>>>> * Do : python3 -c "from petsc4py import PETSc; PETSc.Viewer().createHDF5('dummy.h5')"
>>>>
>>>> Updated attempt at a fix :
>>>> * cd /usr/local/petsc/
>>>> * ./configure PETSC_ARCH= linux-gnu-real-32  PETSC_DIR=/usr/local/petsc --with-hdf5 --force
>>>
>>>
>>> Did it find HDF5? If not, it will shut it off. You need to send us
>>>
>>>   $PETSC_DIR/configure.log
>>>
>>> so we can see what happened in the configure run.
>>>
>>>   Thanks,
>>>
>>>       Matt
>>>
>>>>
>>>> * make PETSC_DIR=/usr/local/petsc PETSC-ARCH= linux-gnu-real-32 all
>>>>
>>>> Still no joy. The same error remains.
>>>>
>>>> Quentin
>>>>
>>>>
>>>>
>>>> On Mon, 6 Dec 2021 at 20:04, Pierre Jolivet <pierre at joliv.et> wrote:
>>>> >
>>>> >
>>>> >
>>>> > On 6 Dec 2021, at 7:42 PM, Quentin Chevalier <quentin.chevalier at polytechnique.edu> wrote:
>>>> >
>>>> > The PETSC_DIR exactly corresponds to the previous one, so I guess that rules option b) out, except if a specific option is required to overwrite a previous installation of PETSc. As for a), well I thought reconfiguring pretty direct, you're welcome to give me a hint as to what could be wrong in the following process.
>>>> >
>>>> > Steps to reproduce this behaviour are as follows :
>>>> > * Run this docker container
>>>> > * Do : python3 -c "from petsc4py import PETSc; PETSc.Viewer().createHDF5('dummy.h5')"
>>>> >
>>>> > After you get the error Unknown PetscViewer type, feel free to try :
>>>> >
>>>> > * cd /usr/local/petsc/
>>>> > * ./configure --with-hfd5
>>>> >
>>>> >
>>>> > It’s hdf5, not hfd5.
>>>> > It’s PETSC_ARCH, not PETSC-ARCH.
>>>> > PETSC_ARCH is missing from your configure line.
>>>> >
>>>> > Thanks,
>>>> > Pierre
>>>> >
>>>> > * make PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 all
>>>> >
>>>> > Then repeat the MWE and observe absolutely no behavioural change whatsoever. I'm afraid I don't know PETSc well enough to be surprised by that.
>>>> >
>>>> > Quentin
>>>> >
>>>> >
>>>> >
>>>> > Quentin CHEVALIER – IA parcours recherche
>>>> >
>>>> > LadHyX - Ecole polytechnique
>>>> >
>>>> > __________
>>>> >
>>>> >
>>>> >
>>>> > On Mon, 6 Dec 2021 at 19:24, Matthew Knepley <knepley at gmail.com> wrote:
>>>> >>
>>>> >> On Mon, Dec 6, 2021 at 1:22 PM Quentin Chevalier <quentin.chevalier at polytechnique.edu> wrote:
>>>> >>>
>>>> >>> It failed all of the tests included in `make
>>>> >>> PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 check`, with
>>>> >>> the error `/usr/bin/bash: line 1: cd: src/snes/tutorials: No such file
>>>> >>> or directory`
>>>> >>>
>>>> >>> I am therefore fairly confident this a "file absence" problem, and not
>>>> >>> a compilation problem.
>>>> >>>
>>>> >>> I repeat that there was no error at compilation stage. The final stage
>>>> >>> did present `gmake[3]: Nothing to be done for 'libs'.` but that's all.
>>>> >>>
>>>> >>> Again, running `./configure --with-hdf5` followed by a `make
>>>> >>> PETSC_DIR=/usr/local/petsc PETSC-ARCH=linux-gnu-real-32 all` does not
>>>> >>> change the problem. I get the same error at the same position as
>>>> >>> before.
>>>> >>
>>>> >>
>>>> >> If you reconfigured and rebuilt, it is impossible to get the same error, so
>>>> >>
>>>> >>   a) You did not reconfigure
>>>> >>
>>>> >>   b) Your new build is somewhere else on the machine
>>>> >>
>>>> >>   Thanks,
>>>> >>
>>>> >>      Matt
>>>> >>
>>>> >>>
>>>> >>> I will comment I am running on OpenSUSE.
>>>> >>>
>>>> >>> Quentin
>>>> >>>
>>>> >>> On Mon, 6 Dec 2021 at 19:09, Matthew Knepley <knepley at gmail.com> wrote:
>>>> >>> >
>>>> >>> > On Mon, Dec 6, 2021 at 1:08 PM Quentin Chevalier <quentin.chevalier at polytechnique.edu> wrote:
>>>> >>> >>
>>>> >>> >> Hello Matthew and thanks for your quick response.
>>>> >>> >>
>>>> >>> >> I'm afraid I did try to snoop around the container and rerun PETSc's
>>>> >>> >> configure with the --with-hdf5 option, to absolutely no avail.
>>>> >>> >>
>>>> >>> >> I didn't see any errors during config or make, but it failed the tests
>>>> >>> >> (which aren't included in the minimal container I suppose)
>>>> >>> >
>>>> >>> >
>>>> >>> > Failed which tests? What was the error?
>>>> >>> >
>>>> >>> >   Thanks,
>>>> >>> >
>>>> >>> >     Matt
>>>> >>> >
>>>> >>> >>
>>>> >>> >> Quentin
>>>> >>> >>
>>>> >>> >>
>>>> >>> >>
>>>> >>> >> Quentin CHEVALIER – IA parcours recherche
>>>> >>> >>
>>>> >>> >> LadHyX - Ecole polytechnique
>>>> >>> >>
>>>> >>> >> __________
>>>> >>> >>
>>>> >>> >>
>>>> >>> >>
>>>> >>> >> On Mon, 6 Dec 2021 at 19:02, Matthew Knepley <knepley at gmail.com> wrote:
>>>> >>> >> >
>>>> >>> >> > On Mon, Dec 6, 2021 at 11:28 AM Quentin Chevalier <quentin.chevalier at polytechnique.edu> wrote:
>>>> >>> >> >>
>>>> >>> >> >> Hello PETSc users,
>>>> >>> >> >>
>>>> >>> >> >> This email is a duplicata of this gitlab issue, sorry for any inconvenience caused.
>>>> >>> >> >>
>>>> >>> >> >> I want to compute a PETSc vector in real mode, than perform calculations with it in complex mode. I want as much of this process to be parallel as possible. Right now, I compile PETSc in real mode, compute my vector and save it to a file, then switch to complex mode, read it, and move on.
>>>> >>> >> >>
>>>> >>> >> >> This creates unexpected behaviour using MPIIO, so on Lisandro Dalcinl's advice I'm moving to HDF5 format. My code is as follows (taking inspiration from petsc4py doc, a bitbucket example and another one, all top Google results for 'petsc hdf5') :
>>>> >>> >> >>>
>>>> >>> >> >>> viewer = PETSc.Viewer().createHDF5(file_name, 'r', COMM_WORLD)
>>>> >>> >> >>> q.load(viewer)
>>>> >>> >> >>> q.ghostUpdate(addv=PETSc.InsertMode.INSERT, mode=PETSc.ScatterMode.FORWARD)
>>>> >>> >> >>
>>>> >>> >> >>
>>>> >>> >> >> This crashes my code. I obtain traceback :
>>>> >>> >> >>>
>>>> >>> >> >>>   File "/home/shared/code.py", line 121, in Load
>>>> >>> >> >>>     viewer = PETSc.Viewer().createHDF5(file_name, 'r', COMM_WORLD)
>>>> >>> >> >>>   File "PETSc/Viewer.pyx", line 182, in petsc4py.PETSc.Viewer.createHDF5
>>>> >>> >> >>> petsc4py.PETSc.Error: error code 86
>>>> >>> >> >>> [0] PetscViewerSetType() at /usr/local/petsc/src/sys/classes/viewer/interface/viewreg.c:442
>>>> >>> >> >>> [0] Unknown type. Check for miss-spelling or missing package: https://petsc.org/release/install/install/#external-packages
>>>> >>> >> >>> [0] Unknown PetscViewer type given: hdf5
>>>> >>> >> >
>>>> >>> >> > This means that PETSc has not been configured with HDF5 (--with-hdf5 or --download-hdf5), so the container should be updated.
>>>> >>> >> >
>>>> >>> >> >   THanks,
>>>> >>> >> >
>>>> >>> >> >     Matt
>>>> >>> >> >
>>>> >>> >> >>
>>>> >>> >> >> I have petsc4py 3.16 from this docker container (list of dependencies include PETSc and petsc4py).
>>>> >>> >> >>
>>>> >>> >> >> I'm pretty sure this is not intended behaviour. Any insight as to how to fix this issue (I tried running ./configure --with-hdf5 to no avail) or more generally to perform this jiggling between real and complex would be much appreciated,
>>>> >>> >> >>
>>>> >>> >> >> Kind regards.
>>>> >>> >> >>
>>>> >>> >> >> Quentin
>>>> >>> >> >
>>>> >>> >> >
>>>> >>> >> >
>>>> >>> >> > --
>>>> >>> >> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>>> >>> >> > -- Norbert Wiener
>>>> >>> >> >
>>>> >>> >> > https://www.cse.buffalo.edu/~knepley/
>>>> >>> >
>>>> >>> >
>>>> >>> >
>>>> >>> > --
>>>> >>> > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>>> >>> > -- Norbert Wiener
>>>> >>> >
>>>> >>> > https://www.cse.buffalo.edu/~knepley/
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>>> >> -- Norbert Wiener
>>>> >>
>>>> >> https://www.cse.buffalo.edu/~knepley/
>>>> >
>>>> > <image003.jpg>
>>>> >
>>>> >
>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>> -- Norbert Wiener
>>>
>>> https://www.cse.buffalo.edu/~knepley/
>
>
>
> --
> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: configure.log
Type: text/x-log
Size: 863767 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211207/8ceebb31/attachment-0001.bin>

More information about the petsc-users mailing list