[petsc-users] How to read/write a HDF5 file using petsc4py ?

Tue Dec 7 10:47:26 CST 2021

  Something is really wrong with the process, you should not have to waste all this time on a simple build that in theory is super simple given it is using docker. 

  We debug these things by looking at the output files from PETSc: configure.log make.log and the output from make check. Are these files available when you do the docker install business? Without these files we are in the dark just completely guessing what is happening. If the docker is using a prebuilt PETSc inside iteself then there is no way to add hdf5 after the fact.

  Barry

> On Dec 7, 2021, at 11:15 AM, Quentin Chevalier <quentin.chevalier at polytechnique.edu> wrote:
> 
> @Lawrence, thanks for the details, but point 7 fails with : 
> 
> ERROR: Invalid requirement: '/usr/local/petsc/src/binding/petsc4py'
> Hint: It looks like a path. File '/usr/local/petsc/src/binding/petsc4py' does not exist.
> 
> Just like @wence's tentative command. I've changed the dockerfile of dolfinx to add the --with-hdf5 flag and I'm trying to build a new optimised docker image with :
> 
> echo '{ "cffi_extra_compile_args" : ["-O2", "-march=native" ] }' > dolfinx_jit_parameters.json
> docker build --target dolfinx --file Dockerfile --build-arg PETSC_SLEPC_OPTFLAGS="-O2 -march=native" --build-arg DOLFINX_CMAKE_CXX_FLAGS="-march=native" .
> 
> It's slowgoing, but it might eventually do the trick I guess.
> 
> @bsmith, the --with-petsc4py flag changes nothing.
>  
> MWE is unchanged :
> * Run this docker container <https://hub.docker.com/r/dolfinx/dolfinx>
> * Do : python3 -c "from petsc4py import PETSc; PETSc.Viewer().createHDF5('dummy.h5')"
> 
> Updated attempt at a fix :
> * cd /usr/local/petsc/
> * ./configure --with-hdf5 --with-petsc4py --force (turns out the container sets PETSC_ARCH and PETSC_DIR as environment variables by default)
> * make all
> 
> This still produces the same error :
> 
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
>   File "PETSc/Viewer.pyx", line 182, in petsc4py.PETSc.Viewer.createHDF5
> petsc4py.PETSc.Error: error code 86
> [0] PetscViewerSetType() at /usr/local/petsc/src/sys/classes/viewer/interface/viewreg.c:442
> [0] Unknown type. Check for miss-spelling or missing package: https://petsc.org/release/install/install/#external-packages <https://petsc.org/release/install/install/#external-packages>
> [0] Unknown PetscViewer type given: hdf5
> 
> Quentin
> 
> 
>  
> 
> 
> Quentin CHEVALIER – IA parcours recherche
> LadHyX - Ecole polytechnique
> 
> __________
> 
> 
> 
> On Tue, 7 Dec 2021 at 16:52, Barry Smith <bsmith at petsc.dev <mailto:bsmith at petsc.dev>> wrote:
> 
>   You can also just add --with-petsc4py to your PETSc configure command and it will manage automatically the petsc4py install. So
> 
> ./configure --with-hdf5 --any-other --configure-flags --you-want --with-petsc4py
> 
> 
>  Some people don't like this approach, I don't understand exactly why not; it should be equivalent (if it is not equivalent then perhaps it could be fixed?).
> 
>> On Dec 7, 2021, at 10:18 AM, Lawrence Mitchell <wence at gmx.li <mailto:wence at gmx.li>> wrote:
>> 
>> Comments inline below:
>> 
>>> On 7 Dec 2021, at 14:43, Quentin Chevalier <quentin.chevalier at polytechnique.edu <mailto:quentin.chevalier at polytechnique.edu>> wrote:
>>> 
>>> @Matthew, as stated before, error output is unchanged, i.e.the python
>>> command below produces the same traceback :
>>> 
>>> # python3 -c "from petsc4py import PETSc; PETSc.Viewer().createHDF5('d.h5')"
>>> Traceback (most recent call last):
>>> File "<string>", line 1, in <module>
>>> File "PETSc/Viewer.pyx", line 182, in petsc4py.PETSc.Viewer.createHDF5
>>> petsc4py.PETSc.Error: error code 86
>>> [0] PetscViewerSetType() at
>>> /usr/local/petsc/src/sys/classes/viewer/interface/viewreg.c:442
>>> [0] Unknown type. Check for miss-spelling or missing package:
>>> https://petsc.org/release/install/install/#external-packages <https://petsc.org/release/install/install/#external-packages>
>>> [0] Unknown PetscViewer type given: hdf5
>>> 
>>> @Wence that makes sense. I'd assumed that the original PETSc had been
>>> overwritten, and if the linking has gone wrong I'm surprised anything
>>> happens with petsc4py at all.
>>> 
>>> Your tentative command gave :
>>> 
>>> ERROR: Invalid requirement: '/usr/local/petsc/src/binding/petsc4py'
>>> Hint: It looks like a path. File
>>> '/usr/local/petsc/src/binding/petsc4py' does not exist.
>>> 
>>> So I tested that global variables PETSC_ARCH & PETSC_DIR were correct
>>> then ran "pip install petsc4py" to restart petsc4py from scratch.
>> 
>> This downloads petsc4py from pypi. It is not guaranteed to give you a version that matches the PETSc version you have installed (which is the source of your error below)
>> 
>> 
>>> This
>>> gives rise to a different error :
>>> 
>> [...]
>>> ImportError: /usr/local/lib/python3.9/dist-packages/petsc4py/lib/linux-gnu-real-32/PETSc.cpython-39-x86_64-linux-gnu.so <http://petsc.cpython-39-x86_64-linux-gnu.so/>:
>>> undefined symbol: petscstack
>>> 
>>> Not sure that it a step forward ; looks like petsc4py is broken now.
>> 
>> The steps to build PETSc with HDF5 support and then get a compatible petsc4py are:
>> 
>> 1. Download the PETSc source somehow (https://petsc.org/release/download/ <https://petsc.org/release/download/>)
>> 
>> I now assume that this source tree lives in .../petsc
>> 
>> 2. cd .../petsc
>> 
>> 3. ./configure --with-hdf5 --any-other --configure-flags --you-want
>> 
>> 4. Run the appropriate "make" command as suggested by configure
>> 
>> 5. Run the appropriate "make check" command as suggested by configure
>> 
>> 6. Set PETSC_DIR and PETSC_ARCH appropriately
>> 
>> 7. pip install src/binding/petsc4py
>> 
>> If you are working the docker container from dolfinx/dolfinx, you can see the commands that are run to install PETSc, and then petsc4py, here https://github.com/FEniCS/dolfinx/blob/main/docker/Dockerfile#L243 <https://github.com/FEniCS/dolfinx/blob/main/docker/Dockerfile#L243>
>> 
>> If you want to reproduce these versions of PETSc but with the addition of HDF5 support, just add --with-hdf5 to all of the relevant configure lines.
>> 
>> Lawrence
>> 
>> 
> 
> <image003.jpg>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20211207/085b4c88/attachment-0001.html>