[petsc-users] How to use f2py on a PETSc/SLEPc-based fortran code

Thu Mar 30 12:01:29 CDT 2017

Great. Best of luck with your Ph.D.
Gaetan

On Thu, Mar 30, 2017 at 7:12 AM, Austin Herrema <aherrema at iastate.edu>
wrote:

> Indeed it did seem to be an issue with the integer of value 500 in that
> function call (8 byte/4 byte? Don't know...). Upon explicitly using a
> PetscInt variable, everything works just fine.
>
> Thank you, everyone, for your patient help!
>
> Best,
> Austin
>
>
>
> On Wed, Mar 29, 2017 at 6:48 PM, Matthew Knepley <knepley at gmail.com>
> wrote:
>
>> On Wed, Mar 29, 2017 at 5:12 PM, Austin Herrema <aherrema at iastate.edu>
>> wrote:
>>
>>> Quick update on this issue in case it brings any other thoughts/ideas to
>>> light. For a very simple, small problem, I am successfully able to use
>>> MatSeqAIJSetPreallocation in a fortran-based code compiled for python via
>>> f2py. I am still unsure why, in a larger code, this particular function
>>> call fails when the code is executed in python (on a setup that runs fine
>>> under pure Fortran). Does the error " nnz cannot be greater than row
>>> length: local row 2 value 1330400321 rowlength 37065" imply that the
>>> program thinks I am trying to allocate 1330400321 nonzeros in a row of
>>> max length 37065?
>>>
>>
>> Yes.
>>
>>   Thanks,
>>
>>     Matt
>>
>>
>>> That is obviously not my intent nor what I think I have coded. I am
>>> trying to skip preallocation and use merely MatSetUp but, as we would
>>> expect, the dynamic allocation is ridiculously slow...
>>>
>>> On Wed, Mar 29, 2017 at 11:27 AM, Austin Herrema <aherrema at iastate.edu>
>>> wrote:
>>>
>>>> Got it--just had to link against other compiled source, as you said.
>>>> I've attached my makefile for doing everything (including variable
>>>> definitions, compiling source, and running requisite f2py commands) in case
>>>> that's helpful for anyone else trying to do something similar. But
>>>> obviously the meat of it is in what Gaetan provided.
>>>>
>>>> I am now able to successfully run simple PETSc-based fortran codes in
>>>> python. For a larger, more complex code, I am getting some PETSc errors
>>>> when running in python that I don't normally get. In particular,
>>>> preallocation is failing--the relevant fortran code block and PETSc error
>>>> is below.
>>>>
>>>>
>>>>     call MatCreate(PETSC_COMM_WORLD, LHS_pc, pc_ier)
>>>>     call MatSetSizes(LHS_pc, PETSC_DECIDE, PETSC_DECIDE, NSD*FUN%NNODE,
>>>> NSD*FUN%NNODE, pc_ier)
>>>>     call MatSetFromOptions(LHS_pc, pc_ier)
>>>>     call MatSeqAIJSetPreallocation(LHS_pc, 500, PETSC_NULL_INTEGER,
>>>> pc_ier)
>>>>
>>>>
>>>> [0]PETSC ERROR: --------------------- Error Message
>>>> --------------------------------------------------------------
>>>> [0]PETSC ERROR: Argument out of range
>>>> [0]PETSC ERROR: nnz cannot be greater than row length: local row 2
>>>> value 1330400321 rowlength 37065
>>>> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
>>>> for trouble shooting.
>>>> [0]PETSC ERROR: Petsc Release Version 3.7.5, Jan, 01, 2017
>>>> [0]PETSC ERROR: Unknown Name on a real named
>>>> austin-ethernet.student.iastate.edu by Austin Wed Mar 29 10:59:33 2017
>>>> [0]PETSC ERROR: Configure options CC=/usr/local/bin/mpicc
>>>> CXX=/usr/local/bin/mpicxx F77=/usr/local/bin/mpif77
>>>> FC=/usr/local/bin/mpif90 --with-shared-libraries=1 --with-pthread=0
>>>> --with-openmp=0 --with-debugging=1 --with-ssl=0
>>>> --with-superlu_dist-include=/usr/local/opt/superlu_dist/include
>>>> --with-superlu_dist-lib="-L/usr/local/opt/superlu_dist/lib
>>>> -lsuperlu_dist" --with-fftw-dir=/usr/local/opt/fftw
>>>> --with-netcdf-dir=/usr/local/opt/netcdf --with-suitesparse-dir=/usr/local/opt/suite-sparse
>>>> --with-hdf5-dir=/usr/local/opt/hdf5 --with-metis-dir=/usr/local/opt/metis
>>>> --with-parmetis-dir=/usr/local/opt/parmetis
>>>> --with-scalapack-dir=/usr/local/opt/scalapack
>>>> --with-mumps-dir=/usr/local/opt/mumps/libexec --with-x=0
>>>> --prefix=/usr/local/Cellar/petsc/3.7.5/real --with-scalar-type=real
>>>> --with-hypre-dir=/usr/local/opt/hypre --with-sundials-dir=/usr/local/opt/sundials
>>>> --with-hwloc-dir=/usr/local/opt/hwloc
>>>> [0]PETSC ERROR: #1 MatSeqAIJSetPreallocation_SeqAIJ() line 3598 in
>>>> /private/tmp/petsc-20170223-508-1xeniyc/petsc-3.7.5/src/mat/
>>>> impls/aij/seq/aij.c
>>>> [0]PETSC ERROR: #2 MatSeqAIJSetPreallocation() line 3570 in
>>>> /private/tmp/petsc-20170223-508-1xeniyc/petsc-3.7.5/src/mat/
>>>> impls/aij/seq/aij.c
>>>>
>>>>
>>>> Is there anything about the MatSeqAIJSetPreallocation function that
>>>> would make it not work correctly in Python even though everything else
>>>> seems to work properly? If anyone has thoughts on this that would be great.
>>>> But, again, I do realize I'm venturing into potentially unsupported
>>>> territory.
>>>>
>>>>
>>>> On Tue, Mar 28, 2017 at 4:53 PM, Gaetan Kenway <gaetank at gmail.com>
>>>> wrote:
>>>>
>>>>> Looks like it isn't finding your source from run_analysis.f90. You
>>>>> still need to compile that yourself and include in the final link. In my
>>>>> example, all the "original" source code was precompiled into a library from
>>>>> a different makefile and then this was run after-the-fact.
>>>>>
>>>>> Gaetan
>>>>>
>>>>> On Tue, Mar 28, 2017 at 2:38 PM, Austin Herrema <aherrema at iastate.edu>
>>>>> wrote:
>>>>>
>>>>>> Gotcha. In that case, it seems I should be good without that line.
>>>>>> I've gotten the compile to succeed, but upon attempting to import the
>>>>>> module I get the following:
>>>>>>
>>>>>> >>> import run_analysis_final
>>>>>> Traceback (most recent call last):
>>>>>>   File "<stdin>", line 1, in <module>
>>>>>> ImportError: dlopen(./run_analysis_final.so, 2): Symbol not found:
>>>>>> _run_analysis_
>>>>>>   Referenced from: ./run_analysis_final.so
>>>>>>   Expected in: flat namespace
>>>>>>  in ./run_analysis_final.so
>>>>>>
>>>>>> Seems I may have gotten the linking wrong somehow. Will keep
>>>>>> searching, but the simplified makefile that I used is attached in case
>>>>>> anyone thinks they might be able to spot the issue in it. That said, I do
>>>>>> realize that this may be starting to reach beyond the scope of this mailing
>>>>>> list so feel free to ignore...
>>>>>>
>>>>>> On Tue, Mar 28, 2017 at 2:31 PM, Gaetan Kenway <gaetank at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> You only get that file if you have wrapped a module explicitly in
>>>>>>> the .pyf file. If you haven't wrapped a module, that doesn't get created.
>>>>>>>
>>>>>>> Gaetan
>>>>>>>
>>>>>>> On Tue, Mar 28, 2017 at 12:28 PM, Austin Herrema <
>>>>>>> aherrema at iastate.edu> wrote:
>>>>>>>
>>>>>>>> Gaetan,
>>>>>>>>
>>>>>>>> Thank you for this. With your help, I think I am getting close to
>>>>>>>> getting this to work for my case. At the moment, I am hung up on the line
>>>>>>>> of your makefile which reads "$(FF90) $(FF90_ALL_FLAGS) -I$(MAIN_DIR)/mod
>>>>>>>> -c warpustruct-f2pywrappers2.f90". Am I correct that
>>>>>>>> warpustruct-f2pywrappers2.f90 should be generated by f2py? If so, do you
>>>>>>>> (or does anyone else) know the command for telling f2py to do so? At the
>>>>>>>> moment I am using:
>>>>>>>>
>>>>>>>> f2py run_analysis.f90 -m run_analysis -h run_analysis.pyf
>>>>>>>>
>>>>>>>> to get the requisite .pyf and .c files, but no .f90 file. If I am
>>>>>>>> wrong about the origin of this file, please do tell me!
>>>>>>>>
>>>>>>>> Thank you,
>>>>>>>> Austin
>>>>>>>>
>>>>>>>> On Mon, Mar 27, 2017 at 5:13 PM, Gaetan Kenway <gaetank at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Austin
>>>>>>>>>
>>>>>>>>> Here is the full makefile for a code we use. The variables defined
>>>>>>>>> externally in a separate config file are:
>>>>>>>>> $(FF90)
>>>>>>>>> $(FF90_FLAGS)
>>>>>>>>> $(LIBDIR)
>>>>>>>>> $(PETSC_LINKER_FLAGS)
>>>>>>>>> $(LINKER_FLAGS)
>>>>>>>>> $(CGNS_LINKER_FLAGS)
>>>>>>>>>
>>>>>>>>> $(PYTHON)
>>>>>>>>> $(PYTHON-CONIFG)
>>>>>>>>> $(F2PY)
>>>>>>>>> (These are usually use python, python-config and f2py. You can
>>>>>>>>> overwrite as necessary)
>>>>>>>>>
>>>>>>>>> $(CC)
>>>>>>>>> $(CC_ALL_FLAGS)
>>>>>>>>>
>>>>>>>>> This essentially just mimics what f2py does automatically but we
>>>>>>>>> found it easier to control exactly what is going on. Essentially you are
>>>>>>>>> just compiling exactly as you normally an executable, but instead make a
>>>>>>>>> .so (with the -shared option) and including the additional .o files
>>>>>>>>> generated by compiling the f2py-generated wrappers.
>>>>>>>>>
>>>>>>>>> Hope this helps,
>>>>>>>>> Gaetan
>>>>>>>>>
>>>>>>>>> On Sat, Mar 25, 2017 at 5:38 AM, Lisandro Dalcin <
>>>>>>>>> dalcinl at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 22 March 2017 at 20:29, Barry Smith <bsmith at mcs.anl.gov>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>   Lisandro,
>>>>>>>>>>>
>>>>>>>>>>>     We've had a couple questions similar to this with f2py; is
>>>>>>>>>>> there a way we could add to the PETSc/SLEPc makefile rules something to
>>>>>>>>>>> allow people to trivially use f2py without having to make their own (often
>>>>>>>>>>> incorrect) manual command lines?
>>>>>>>>>>>
>>>>>>>>>>>    Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Barry, it is quite hard and hacky to get f2py working in the
>>>>>>>>>> general case. I think the email from Gaetan in this thread proves my point.
>>>>>>>>>>
>>>>>>>>>> IMHO, it is easier to write a small Fortran source exposing the
>>>>>>>>>> API to call using ISO_C_BINDINGS, then wrap that code with the more
>>>>>>>>>> traditional C-based "static" tools (SWIG, Cython) or even "dynamically"
>>>>>>>>>> with ctypes or cffi (which use dlopen'ing).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Lisandro Dalcin
>>>>>>>>>> ============
>>>>>>>>>> Research Scientist
>>>>>>>>>> Computer, Electrical and Mathematical Sciences & Engineering
>>>>>>>>>> (CEMSE)
>>>>>>>>>> Extreme Computing Research Center (ECRC)
>>>>>>>>>> King Abdullah University of Science and Technology (KAUST)
>>>>>>>>>> http://ecrc.kaust.edu.sa/
>>>>>>>>>>
>>>>>>>>>> 4700 King Abdullah University of Science and Technology
>>>>>>>>>> al-Khawarizmi Bldg (Bldg 1), Office # 0109
>>>>>>>>>> Thuwal 23955-6900, Kingdom of Saudi Arabia
>>>>>>>>>> http://www.kaust.edu.sa
>>>>>>>>>>
>>>>>>>>>> Office Phone: +966 12 808-0459 <+966%2012%20808%200459>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Austin Herrema*
>>>>>>>> PhD Student | Graduate Research Assistant | Iowa State University
>>>>>>>> Wind Energy Science, Engineering, and Policy | Mechanical
>>>>>>>> Engineering
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Austin Herrema*
>>>>>> PhD Student | Graduate Research Assistant | Iowa State University
>>>>>> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Austin Herrema*
>>>> PhD Student | Graduate Research Assistant | Iowa State University
>>>> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>>>>
>>>
>>>
>>>
>>> --
>>> *Austin Herrema*
>>> PhD Student | Graduate Research Assistant | Iowa State University
>>> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>>>
>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>
>
>
> --
> *Austin Herrema*
> PhD Student | Graduate Research Assistant | Iowa State University
> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170330/63781f04/attachment.html>