[petsc-users] How to use f2py on a PETSc/SLEPc-based fortran code

[Gaetan Kenway]

We do this all the time. The trick is that you can't use f2py do actually
do *any* of the compiling/linking for you. You just have to use it to get
the <modulename>module.c file and .f90 file. Then compile that yourself and
use whatever compile flags, linking etc you would normally use to get a
.so. We've had issues with f2py using gcc to do the final link, instead of
the fortran compiler. The following approach we've used on a half a dozen
different wrapped fortran codes on a half a dozen different
computers/clusters and it has proven to work much more robustly than using
f2py directly.

Hope this helps
Gaetan

# Generate Python include directory
      $(eval PYTHON_INCLUDES = $(shell $(PYTHON-CONFIG) --includes))
     @ echo "#------------------------------------------------------#"
     @echo Python Inclue Flags $(PYTHON_INCLUDES)
     @echo "#------------------------------------------------------#"

# Generate Numpy inlude directory
$(eval NUMPY_INCLUDES = $(shell $(PYTHON) -c 'import numpy; print
numpy.get_include()'))
@echo "#------------------------------------------------------#"
@echo Numpy Include Directory: $(NUMPY_INCLUDES)
@echo "#------------------------------------------------------#"

# Generate f2py root directory
$(eval F2PY_ROOT = $(shell $(PYTHON) get_f2py.py))
@echo "#------------------------------------------------------#"
@echo f2py root directory: $(F2PY_ROOT)
@echo "#------------------------------------------------------#"

f2py warpustruct.pyf
@echo " "
$(CC) $(CC_ALL_FLAGS) $(PYTHON_INCLUDES) -I$(NUMPY_INCLUDES) \
-I$(F2PY_ROOT)/src -c warpustructmodule.c

$(CC) $(CC_ALL_FLAGS) $(PYTHON_INCLUDES) -I$(NUMPY_INCLUDES) -c \
$(F2PY_ROOT)/src/fortranobject.c -o fortranobject.o

        # Compiled f2py wrapper file
$(FF90) $(FF90_ALL_FLAGS) -I$(MAIN_DIR)/mod -c
warpustruct-f2pywrappers2.f90

# Final LInk

    enter code here

$(FF90) -shared $(PYTHON_OBJECTS) $(LINKER_ALL_FLAGS) -o warpustruct.so


On Thu, Mar 23, 2017 at 1:14 PM, Austin Herrema <aherrema at iastate.edu>
wrote:

> Sorry to self-bump this. Sounds like there are no quick and easy answers
> to get this solved, which I understand. I just wanted to say that if
> anybody has any suggestions about what I should be looking into
> specifically (C compiler? f2py? etc...) then even that would be helpful as
> I'm pretty stuck on this. And I have also posted a discussion on
> StackOverflow (http://stackoverflow.com/questions/42978049/unable-to-
> use-f2py-to-link-large-petsc-slepc-fortran-code?noredirect=
> 1#comment73055943_42978049) in case anything there might be useful (it's
> mostly all the same).
>
> Thank you regardless!
> Austin
>
> On Wed, Mar 22, 2017 at 2:08 PM, Austin Herrema <aherrema at iastate.edu>
> wrote:
>
>> Makes sense, and definitely seems to be a more natural way to go now that
>> I see it.
>>
>> When compiling using this rule it seems to get close but doesn't compile
>> all the way. Here is the output (in reality, what I was referring to as
>> "modname.so" is "iga_blade_py.so" and the "outer_driver.f90" is called
>> merely "run_analysis.f90"--sorry for the confusion):
>>
>> running build
>> running config_cc
>> unifing config_cc, config, build_clib, build_ext, build commands
>> --compiler options
>> running config_fc
>> unifing config_fc, config, build_clib, build_ext, build commands
>> --fcompiler options
>> running build_src
>> build_src
>> building extension "iga_blade_py" sources
>> f2py options: []
>> f2py:> /tmp/tmpIH70ZJ/src.macosx-10.10-x86_64-2.7/iga_blade_pymodule.c
>> creating /tmp/tmpIH70ZJ/src.macosx-10.10-x86_64-2.7
>> Reading fortran codes...
>> Reading file 'run_analysis.f90' (format:free)
>> Post-processing...
>> Block: iga_blade_py
>> Block: run_analysis
>> Post-processing (stage 2)...
>> Building modules...
>> Building module "iga_blade_py"...
>> Constructing wrapper function "run_analysis"...
>>  run_analysis()
>> Wrote C/API module "iga_blade_py" to file "/tmp/tmpIH70ZJ/src.macosx-10.
>> 10-x86_64-2.7/iga_blade_pymodule.c"
>>   adding '/tmp/tmpIH70ZJ/src.macosx-10.10-x86_64-2.7/fortranobject.c' to
>> sources.
>>   adding '/tmp/tmpIH70ZJ/src.macosx-10.10-x86_64-2.7' to include_dirs.
>> copying /usr/local/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.c
>> -> /tmp/tmpIH70ZJ/src.macosx-10.10-x86_64-2.7
>> copying /usr/local/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.h
>> -> /tmp/tmpIH70ZJ/src.macosx-10.10-x86_64-2.7
>> build_src: building npy-pkg config files
>> running build_ext
>> customize UnixCCompiler
>> customize UnixCCompiler using build_ext
>> customize Gnu95FCompiler
>> Found executable /usr/local/bin/gfortran
>> customize Gnu95FCompiler
>> customize Gnu95FCompiler using build_ext
>> building 'iga_blade_py' extension
>> compiling C sources
>> C compiler: clang -fno-strict-aliasing -fno-common -dynamic -g -O2
>> -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
>>
>> error: unknown file type '' (from '-Wl,-rpath,/usr/local/Cellar/
>> slepc/3.7.3_4/real/lib')
>> make: *** [iga_blade_py.so] Error 1
>>
>> On Wed, Mar 22, 2017 at 1:39 PM, Jose E. Roman <jroman at dsic.upv.es>
>> wrote:
>>
>>>
>>> > El 22 mar 2017, a las 19:23, Barry Smith <bsmith at mcs.anl.gov>
>>> escribió:
>>> >
>>> >
>>> >> On Mar 22, 2017, at 1:08 PM, Austin Herrema <aherrema at iastate.edu>
>>> wrote:
>>> >>
>>> >> Thank you for the suggestion! Seems like a reasonable way to go. Not
>>> working for me, however, I suspect because I'm using homebrew installations
>>> of PETSc and SLEPc (I don't think all the makefiles are kept). Any other
>>> way to do the same thing by chance? Worst case I could use a non-homebrew
>>> installation but I'd prefer not to mess with that if I can help it...
>>> >
>>> >   How do you link a "regular" SLEPc C program using the home-brew
>>> libraries? You need basically the same link line for f2py as you need for C
>>> programs.
>>>
>>>
>>> What Barry may be suggesting is: instead of using a script to invoke
>>> f2py, add a rule to your makefile
>>>
>>> modname.so: outer_driver.f90
>>>         f2py -c -m modname outer_driver.f90 file1.o file2.o file3.o
>>> ${SLEPC_EPS_LIB}
>>>
>>> Then 'make modname.so' will pick the libraries from SLEPc makefiles.
>>>
>>> Jose
>>>
>>> >
>>> >>
>>> >> Thanks,
>>> >> Austin
>>> >>
>>> >> On Wed, Mar 22, 2017 at 11:20 AM Jose E. Roman <jroman at dsic.upv.es>
>>> wrote:
>>> >> Try the following:
>>> >> $ cd $SLEPC_DIR
>>> >> $ make getlinklibs_slepc
>>> >> Then copy the output and paste it at the end of your f2py command.
>>> >>
>>> >> Jose
>>> >>
>>> >>
>>> >>> El 22 mar 2017, a las 16:38, Austin Herrema <aherrema at iastate.edu>
>>> escribió:
>>> >>>
>>> >>> Hello all,
>>> >>>
>>> >>> I am trying to do as the subject line describes--use f2py to run a
>>> large PETSc/SLEPc fortran finite element code through python. I really only
>>> need to wrap the outermost function of the fortran code--don't need any
>>> access to subroutines. I'll describe what I'm doing, some of which I'm not
>>> 100% confident is correct (not much experience with f2py)--feel free to
>>> correct/redirect any of it.
>>> >>>
>>> >>> First, I'm editing the fortran code so that the top-level function
>>> is a subroutine rather than a main program (it's my understanding that this
>>> is required for f2py?).
>>> >>>
>>> >>> I use my regular makefile (modeled after a standard SLEPc makefile
>>> from the user guide) to compile all of the .f90/.F90 files (many of them)
>>> to .o files using SLEPc/PETSc rules. The final linking phase fails since
>>> there isn't a main program, but I'm just ignoring that for now since that's
>>> not what I ultimately need...
>>> >>>
>>> >>> Using a python script, I set up and run the f2py command. Right now
>>> it has the form...
>>> >>> "f2py -c -m modname outer_driver.f90 file1.o file2.o file3.o..." etc.
>>> >>>
>>> >>> This appears to work, but upon attempting to import, it cannot find
>>> the SLEPc (and, I presume, PETSc) objects/functions:
>>> >>>
>>> >>>>>> import mod_name
>>> >>> Traceback (most recent call last):
>>> >>>  File "<stdin>", line 1, in <module>
>>> >>> ImportError: dlopen(./mod_name.so, 2): Symbol not found: _epscreate_
>>> >>>  Referenced from: ./mod_name.so
>>> >>>  Expected in: flat namespace
>>> >>> in ./mod_name.so
>>> >>>
>>> >>> Based on this discussion, I believe I need to somehow include
>>> PETSc/SLEPc info when linking with f2py. Is that correct? Any direction on
>>> how to do that? I don't quite understand what the OP of that question
>>> ultimately ended up doing to get it to work. I tried using the -I flag
>>> pointing to the slepc_common file (like the SLEPc makefile does). The
>>> problem is that that is a file, not a directory, which contains a number of
>>> other makefile-style variables--so it works to include it in a makefile,
>>> but doesn't work in python. Maybe there are only a few directories I really
>>> need to include? Or is it possible to somehow run f2py through a makefile?
>>> I'm a bit ignorant in that realm as well.
>>> >>>
>>> >>> Thank you for any help or suggestions!
>>> >>> Austin
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Austin Herrema
>>> >>> PhD Student | Graduate Research Assistant | Iowa State University
>>> >>> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>>> >>
>>> >> --
>>> >> Austin Herrema
>>> >> PhD Student | Graduate Research Assistant | Iowa State University
>>> >> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>>> >
>>>
>>>
>>
>>
>> --
>> *Austin Herrema*
>> PhD Student | Graduate Research Assistant | Iowa State University
>> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>>
>
>
>
> --
> *Austin Herrema*
> PhD Student | Graduate Research Assistant | Iowa State University
> Wind Energy Science, Engineering, and Policy | Mechanical Engineering
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170323/02067274/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: get_f2py.py
Type: text/x-python
Size: 565 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170323/02067274/attachment-0001.py>