[petsc-dev] integrate petsc4py tarball generation with petsc tarball generation

[Patrick Sanan]

I tried earlier but I don't know what I'm doing, so was hoping Lisandro would help. If it's worth anything this is what I did:

# set PETSC_DIR and PETSC_ARCH for minimal docs build as used for Sphinx (basically just c2html and sowing).
# had to remember to actually build the library or you get a link error with -lpetsc 
cd src/binding/petsc4py
# I have all the requisite packages in my base Python 3.7 conda environment (I think), including epydoc
make PYTHON=python PYTHON2=python PYTHON3=python docs

/Users/patrick/opt/miniconda3/bin/rst2html.py --input-encoding=utf-8 --no-compact-lists --cloak-email-addresses ./LICENSE.rst  > docs/LICENSE.html
/Users/patrick/opt/miniconda3/bin/rst2html.py --input-encoding=utf-8 --no-compact-lists --cloak-email-addresses ./CHANGES.rst  > docs/CHANGES.html
/Users/patrick/opt/miniconda3/bin/rst2html.py --input-encoding=utf-8 --no-compact-lists --cloak-email-addresses docs/index.rst > docs/index.html
mkdir -p build/doctrees docs/usrman
sphinx-build -b html -d build/doctrees  \
	docs/source docs/usrman
Sphinx v2.4.4 in Verwendung
loading pickled environment... erledigt
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.
build abgeschlossen.

The HTML pages are in docs/usrman.
rm -f docs/usrman/.buildinfo
python setup.py build_src
running build_src
mkdir -p docs/apiref
env CFLAGS=-O0 python setup.py -q build --build-lib build/lib.py2
env PYTHONPATH=$PWD/build/lib.py2 python -c 'import petsc4py.PETSc'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/patrick/code/petsc-doc/src/binding/petsc4py/build/lib.py2/petsc4py/PETSc.py", line 3, in <module>
    PETSc = ImportPETSc(ARCH)
  File "/Users/patrick/code/petsc-doc/src/binding/petsc4py/build/lib.py2/petsc4py/lib/__init__.py", line 29, in ImportPETSc
    return Import('petsc4py', 'PETSc', path, arch)
  File "/Users/patrick/code/petsc-doc/src/binding/petsc4py/build/lib.py2/petsc4py/lib/__init__.py", line 73, in Import
    module = import_module(pkg, name, path, arch)
  File "/Users/patrick/code/petsc-doc/src/binding/petsc4py/build/lib.py2/petsc4py/lib/__init__.py", line 58, in import_module
    with f: return imp.load_module(fullname, f, fn, info)
  File "/Users/patrick/opt/miniconda3/lib/python3.7/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/Users/patrick/opt/miniconda3/lib/python3.7/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: dlopen(/Users/patrick/code/petsc-doc/src/binding/petsc4py/build/lib.py2/petsc4py/lib/arch-classic-docs/PETSc.cpython-37m-darwin.so, 2): Symbol not found: _dasum
  Referenced from: /Users/patrick/code/petsc-doc/arch-classic-docs/lib/libpetsc.3.014.dylib
  Expected in: flat namespace
 in /Users/patrick/code/petsc-doc/arch-classic-docs/lib/libpetsc.3.014.dylib
make: *** [epydoc-html] Error 1


> Am 15.03.2021 um 16:59 schrieb Barry Smith <bsmith at petsc.dev>:
> 
> 
>    Well someone tried importing epydoc for python3 and it did not generate an error, did they try generating the petsc4py docs with it. That would tell you if it works?
> 
>    Yes, we do want to be able to build all the PETSc docs together in a portable way.
> 
>> On Mar 15, 2021, at 10:56 AM, Patrick Sanan <patrick.sanan at gmail.com <mailto:patrick.sanan at gmail.com>> wrote:
>> 
>> 
>> 
>>> Am 15.03.2021 um 16:26 schrieb Satish Balay <balay at mcs.anl.gov <mailto:balay at mcs.anl.gov>>:
>>> 
>>> On Mon, 15 Mar 2021, Lisandro Dalcin wrote:
>>> 
>>>> On Mon, 15 Mar 2021 at 07:06, Satish Balay <balay at mcs.anl.gov <mailto:balay at mcs.anl.gov>> wrote:
>>>> 
>>>>> Lisandro,
>>>>> 
>>>>> For the upcoming release its best to update lib/petsc/bin/maint/builddist
>>>>> to also build petsc4py tarball in sync with petsc tarball.
>>>>> 
>>>>> What is the current process to generate petsc4py tarball?
>>>>> 
>>>>> 
>>>>> 
>>>>> BTW: I stumbled into a couple of issues with building petsc4py docs
>>>>> 
>>>>> 1. the docs build process requires petsc library to be built?
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>>>>>>> 
>>>>> python setup.py build_src
>>>>> running build_src
>>>>> cythonizing 'petsc4py.PETSc.pyx' -> 'petsc4py.PETSc.c'
>>>>> cythonizing 'libpetsc4py/libpetsc4py.pyx' -> 'libpetsc4py/libpetsc4py.c'
>>>>> mkdir -p docs/apiref
>>>>> env CFLAGS=-O0 python2 setup.py -q build --build-lib build/lib.py2
>>>>> /usr/bin/ld: cannot find -lpetsc
>>>>> collect2: error: ld returned 1 exit status
>>>>> error: command 'gcc' failed with exit status 1
>>>>> make[2]: *** [makefile:110: epydoc-html] Error 1
>>>>> gmake[1]: [makefile:422: sphinx-docs-all] Error 2 (ignored)
>>>>> <<<
>>>>> 
>>>>> 2. Any particular reason it needs python2? I see it requires
>>>>> docutils,epydoc - but I see python3 is able to install them.
>>>>> 
>>>>> 
>>>> It requires petsc4py to be installed in Python 2, such that epydoc can
>>>> build the API reference.
>>> 
>>> But epydoc can be installed with python3. So this is more of petsc4py code that uses epydoc - than epydoc code?
>>> 
>>>> I have not found a nice replacement for epydoc-generated documentation.
>>>> If Python 2 is an annoyance, then just remove, comment-out anything related
>>>> to epydoc in "makefile"
>>> 
>>> Well I can work around it. [by installing both python2 and python3 versions of epydoc via pip]
>>> 
>>> However its not clear to me where the python2 requirement is coming
>>> from - and if things can be unified using python3
>>> 
>>> A related issue: I think Patrick is working on migrating some of the
>>> petsc4py docs to sphynx. And curently sphynx is installed in python3
>>> virt-env [so now we need it for both python3 for petsc side, and
>>> python2 for petsc4py side]
>> 
>> I haven't dug into the petsc4py docs yet, but I was very much hoping that we could at least make all the docs build at once and be deployed to the same URL, even if for now they are building with an assortment of tools. (We could include the petsc4py docs, for now, as we're doing with the HTML man pages and sources from c2html, specifying them as "extra html" with Sphinx) . As Satish says, it'd be nice to do this with the same Python environment we use to build the Sphinx docs - I would naively hope/assume that since epydoc exists for python3 and petsc4py works with python3, it would be possible.
>> 
>>> 
>>>> and run "make sdist".
>>> 
>>> Ah - ok. Will check this to see if I can generate the tarball in sync
>>> with petsc tarball. What source files need updating for
>>> release/version info?
>>> 
>>>> But this way the tarball will miss the API reference.
>>> 
>>>> PS: All this could be converted to a script that installs petsc4py and
>>>> epydoc in a Python 2 virtual environment, and next users another venv for
>>>> the Python 3 Sphinx stuff.
>>> 
>>> One issue: Fedora-33 does not have 'python2-pip' anymore. I was able
>>> to manually install it.  However this might become an issue for others
>>> who want to build docs [even if we automate the docs build to
>>> install/use pyton2-venv]
>>> 
>>> Satish
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20210315/414fb86b/attachment-0001.html>