[petsc-dev] including PETSc4py (and Tao4py) inside PETSc repository

[Satish Balay]

The current infrastruture tests --download-petsc4py in petsc 'next'
and 'master' branches.

Wrt next - usually a change in a feature branch (when it gets merged
to next) triggers this breakage.  And usual thing to do is to fix it
in the feature branch (and merge to next again)

Wrt to petsc4py - this would mean updating petsc4py.py [in petsc] with
the petsc4py repo gitcommit [optionally also tarball ] that
corresponds to the fix. [one way to handle this on petsc4py repo is to
create a feature branch for this fix]

When this (petsc) feature branch is merged into master - the
corresponding feature branch on the petsc4py side can be merged into
petsc4py master [if petsc4py-master is to be in sync with
petsc-master].

[if there are multiple feature branches on the petsc side requiring
corresponding petsc4py changes - somehow that has to be handled]

Perhaps testing petsc4py via petsc infrastructure [using
--download-petsc4py] is not the appropriate approach?

Or we should do this only on master - not next?  [or just have a
single --download-petsc4py test - thats not mixed with other tests? -
so its breakage is not critical for other things]

Also we don't currently run any petsc4py tests. [we just test if it
compiles or not]

Satish

On Thu, 22 Oct 2015, Barry Smith wrote:

> 
>   Lisandro,
> 
>    It is tested nightly but there is apparently currently no email sent on failure. See the bottom of, for example, http://ftp.mcs.anl.gov/pub/petsc/nightlylogs/archive/2015/10/22/make_next_arch-linux-pkgs-opt_crank.log
> 
>    I think what should happen is the petsc4py log file should be published to the web (like the other log files) and a script look through it and decide to send email if it detects a problem. Like we currently have for the other log files. Someone has to dig into the nightly test infrastructure to figure out where the new logic needs to go. Currently Satish and Karl understand that infrastructure the best.
> 
>   Barry
> 
> 
> 
> 
> 
> > On Oct 22, 2015, at 6:44 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> > 
> > On 22 October 2015 at 23:20, Barry Smith <bsmith at mcs.anl.gov> wrote:
> >> 
> >>   Lisandro,
> >> 
> >>    You never responded to this. I assume it was because you did not like the idea? This is not an attempt to take control of petsc4py or to take credit away from petsc4py from you. Petsc4py is your package and you will always be the one who deserves the credit.
> >> 
> > 
> > Oh, sorry! I missed this email.
> > 
> > 
> >>    The problem is that mutual efficient development of very intertwined packages is difficult if they are in different respositories. For example petsc4py has been out of date with petsc master for many weeks now because it is too much effort to update the petsc4py repository each time we make some change in PETSc. Thus it becomes a burden on you to go in every once in a while and fix up petsc4py. With one repository changes would happen in both PETSc source and petsc4py source at the same time and tests would detect problems immediately.
> >> 
> >>   Thoughts?
> >> 
> > 
> > I general terms, I prefer to keep things separate as much a possible.
> > IMHO, a much important step would be to incorporate petsc4py to the
> > nightly tests, clearly separating the petsc and petsc4py tests runs. I
> > would love to get an email every time petsc4py nightly breaks, this
> > way I would be able to push fixes within a day or two, say.
> > Maintaining petsc4py requires knowledge of PETSc+Python+Cython, so
> > ultimately I'm the one in best position to do such work, if you guys
> > start adding commits to petsc4py withing the main petsc repo, I'm
> > likely going to miss these changes. For example, today Michael Lange
> > made some changes, I got an email from Bitbucket, then did a review
> > and spotted a few issues.
> > 
> > My hesitation to incorporate petsc4py within petsc repo is, in
> > Python's [and linear algebra :-)] Zen : "Sparse is better than dense".
> > If we continue adding stuff to core PETSc, at same point we will endup
> > like Trilinos ( DON'T MISS the sencond entry in the FAQ:
> > https://trilinos.org/download/public-git-repository/).
> > 
> > All that being said, I don't have any problem at all adding petsc4py
> > to the main PETSc repository. But I still think this is not a good
> > idea, it will not solve all the problems. Nightly tests with automatic
> > notifications to me about breakages is perhaps is all what we need to
> > keep petsc4py in close sync with petsc/master. What about trying this
> > first to see how it goes?
> > 
> > 
> > PS: Please understand my position has nothing to do with credits about
> > the project or ego-related crap. Actually, your proposal would mean
> > less responsibility and maintenance work for me. But I don't think it
> > is a move in the right direction.
> > 
> > 
> > -- 
> > Lisandro Dalcin
> > ============
> > Research Scientist
> > Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
> > Numerical Porous Media Center (NumPor)
> > King Abdullah University of Science and Technology (KAUST)
> > http://numpor.kaust.edu.sa/
> > 
> > 4700 King Abdullah University of Science and Technology
> > al-Khawarizmi Bldg (Bldg 1), Office # 4332
> > Thuwal 23955-6900, Kingdom of Saudi Arabia
> > http://www.kaust.edu.sa
> > 
> > Office Phone: +966 12 808-0459
> 
>