[petsc-dev] [petsc-users] new book introducing PETSc for PDEs

[Jed Brown]

Barry Smith <bsmith at petsc.dev> writes:

>>> git subtree does require use of new commands every time you mess with it (say every three months) that we do not know and since each of 
>>> us will do this infrequently it is likely we will not remember them (I won't)  while my approach does not require remembering new commands. 
>>> My approach only requires branching, push, and pulling on the fork and updating EdsRepo.py which is things all the developers know about and 
>>> do regularly since we update other external packages.
>
>>> Circular dependencies (PETSc CI depends on Ed's p4pdes depends on PETSc) are significantly more labor-intensive and it would need to be done on each change, versus once per release cycle.
>
>     I think we are not communicating on exactly the  same wave length. 
>
>     I am not advocating we test on Ed's I am advocating we test on our fork of Ed's and how often our fork gets passed to Ed with MR is a choice all of us make together, now we could do it once at release time with all the fixes we made, every month or each time. Totally up to Ed how often he wants to get them, and he is free to ignore them for weeks (up to the next release if he likes) since our testing will continue because we use our fork. How often we pass them on to Ed is not related to how we maintain our CI.
>
>     There is no circular dependency on my approach with Ed's p4pdes.

The circular dependency is on our fork of p4pdes.

>     I vote to fix things in our fork or gittree thing continuously since it makes it easier to fix things rather than wait to the release when we try to find and fix everything and it also helps tell us if we introduced a real bug into PETSc and fix PETSc immediately instead of waiting up to 6 months, just like we now we test immediately with Petsc4py and we should do with SLEPc.  How often we give the updates to Ed is a completely different issue. 
>
>    So again back to my original statement ,it comes down to if the subtree or the fork approach is easier for all the PETSc developers who do not currently know gittree and would need to learn it with your approach. I don't know which is easier learning to use gittree which has its own gotcha's or using mine which we all know but may require an extra step (not involving Ed, just updating the p4pdes.py commit each time we change something in the fork.)
>
>   I think using --download-p4pdes on a couple of systems in the CI is enough, I don't think we need to put it all CI pipelines (I would like slepc in all pipelines).but we could put in all pipelines if we want.
>
>   For completeness I show the exact the work flow for my suggestion
>
>    pipelines --download-p4pdes and runs its tests
>    if it breaks the developer uses --download-p4pdes  on their system 
>      they fix the problem either by fixing PETSc or what is downloaded from the p4pde fork
>         if the fix is in the p4pdes fork they make a branch in the p4pdes fork, which they already have since they used --download-p4pdes and thus 
>                               have the fork on their system
>             they put the fix in new branch in the p4pdes fork and push it
>             they edit p4pdes.py and put a new commit in it pointing to their branch in the fork
>    run the pipeline again
>        if fails with p4pdes they do the above again
>        else the PETSc branch gets accepted and merged to master
>           depending on Ed's choice we make an MR for p4pdes depending on the agreed upon cycle. If Ed puts the fix into his master then we just update
>                our fork with his latest master with a simple merge of his.  This will then become the new one we test against. If Ed doesn't respond to the MR all is fine we 
>               just continue on our fork. If he puts other things in his branch but not our MR we just merge that into our fork and so are still testing with his latest master.

As you've enumerated here, this requires three MRs per change:

  1. in PETSc with the actual change and to point at a p4pdes commit
  2. in p4pdes (Ed's or our fork) to implement needed changes (note: this can't merge until #1 merges)
  3. in PETSc to point at the merge commit of p4pdes (after #2 merges)

With subtree, there is only one MR and it's in the PETSc repository.

Recall that we exchanged hundreds of emails to subtree in BuildSystem long ago, then quite a lot more to subtree in petsc4py recently, and now we're doing it again.

The arguments against subtreeing p4pdes are

  1. We want an extra hurdle so developers think twice before changing those interfaces
  2. We want to drive traffic to Ed's repo
  3. The repository is so big we don't want every `git clone gitlab:petsc/petsc` to include it

This repo is small so I'm most sympathetic to #2, but nobody has made these arguments yet.

>   This happens to be the exact same thing I do now with slepc and any other git based package that I use now (except for some I need to make my own fork of the external package because we don't have one always available eventually likely we will likely put more forks into gitlab/petsc so each person doesn't constantly need to make fresh forks of external packages)
>
>   If subtree requires fewer steps than this and has no new oddball git subtree commands then we should definitely use subtree, if it requires other steps involving subtree we need see those commands written in a workflow like I have written above and decide if it is still simpler or not.
>   
>   I am not rejecting subtree, we just need to explicitly see its complete work flow and hence the differences to decide, that is what I asking for.
>
>   Barry
>
>   It seems to me that with subtree we also need to maintain a fork of Ed's stuff or else that will have a circular dependency. But perhaps I do not understand it.

Such a fork would only be used once per release to submit PRs to Ed.