[petsc-dev] Is master broken?

[Karl Rupp]

Hi,

to clarify, these commits are already in master (either directly, or in 
a condensed way in the course of different pull requests, including 
additional fixes of memory leaks and output files):

 > 5e3edcb81f added check in Inode to skip GPUs for not Factor
 > 346e0b6564 changed reduction logic a little and cleaned up format
 > 3a282b2d9d added guards for empty process solves in CUDA
 > af6bc10070 same bug fix
 > 0365c3a97f try a fix
 > 9c37fbf3e7 bug fix
 > 72f2ad35b3 fixed bug with CUDA transpose mat-vec
 > ab62ce3476 remove vecset as per Barry
 > c26191aaa4 use non-collective VecSet
 > 8bcb2d50b7 fixed MPI lock from call to collective method
 > 3c46958f6d fix bug with empty processor
 > 54cfeb1831 added missing settypes

this commit is not in master because I couldn't get the example to run:
 > fd2e5db618 added cude test to ex56

these commits are ViennaCL-specific and don't need further consideration 
(the wrapper will take care of it):

 > 12042c4bfa removing ViennaCL fix to GAMG
 > 9508265e8e adding support for MatTranspose
 > e5a6000419 adding fix for ViennaCL in MG


The *only* commit with functionality that is not in master is:

 > 57224a7035 fixed up pinning CUDA to CPUs

for which PR #1954 holds the discussion of why this commit has 
problematic parts.

I've opened a pull request for Barry's fixes to the builds on Summit in 
PR #1963. If there are no surprises, this PR will be in master by the 
end of the day.

**Overall**: Please start freshly off master (and merge PR 1963 if 
needed). Then, have a look at the pinning-to-CPUs-commit and decide 
whether it needs to be reworked as for what is discussed in PR #1954. Do 
not try to rebase
  mark/gamg-fix-viennacl-rebased
or any of its offsprings any further, as this will only cause headaches 
and create conflicts with code that has already been fixed.

Best regards,
Karli



On 8/12/19 5:23 PM, Balay, Satish wrote:
> I don't really understand the workflow here [with merged branches (to
> master) and cherry-picking other stuff]
> 
> I've attempted to rebase this branch against latest master. The result is at:
> 
> mark/gamg-fix-viennacl-rebased-v3
> 
> There were too many merge conflicts that I had to resolve. Its
> possible I made mistakes here.
> 
> So if this branch is what you need - its best to check each commit
> here and verify the changes - before using them.
> 
> As Karl suggests - its probably best to pick commits that you need and
> fix them [if they have bad code - from merge conflict resolution]
> 
> [and some of them can be collapsed]
> 
> Satish
> 
> ------
> 
> balay at sb /home/balay/petsc (mark/gamg-fix-viennacl-rebased-v3=)
> $ git log --oneline master..
> dee2b8b21b (HEAD -> mark/gamg-fix-viennacl-rebased-v3, origin/mark/gamg-fix-viennacl-rebased-v3, mark/gamg-fix-viennacl-rebased-v2) protected pinnedtocpu
> 03d489bde4 1) When detecting version info handle blanks introducted by preprocessor, error if needed version cannot be detected
> 056432fa93 Use outputPreprocess instead of preprocess since it prints source to log
> 94885b4f80 add back code missing from rebaseing over latest master?
> 2a748c2fa1 fixed compile errors
> 5e3edcb81f added check in Inode to skip GPUs for not Factor
> fc14e5b821 removed comment
> 57224a7035 fixed up pinning CUDA to CPUs
> 346e0b6564 changed reduction logic a little and cleaned up format
> 3a282b2d9d added guards for empty process solves in CUDA
> af6bc10070 same bug fix
> 0365c3a97f try a fix
> 9c37fbf3e7 bug fix
> fd2e5db618 added cude test to ex56
> 72f2ad35b3 fixed bug with CUDA transpose mat-vec
> ab62ce3476 remove vecset as per Barry
> c26191aaa4 use non-collective VecSet
> 12042c4bfa removing ViennaCL fix to GAMG
> 3c46958f6d fix bug with empty processor
> 8bcb2d50b7 fixed MPI lock from call to collective method
> 54cfeb1831 added missing settypes
> 9508265e8e adding support for MatTranspose
> e5a6000419 adding fix for ViennaCL in MG
> 
> 
> On Mon, 12 Aug 2019, Karl Rupp via petsc-dev wrote:
> 
>> Hi Mark,
>>
>> most of the CUDA-related fixes from your PR are now in master. Thank you!
>>
>> The pinning of GPU-matrices to CPUs is not in master because it had several
>> issues:
>>
>> https://bitbucket.org/petsc/petsc/pull-requests/1954/cuda-fixes-to-pinning-onto-cpu/diff
>>
>> The ViennaCL-related changes in mark/gamg-fix-viennacl-rebased can be safely
>> discarded as the new GPU wrapper will come in place over the next days. ex56
>> has not been pulled over as it's not running properly on GPUs yet (the pinning
>> in your branch effectively turned GPU matrices into normal PETSc matrices,
>> effectively running (almost) everything on the CPU again)
>>
>> So at this point I recommend to start a new branch off master and manually
>> transfer over any bits from the pinning that you want to keep.
>>
>> Best regards,
>> Karli
>>
>>
>> On 8/3/19 8:47 PM, Mark Adams wrote:
>>> Karl,
>>> Did you want me to do anything at this point? (on vacation this week) I will
>>> verify that master is all fixed if you get all my stuff integrated when I
>>> get back to work in a week.
>>> Thanks,
>>> Mark
>>>
>>> On Sat, Aug 3, 2019 at 10:50 AM Karl Rupp <rupp at iue.tuwien.ac.at
>>> <mailto:rupp at iue.tuwien.ac.at>> wrote:
>>>
>>>      If you ignore the initial ViennaCL-related commits and check against
>>>      current master (that just received cherry-picked updates from your PR),
>>>      then there are really only a few commits left that are not yet
>>>      integrated.
>>>
>>>      (I'll extract two more PRs on Monday, so master will soon have your
>>>      fixes in.)
>>>
>>>      Best regards,
>>>      Karli
>>>
>>>
>>>      On 8/3/19 5:21 AM, Balay, Satish wrote:
>>>       > I've attempted to rebase this branch over latest master - and pushed
>>>       > my changes to branch mark/gamg-fix-viennacl-rebased-v2
>>>       >
>>>       > You might want to check each of your commits in this branch to see if
>>>       > they are ok. I had to add one extra commit - to make it match 'merge
>>>       > of mark/gamg-fix-viennacl-rebased and master'.
>>>       >
>>>       > This branch has 21 commits. I think its best if you can collapse them
>>>       > into reasonable chunks of changes. [presumably a single commit
>>>      for all
>>>       > the changes is not the correct thing here. But the current set of 21
>>>       > commits are all over the place]
>>>       >
>>>       > If you are able to migrate to this branch - its best to delete
>>>      the old
>>>       > one [i.e origin/mark/gamg-fix-viennacl-rebased]
>>>       >
>>>       > Satish
>>>       >
>>>       > On Fri, 2 Aug 2019, Mark Adams via petsc-dev wrote:
>>>       >
>>>       >> I have been cherry-picking, etc, branch
>>>      mark/gamg-fix-viennacl-rebased and
>>>       >> it is very messed up. Can someone please update this branch when
>>>      all the
>>>       >> fixes are settled down? eg, I am seeing dozens of modified files
>>>      that I
>>>       >> don't know anything about and I certainly don't want to put in a
>>>      PR for
>>>       >> them.
>>>       >>
>>>       >> I also seem to lose my pinToCPU method for cuda matrices. I don't
>>>       >> understand how that conflicted with anyone else but it did.
>>>       >>
>>>       >> Thanks,
>>>       >> Mark
>>>       >>
>>>       >
>>>
>>
>>
>