[petsc-dev] Integrating PFLOTRAN, PETSC & SAMRAI

[Bobby Philip]

On Jun 6, 2011, at 3:57 PM, Boyce Griffith wrote:

> Does anyone actually use the PETSc wrappers that are provided by SAMRAI? 
>  (Even you, Bobby? :-) )
> 

I do - only I maintain my own version which is consistent with the release and dev versions 
of SAMRAI. However, each new update of PETSc or SAMRAI invariably does introduce
some problems


> My impression is that none of the LLNL SAMRAI folks use the SAMRAI-PETSc 
> interface, nor do any of the LLNL SAMRAI-based codes, which means that 
> it rapidly gets out of date and/or buggy.
> 

I agree.

> If SAMRAI support is something that could get rolled into petsc-dev, I'd 
> be happy to provide some code.
> 

The code (updated and working) already exists. It's ensuring that it continues to
work that is the problem. I've basically been maintaining the updated interface
with the help of the PETSc folks. 

> -- Boyce
> 
> On 6/6/11 3:47 PM, Barry Smith wrote:
>> 
>> On Jun 6, 2011, at 2:40 PM, Bobby Philip wrote:
>> 
>>> What usually breaks for me is VecNorm caching, arguments for functions,
>>> and introduction of new functions. Once in a while the PETScObject structure
>>> changes and that causes trouble too. But this is my usual list.
>>> 
>>> Any time petsc-dev changes, I end up having to build:
>>> 
>>> 1. SAMRAI
>>> 2. SAMRUtils
>>> 3. SAMRSolvers
>>> 4. PFLOTRAN
>>> 
>>> with stops along the way to fix anything that broke mainly in 1 and 4. Also, SAMRAI
>>> continues to only want to support version 2.3.3 of PETSc so I end up keeping tabs
>>> on what needs to be changed with each new SAMRAI version. A set of automated
>>> tests I hope might be the right catalyst to move them
>> 
>>   Bobby,
>> 
>>      Who is them? Frankly this is absurd, that's like saying they are not going to use a computer after the Sparc 2. If anyone is "using"/developing SAMRAI with 2.3.3 that is not real work, it is merely pretending. Whose butt should we kick?
>> 
>> 
>>    Barry
>> 
>> 
>> 
>>> to the present generation(s) of PETSc.
>>> 
>>> Bobby
>>> 
>>> On Jun 4, 2011, at 9:09 AM, Matthew Knepley wrote:
>>> 
>>>> On Sat, Jun 4, 2011 at 8:03 AM, Bobby Philip<philipb at ornl.gov>  wrote:
>>>> 
>>>>> Barry:
>>>>> 
>>>>> I heartily agree with all that you have said. Thanks for your suggestions.
>>>>> I do believe what you have suggested is the best both in the shorter and
>>>>> longer term. As a clarification on freezing the petsc-dev version, my
>>>>> suggestion was to freeze
>>>>> the version for a few months at a time as it reduces the burden on me on
>>>>> fixing frequent breakages.
>>>>> 
>>>> 
>>>> We don't have to bog down pflotran-dev with particulars, but can you
>>>> characterize what breaks
>>>> most often for you, or what you find yourself doing most often for SAMRAI?
>>>> 
>>>> Thanks,
>>>> 
>>>>    Matt
>>>> 
>>>> 
>>>>> Regards,
>>>>> Bobby
>>>>> 
>>>>> 
>>>>>> 
>>>>>> Peter,
>>>>>> 
>>>>>>  In the mid to longer term we need to make sure that SAMRAI has more
>>>>> resources than a one man operation, so this is something that needs coverage
>>>>> in the next round of SciDAC.
>>>>>> 
>>>>>>  When dealing with tracking package changes I've found the best way is
>>>>> to track them continuously so each new issue (due to a change) gets fixed
>>>>> immediately rather than waiting until a bunch of stuff is broken before
>>>>> fixing it.  This way determining what change broke what, is much easier.
>>>>> This means we should set up nightly tests of PETSc+SAMRAI+PFLOTRAN and deal
>>>>> with each issue the day after it breaks instead of weeks later. Probably
>>>>> Richard could set up such a test and have it send email to say Richard,
>>>>> Bobby and I each time a breakage occurs.
>>>>>> 
>>>>>> 
>>>>>> Barry
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Jun 3, 2011, at 5:49 PM, Lichtner, Peter C wrote:
>>>>>> 
>>>>>>> Folks: we have a problem trying to integrate PFLOTRAN with PETSc and
>>>>> SAMRAI. The basic problem is that when PETSc is updated it often breaks
>>>>> SAMRAI in nontrivial ways. For PFLOTRAN itself the changes needed are
>>>>> usually minor to bring PFLOTRAN back in sync with PETSc. But this is not the
>>>>> case for SAMRAI. Bobby Philip, the lead on SAMRAI, does not have the
>>>>> resources to develop the PFLOTRAN implementation of SAMRAI and keep it up to
>>>>> date with changes in PETSc. The question is how to resolve this difficulty.
>>>>> We have expended a considerable effort in adding SAMRAI to PFLOTRAN and
>>>>> several directions of PFLOTRAN development rely on SAMRAI including
>>>>> applications to CO2 sequestration, the Hanford uranium plume, and a hybrid
>>>>> model using the lattice Boltzmann method at the pore scale coupled to the
>>>>> Darcy continuum scale with SAMRAI providing the bridge between the two
>>>>> scales. Bobby has suggested freezing the PETSc developer version used in
>>>>> PFLOTRAN. However, this would potentially limit the development of PFLOTRAN.
>>>>> An alternative approach is to create a separate branch of PFLOTRAN for use
>>>>> with SAMRAI. However, this would limit our ability to develop a hybrid
>>>>> model, not to mention causing difficulties down the road when the two
>>>>> branches are finally merged. Any thoughts, suggestions, comments, etc.
>>>>> appreciated on how we can best resolve this issue.
>>>>>>> Thanks, ...Peter
>>>>>>> <><><><><><><><><><><><><><><><><><><><><>
>>>>>>> Peter C. Lichtner (lichtner at lanl.gov)
>>>>>>> LANL EES-16: MS D469
>>>>>>> (505) 667-3420 (o), 665-3285 (fax), 795-2881 (cell)
>>>>>>> Los Alamos National Laboratory
>>>>>>> SM-30 Bikini Atoll Road
>>>>>>> Los Alamos, NM 87545
>>>>>>> blockedblockedblockedhttp://www.ees.lanl.gov/pflotran/
>>>>>>> <><><><><><><><><><><><><><><><><><><><><>
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> What most experimenters take for granted before they begin their experiments
>>>> is infinitely more interesting than any results to which their experiments
>>>> lead.
>>>> -- Norbert Wiener
>>> 
>> 
>> 
>