[petsc-dev] Integrating PFLOTRAN, PETSC & SAMRAI

[Boyce Griffith]

I maintain similar SAMRAI-PETSc stuff, although I'm only synched up with 
PETSc 3.1, not petsc-dev.  VecNorm caching caused me endless grief until 
I started making lots of calls to PetscObjectStateIncrease(), both in 
the Vec routines and in the Mat/nonlinear function wrappers.

If you are "building your own" Vec class, is there a better/safer way to 
ensure that cached Vec data is properly invalidated?

-- Boyce

On 6/6/11 3:40 PM, Bobby Philip wrote:
> What usually breaks for me is VecNorm caching, arguments for functions,
> and introduction of new functions. Once in a while the PETScObject structure
> changes and that causes trouble too. But this is my usual list.
>
> Any time petsc-dev changes, I end up having to build:
>
> 1. SAMRAI
> 2. SAMRUtils
> 3. SAMRSolvers
> 4. PFLOTRAN
>
> with stops along the way to fix anything that broke mainly in 1 and 4. Also, SAMRAI
> continues to only want to support version 2.3.3 of PETSc so I end up keeping tabs
> on what needs to be changed with each new SAMRAI version. A set of automated
> tests I hope might be the right catalyst to move them to the present generation(s) of PETSc.
>
> Bobby
>
> On Jun 4, 2011, at 9:09 AM, Matthew Knepley wrote:
>
>> On Sat, Jun 4, 2011 at 8:03 AM, Bobby Philip<philipb at ornl.gov>  wrote:
>>
>>> Barry:
>>>
>>> I heartily agree with all that you have said. Thanks for your suggestions.
>>> I do believe what you have suggested is the best both in the shorter and
>>> longer term. As a clarification on freezing the petsc-dev version, my
>>> suggestion was to freeze
>>> the version for a few months at a time as it reduces the burden on me on
>>> fixing frequent breakages.
>>>
>>
>> We don't have to bog down pflotran-dev with particulars, but can you
>> characterize what breaks
>> most often for you, or what you find yourself doing most often for SAMRAI?
>>
>>   Thanks,
>>
>>      Matt
>>
>>
>>> Regards,
>>> Bobby
>>>
>>>
>>>>
>>>> Peter,
>>>>
>>>>    In the mid to longer term we need to make sure that SAMRAI has more
>>> resources than a one man operation, so this is something that needs coverage
>>> in the next round of SciDAC.
>>>>
>>>>    When dealing with tracking package changes I've found the best way is
>>> to track them continuously so each new issue (due to a change) gets fixed
>>> immediately rather than waiting until a bunch of stuff is broken before
>>> fixing it.  This way determining what change broke what, is much easier.
>>> This means we should set up nightly tests of PETSc+SAMRAI+PFLOTRAN and deal
>>> with each issue the day after it breaks instead of weeks later. Probably
>>> Richard could set up such a test and have it send email to say Richard,
>>> Bobby and I each time a breakage occurs.
>>>>
>>>>
>>>>   Barry
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Jun 3, 2011, at 5:49 PM, Lichtner, Peter C wrote:
>>>>
>>>>> Folks: we have a problem trying to integrate PFLOTRAN with PETSc and
>>> SAMRAI. The basic problem is that when PETSc is updated it often breaks
>>> SAMRAI in nontrivial ways. For PFLOTRAN itself the changes needed are
>>> usually minor to bring PFLOTRAN back in sync with PETSc. But this is not the
>>> case for SAMRAI. Bobby Philip, the lead on SAMRAI, does not have the
>>> resources to develop the PFLOTRAN implementation of SAMRAI and keep it up to
>>> date with changes in PETSc. The question is how to resolve this difficulty.
>>> We have expended a considerable effort in adding SAMRAI to PFLOTRAN and
>>> several directions of PFLOTRAN development rely on SAMRAI including
>>> applications to CO2 sequestration, the Hanford uranium plume, and a hybrid
>>> model using the lattice Boltzmann method at the pore scale coupled to the
>>> Darcy continuum scale with SAMRAI providing the bridge between the two
>>> scales. Bobby has suggested freezing the PETSc developer version used in
>>> PFLOTRAN. However, this would potentially limit the development of PFLOTRAN.
>>> An alternative approach is to create a separate branch of PFLOTRAN for use
>>> with SAMRAI. However, this would limit our ability to develop a hybrid
>>> model, not to mention causing difficulties down the road when the two
>>> branches are finally merged. Any thoughts, suggestions, comments, etc.
>>> appreciated on how we can best resolve this issue.
>>>>> Thanks, ...Peter
>>>>> <><><><><><><><><><><><><><><><><><><><><>
>>>>> Peter C. Lichtner (lichtner at lanl.gov)
>>>>> LANL EES-16: MS D469
>>>>> (505) 667-3420 (o), 665-3285 (fax), 795-2881 (cell)
>>>>> Los Alamos National Laboratory
>>>>> SM-30 Bikini Atoll Road
>>>>> Los Alamos, NM 87545
>>>>> blockedblockedhttp://www.ees.lanl.gov/pflotran/
>>>>> <><><><><><><><><><><><><><><><><><><><><>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their experiments
>> is infinitely more interesting than any results to which their experiments
>> lead.
>> -- Norbert Wiener
>
>