[petsc-users] DMNetworkGetEdgeRange() in parallel

Matthew Knepley knepley at gmail.com
Tue Feb 24 18:49:32 CST 2015 

On Tue, Feb 24, 2015 at 6:42 PM, Miguel Angel Salazar de Troya <
salazardetroya at gmail.com> wrote:

> I implemented the code as agreed, but I don't get the results I expected.
> When I create the vector with DMCreateGlobalVector(), I obtain a vector
> with a layout similar to the original DMNetwork, instead of the cloned
> network with the new PetscSection. The code is as follows:
>
> DMClone(dm, &dmEdge);
>
> PetscSectionCreate(PETSC_COMM_WORLD, &s);
> PetscSectionSetNumFields(s, 1);
> PetscSectionSetFieldComponents(s, 0, 1);
>
> // Now to set the chart, I pick the edge range
>
> DMNetworkGetEdgeRange(dmEdge, & eStart, & eEnd)
>
> PetscSectionSetChart(s, eStart, eEnd);
>
> for(PetscInt e = eStart; c < eEnd; ++e) {
>      PetscSectionSetDof(s, e, 1);
>      PetscSectionSetFieldDof(s, e, 0, 1);
> }
> PetscSectionSetUp(s);
>
> DMSetDefaultSection(dmEdge s);
> DMCreateGlobalVector(dmEdge, &globalVec);
>
> When I get into DMCreateGlobalVector(dmEdge, &globalVec) in the debugger,
> in the function DMCreateSubDM_Section_Private() I call PetscSectionView()
> on the section
>

I have no idea why you would be in DMCreateSubDM().

Just view globalVec. If the code is as above, it will give you a vector
with that layout. If not
it should be trivial to make a small code and send it. I do this everywhere
is PETSc, so the
basic mechanism certainly works.

  Thanks,

     Matt


> obtained by DMGetDefaultGlobalSection(dm, &sectionGlobal), and I obtain a
> PetscSection nothing like the one I see when I call PetscSectionView() on
> the PetscSection I created above. Does this have anything to do? I tried
> to compare this strange PetscSection with the one from the original
> DMNetwork, I call DMGetDefaultGlobalSection(dm, &sectionGlobal) before
> the first line of the snippet above and I get this error message.
>
> 0]PETSC ERROR: --------------------- Error Message
> --------------------------------------------------------------
> [0]PETSC ERROR: Object is in wrong state
> [0]PETSC ERROR: DM must have a default PetscSection in order to create a
> global PetscSection
>
> Thanks in advance
> Miguel
>
>
> On Mon, Feb 23, 2015 at 3:24 PM, Matthew Knepley <knepley at gmail.com>
> wrote:
>
>> On Mon, Feb 23, 2015 at 2:15 PM, Miguel Angel Salazar de Troya <
>> salazardetroya at gmail.com> wrote:
>>
>>> Thanks a lot, the partition should be done before setting up the
>>> section, right?
>>>
>>
>> The partition will be automatic. All you have to do is make the local
>> section. The DM is already partitioned,
>> and the Section will inherit that.
>>
>>   Matt
>>
>>
>>> Miguel
>>>
>>> On Mon, Feb 23, 2015 at 2:05 PM, Matthew Knepley <knepley at gmail.com>
>>> wrote:
>>>
>>>> On Mon, Feb 23, 2015 at 1:40 PM, Miguel Angel Salazar de Troya <
>>>> salazardetroya at gmail.com> wrote:
>>>>
>>>>> Wouldn't including the edge variables in the global vector make the
>>>>> code slower? I'm using the global vector in a TS, using one of the explicit
>>>>> RK schemes. The edge variables would not be updated in the RHSFunction
>>>>> evaluation. I only change the edge variables in the TSUpdate. If the global
>>>>> vector had the edge variables, it would be a much larger vector, and all
>>>>> the vector operations performed by the TS would be slower. Although the
>>>>> vector F returned by the RHSFunction would be zero in the edge variable
>>>>> components. I guess that being the vector sparse that would not be a
>>>>> problem.
>>>>>
>>>>> I think I'm more interested in the PetscSection approach because it
>>>>> might require less modifications in my code. However, I don't know how I
>>>>> could do this. Maybe something like this?
>>>>>
>>>>> PetscSectionCreate(PETSC_COMM_WORLD, &s);
>>>>> PetscSectionSetNumFields(s, 1);
>>>>> PetscSectionSetFieldComponents(s, 0, 1);
>>>>>
>>>>> // Now to set the chart, I pick the edge range
>>>>>
>>>>> DMNetworkGetEdgeRange(dm, & eStart, & eEnd
>>>>>
>>>>> PetscSectionSetChart(s, eStart, eEnd);
>>>>>
>>>>> for(PetscInt e = eStart; c < eEnd; ++e) {
>>>>>      PetscSectionSetDof(s, e, 1);
>>>>>      PetscSectionSetFieldDof(s, e, 1, 1);
>>>>>
>>>>
>>>> It should be PetscSectionSetFieldDof(s, e, 0, 1);
>>>>
>>>>
>>>>> }
>>>>> PetscSectionSetUp(s);
>>>>>
>>>>> Now in the manual I see this:
>>>>>
>>>>
>>>> First you want to do:
>>>>
>>>>   DMClone(dm, &dmEdge);
>>>>
>>>> and then use dmEdge below.
>>>>
>>>>
>>>>> DMSetDefaultSection(dm, s);
>>>>> DMGetLocalVector(dm, &localVec);
>>>>> DMGetGlobalVector(dm, &globalVec);
>>>>>
>>>>> Setting up the default section in the DM would interfere with the
>>>>> section already set up with the variables in the vertices?
>>>>>
>>>>
>>>> Yep, thats why you would use a clone.
>>>>
>>>>   Thanks,
>>>>
>>>>      Matt
>>>>
>>>>
>>>>> Thanks a lot for your responses.
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Feb 23, 2015 at 11:37 AM, Matthew Knepley <knepley at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> On Mon, Feb 23, 2015 at 9:27 AM, Miguel Angel Salazar de Troya <
>>>>>> salazardetroya at gmail.com> wrote:
>>>>>>
>>>>>>> I'm iterating through local edges given in DMNetworkGetEdgeRange().
>>>>>>> For each edge, I extract or modify its corresponding value in a global
>>>>>>> petsc vector. Therefore that vector must have as many components as edges
>>>>>>> there are in the network. To extract the value in the vector, I use
>>>>>>> VecGetArray() and a variable counter that is incremented in each iteration.
>>>>>>> The array that I obtain in VecGetArray() has to be the same size
>>>>>>> than the edge range. That variable counter starts as 0, so if the array
>>>>>>> that I obtained in VecGetArray() is x_array, x_array[0] must be the
>>>>>>> component in the global vector that corresponds with the start edge given
>>>>>>> in DMNetworkGetEdgeRange()
>>>>>>>
>>>>>>> I need that global petsc vector because I will use it in other
>>>>>>> operations, it's not just data. Sorry for the confusion. Thanks in advance.
>>>>>>>
>>>>>>
>>>>>> This sounds like an assembly operation. The usual paradigm is to
>>>>>> compute in the local space, and then communicate to get to the global
>>>>>> space. So you would make a PetscSection that had 1 (or some) unknowns on
>>>>>> each cell (edge) and then you can use DMCreateGlobal/LocalVector() and
>>>>>> DMLocalToGlobal() to do this.
>>>>>>
>>>>>> Does that make sense?
>>>>>>
>>>>>>   Thanks,
>>>>>>
>>>>>>      Matt
>>>>>>
>>>>>>
>>>>>>> Miguel
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 23, 2015 at 9:09 AM, Matthew Knepley <knepley at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> On Mon, Feb 23, 2015 at 8:42 AM, Miguel Angel Salazar de Troya <
>>>>>>>> salazardetroya at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Thanks, that will help me. Now what I would like to have is the
>>>>>>>>> following: if I have two processors and ten edges, the partitioning results
>>>>>>>>> in the first processor having the edges 0-4 and the second processor, the
>>>>>>>>> edges 5-9. I also have a global vector with as many components as edges,
>>>>>>>>> 10. How can I partition it so the first processor also has the 0-4
>>>>>>>>> components and the second, the 5-9 components of the vector?
>>>>>>>>>
>>>>>>>> I think it would help to know what you want to accomplish. This is
>>>>>>>> how you are proposing to do it.'
>>>>>>>>
>>>>>>>> If you just want to put data on edges, DMNetwork has a facility for
>>>>>>>> that already.
>>>>>>>>
>>>>>>>>   Thanks,
>>>>>>>>
>>>>>>>>      Matt
>>>>>>>>
>>>>>>>>
>>>>>>>>> Miguel
>>>>>>>>> On Feb 23, 2015 8:08 AM, "Abhyankar, Shrirang G." <
>>>>>>>>> abhyshr at mcs.anl.gov> wrote:
>>>>>>>>>
>>>>>>>>>>  Miguel,
>>>>>>>>>>    One possible way is to store the global numbering of any
>>>>>>>>>> edge/vertex in the "component" attached to it. Once the mesh gets
>>>>>>>>>> partitioned, the components are also distributed so you can easily retrieve
>>>>>>>>>> the global number of any edge/vertex by accessing its component. This is
>>>>>>>>>> what is done in the DMNetwork example pf.c although the global numbering is
>>>>>>>>>> not used for anything.
>>>>>>>>>>
>>>>>>>>>>  Shri
>>>>>>>>>>  From: Matthew Knepley <knepley at gmail.com>
>>>>>>>>>> Date: Mon, 23 Feb 2015 07:54:34 -0600
>>>>>>>>>> To: Miguel Angel Salazar de Troya <salazardetroya at gmail.com>
>>>>>>>>>> Cc: "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
>>>>>>>>>> Subject: Re: [petsc-users] DMNetworkGetEdgeRange() in parallel
>>>>>>>>>>
>>>>>>>>>>   On Sun, Feb 22, 2015 at 3:59 PM, Miguel Angel Salazar de Troya
>>>>>>>>>> <salazardetroya at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks. Once I obtain that Index Set with the routine DMPlexCreateCellNumbering()
>>>>>>>>>>> (I assume that the edges in DMNetwork correspond to cells in DMPlex) can I
>>>>>>>>>>> use it to partition a vector with as many components as edges I have in my
>>>>>>>>>>> network?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  I do not completely understand the question.
>>>>>>>>>>
>>>>>>>>>>  If you want a partition of the edges, you can use
>>>>>>>>>> DMPlexCreatePartition() and its friend DMPlexDistribute(). What
>>>>>>>>>> are you trying to do?
>>>>>>>>>>
>>>>>>>>>>     Matt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>  Thanks
>>>>>>>>>>> Miguel
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Feb 22, 2015 at 12:15 PM, Matthew Knepley <
>>>>>>>>>>> knepley at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>  On Sun, Feb 22, 2015 at 11:01 AM, Miguel Angel Salazar de
>>>>>>>>>>>> Troya <salazardetroya at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>>  I noticed that the routine DMNetworkGetEdgeRange() returns
>>>>>>>>>>>>> the local indices for the edge range. Is there any way to obtain the global
>>>>>>>>>>>>> indices? So if my network has 10 edges, the processor 1 has the 0-4 edges
>>>>>>>>>>>>> and the processor 2, the 5-9 edges, how can I obtain this information?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  One of the points of DMPlex is we do not require a global
>>>>>>>>>>>> numbering. Everything is numbered
>>>>>>>>>>>> locally, and the PetscSF maps local numbers to local numbers in
>>>>>>>>>>>> order to determine ownership.
>>>>>>>>>>>>
>>>>>>>>>>>>  If you want to create a global numbering for some reason, you
>>>>>>>>>>>> can using DMPlexCreatePointNumbering().
>>>>>>>>>>>> There are also cell and vertex versions that we use for output,
>>>>>>>>>>>> so you could do it just for edges as well.
>>>>>>>>>>>>
>>>>>>>>>>>>    Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>>       Matt
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>  Thanks
>>>>>>>>>>>>>  Miguel
>>>>>>>>>>>>>
>>>>>>>>>>>>>  --
>>>>>>>>>>>>>  *Miguel Angel Salazar de Troya*
>>>>>>>>>>>>> Graduate Research Assistant
>>>>>>>>>>>>> Department of Mechanical Science and Engineering
>>>>>>>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>>>>>>>> (217) 550-2360
>>>>>>>>>>>>> salaza11 at illinois.edu
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>  --
>>>>>>>>>>>> What most experimenters take for granted before they begin
>>>>>>>>>>>> their experiments is infinitely more interesting than any results to which
>>>>>>>>>>>> their experiments lead.
>>>>>>>>>>>> -- Norbert Wiener
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>  --
>>>>>>>>>>>  *Miguel Angel Salazar de Troya*
>>>>>>>>>>> Graduate Research Assistant
>>>>>>>>>>> Department of Mechanical Science and Engineering
>>>>>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>>>>>> (217) 550-2360
>>>>>>>>>>> salaza11 at illinois.edu
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  --
>>>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>>>>> experiments lead.
>>>>>>>>>> -- Norbert Wiener
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>>> experiments lead.
>>>>>>>> -- Norbert Wiener
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> *Miguel Angel Salazar de Troya*
>>>>>>> Graduate Research Assistant
>>>>>>> Department of Mechanical Science and Engineering
>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>> (217) 550-2360
>>>>>>> salaza11 at illinois.edu
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> What most experimenters take for granted before they begin their
>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>> experiments lead.
>>>>>> -- Norbert Wiener
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Miguel Angel Salazar de Troya*
>>>>> Graduate Research Assistant
>>>>> Department of Mechanical Science and Engineering
>>>>> University of Illinois at Urbana-Champaign
>>>>> (217) 550-2360
>>>>> salaza11 at illinois.edu
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> What most experimenters take for granted before they begin their
>>>> experiments is infinitely more interesting than any results to which their
>>>> experiments lead.
>>>> -- Norbert Wiener
>>>>
>>>
>>>
>>>
>>> --
>>> *Miguel Angel Salazar de Troya*
>>> Graduate Research Assistant
>>> Department of Mechanical Science and Engineering
>>> University of Illinois at Urbana-Champaign
>>> (217) 550-2360
>>> salaza11 at illinois.edu
>>>
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>
>
>
> --
> *Miguel Angel Salazar de Troya*
> Graduate Research Assistant
> Department of Mechanical Science and Engineering
> University of Illinois at Urbana-Champaign
> (217) 550-2360
> salaza11 at illinois.edu
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150224/6db3724e/attachment-0001.html>

More information about the petsc-users mailing list