[petsc-users] DMNetworkGetEdgeRange() in parallel
Miguel Angel Salazar de Troya
salazardetroya at gmail.com
Wed Feb 25 11:31:28 CST 2015
Now it works, thanks a lot!
Miguel
On Wed, Feb 25, 2015 at 10:12 AM, Matthew Knepley <knepley at gmail.com> wrote:
> On Wed, Feb 25, 2015 at 9:59 AM, Abhyankar, Shrirang G. <
> abhyshr at mcs.anl.gov> wrote:
>
>> Miguel,
>> I'm a bit tied up today. I'll try to debug this issue tomorrow and get
>> back to you.
>>
>
> The problem is the way that DMNetwork is using the default section. When
> DMCreateGlobalVec() is called,
> it uses the default section for its included network->plex, but
> DMSetDefaultSection() sets the section associated
> with network. This is inconsistent.
>
> I am sending the code which I hacked to show the correct result by using
> the private header.
>
> I think that
>
> 1) The underlying Plex should be exposed by DMNetworkGetPlex()
>
> 2) The default section business should be made consistent
>
> Thanks,
>
> Matt
>
>
>> Thanks,
>> Shri
>>
>> From: Miguel Angel Salazar de Troya <salazardetroya at gmail.com>
>> Date: Wed, 25 Feb 2015 08:48:10 -0600
>> To: Matthew Knepley <knepley at gmail.com>
>> Cc: Shri <abhyshr at mcs.anl.gov>, "petsc-users at mcs.anl.gov" <
>> petsc-users at mcs.anl.gov>
>>
>> Subject: Re: [petsc-users] DMNetworkGetEdgeRange() in parallel
>>
>> I modified the DMNetwork example to include the new DM with the
>> modified section. It has the same problems. Please find attached the code
>> to this email.
>>
>> Thanks
>>
>> On Tue, Feb 24, 2015 at 6:49 PM, Matthew Knepley <knepley at gmail.com>
>> wrote:
>>
>>> On Tue, Feb 24, 2015 at 6:42 PM, Miguel Angel Salazar de Troya <
>>> salazardetroya at gmail.com> wrote:
>>>
>>>> I implemented the code as agreed, but I don't get the results I
>>>> expected. When I create the vector with DMCreateGlobalVector(), I obtain a
>>>> vector with a layout similar to the original DMNetwork, instead of the
>>>> cloned network with the new PetscSection. The code is as follows:
>>>>
>>>> DMClone(dm, &dmEdge);
>>>>
>>>> PetscSectionCreate(PETSC_COMM_WORLD, &s);
>>>> PetscSectionSetNumFields(s, 1);
>>>> PetscSectionSetFieldComponents(s, 0, 1);
>>>>
>>>> // Now to set the chart, I pick the edge range
>>>>
>>>> DMNetworkGetEdgeRange(dmEdge, & eStart, & eEnd)
>>>>
>>>> PetscSectionSetChart(s, eStart, eEnd);
>>>>
>>>> for(PetscInt e = eStart; c < eEnd; ++e) {
>>>> PetscSectionSetDof(s, e, 1);
>>>> PetscSectionSetFieldDof(s, e, 0, 1);
>>>> }
>>>> PetscSectionSetUp(s);
>>>>
>>>> DMSetDefaultSection(dmEdge s);
>>>> DMCreateGlobalVector(dmEdge, &globalVec);
>>>>
>>>> When I get into DMCreateGlobalVector(dmEdge, &globalVec) in the
>>>> debugger, in the function DMCreateSubDM_Section_Private() I call
>>>> PetscSectionView() on the section
>>>>
>>>
>>> I have no idea why you would be in DMCreateSubDM().
>>>
>>> Just view globalVec. If the code is as above, it will give you a
>>> vector with that layout. If not
>>> it should be trivial to make a small code and send it. I do this
>>> everywhere is PETSc, so the
>>> basic mechanism certainly works.
>>>
>>> Thanks,
>>>
>>> Matt
>>>
>>>
>>>> obtained by DMGetDefaultGlobalSection(dm, §ionGlobal), and I
>>>> obtain a PetscSection nothing like the one I see when I call PetscSectionView()
>>>> on the PetscSection I created above. Does this have anything to do? I
>>>> tried to compare this strange PetscSection with the one from the original
>>>> DMNetwork, I call DMGetDefaultGlobalSection(dm, §ionGlobal) before
>>>> the first line of the snippet above and I get this error message.
>>>>
>>>> 0]PETSC ERROR: --------------------- Error Message
>>>> --------------------------------------------------------------
>>>> [0]PETSC ERROR: Object is in wrong state
>>>> [0]PETSC ERROR: DM must have a default PetscSection in order to create
>>>> a global PetscSection
>>>>
>>>> Thanks in advance
>>>> Miguel
>>>>
>>>>
>>>> On Mon, Feb 23, 2015 at 3:24 PM, Matthew Knepley <knepley at gmail.com>
>>>> wrote:
>>>>
>>>>> On Mon, Feb 23, 2015 at 2:15 PM, Miguel Angel Salazar de Troya <
>>>>> salazardetroya at gmail.com> wrote:
>>>>>
>>>>>> Thanks a lot, the partition should be done before setting up the
>>>>>> section, right?
>>>>>>
>>>>>
>>>>> The partition will be automatic. All you have to do is make the
>>>>> local section. The DM is already partitioned,
>>>>> and the Section will inherit that.
>>>>>
>>>>> Matt
>>>>>
>>>>>
>>>>>> Miguel
>>>>>>
>>>>>> On Mon, Feb 23, 2015 at 2:05 PM, Matthew Knepley <knepley at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> On Mon, Feb 23, 2015 at 1:40 PM, Miguel Angel Salazar de Troya <
>>>>>>> salazardetroya at gmail.com> wrote:
>>>>>>>
>>>>>>>> Wouldn't including the edge variables in the global vector make the
>>>>>>>> code slower? I'm using the global vector in a TS, using one of the explicit
>>>>>>>> RK schemes. The edge variables would not be updated in the RHSFunction
>>>>>>>> evaluation. I only change the edge variables in the TSUpdate. If the global
>>>>>>>> vector had the edge variables, it would be a much larger vector, and all
>>>>>>>> the vector operations performed by the TS would be slower. Although the
>>>>>>>> vector F returned by the RHSFunction would be zero in the edge variable
>>>>>>>> components. I guess that being the vector sparse that would not be a
>>>>>>>> problem.
>>>>>>>>
>>>>>>>> I think I'm more interested in the PetscSection approach because
>>>>>>>> it might require less modifications in my code. However, I don't know how I
>>>>>>>> could do this. Maybe something like this?
>>>>>>>>
>>>>>>>> PetscSectionCreate(PETSC_COMM_WORLD, &s);
>>>>>>>> PetscSectionSetNumFields(s, 1);
>>>>>>>> PetscSectionSetFieldComponents(s, 0, 1);
>>>>>>>>
>>>>>>>> // Now to set the chart, I pick the edge range
>>>>>>>>
>>>>>>>> DMNetworkGetEdgeRange(dm, & eStart, & eEnd
>>>>>>>>
>>>>>>>> PetscSectionSetChart(s, eStart, eEnd);
>>>>>>>>
>>>>>>>> for(PetscInt e = eStart; c < eEnd; ++e) {
>>>>>>>> PetscSectionSetDof(s, e, 1);
>>>>>>>> PetscSectionSetFieldDof(s, e, 1, 1);
>>>>>>>>
>>>>>>>
>>>>>>> It should be PetscSectionSetFieldDof(s, e, 0, 1);
>>>>>>>
>>>>>>>
>>>>>>>> }
>>>>>>>> PetscSectionSetUp(s);
>>>>>>>>
>>>>>>>> Now in the manual I see this:
>>>>>>>>
>>>>>>>
>>>>>>> First you want to do:
>>>>>>>
>>>>>>> DMClone(dm, &dmEdge);
>>>>>>>
>>>>>>> and then use dmEdge below.
>>>>>>>
>>>>>>>
>>>>>>>> DMSetDefaultSection(dm, s);
>>>>>>>> DMGetLocalVector(dm, &localVec);
>>>>>>>> DMGetGlobalVector(dm, &globalVec);
>>>>>>>>
>>>>>>>> Setting up the default section in the DM would interfere with the
>>>>>>>> section already set up with the variables in the vertices?
>>>>>>>>
>>>>>>>
>>>>>>> Yep, thats why you would use a clone.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Matt
>>>>>>>
>>>>>>>
>>>>>>>> Thanks a lot for your responses.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Feb 23, 2015 at 11:37 AM, Matthew Knepley <
>>>>>>>> knepley at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> On Mon, Feb 23, 2015 at 9:27 AM, Miguel Angel Salazar de Troya <
>>>>>>>>> salazardetroya at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I'm iterating through local edges given in DMNetworkGetEdgeRange().
>>>>>>>>>> For each edge, I extract or modify its corresponding value in a global
>>>>>>>>>> petsc vector. Therefore that vector must have as many components as edges
>>>>>>>>>> there are in the network. To extract the value in the vector, I use
>>>>>>>>>> VecGetArray() and a variable counter that is incremented in each iteration.
>>>>>>>>>> The array that I obtain in VecGetArray() has to be the same size
>>>>>>>>>> than the edge range. That variable counter starts as 0, so if the array
>>>>>>>>>> that I obtained in VecGetArray() is x_array, x_array[0] must be
>>>>>>>>>> the component in the global vector that corresponds with the start edge
>>>>>>>>>> given in DMNetworkGetEdgeRange()
>>>>>>>>>>
>>>>>>>>>> I need that global petsc vector because I will use it in other
>>>>>>>>>> operations, it's not just data. Sorry for the confusion. Thanks in advance.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This sounds like an assembly operation. The usual paradigm is to
>>>>>>>>> compute in the local space, and then communicate to get to the global
>>>>>>>>> space. So you would make a PetscSection that had 1 (or some) unknowns on
>>>>>>>>> each cell (edge) and then you can use DMCreateGlobal/LocalVector() and
>>>>>>>>> DMLocalToGlobal() to do this.
>>>>>>>>>
>>>>>>>>> Does that make sense?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Matt
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Miguel
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 23, 2015 at 9:09 AM, Matthew Knepley <
>>>>>>>>>> knepley at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 23, 2015 at 8:42 AM, Miguel Angel Salazar de Troya
>>>>>>>>>>> <salazardetroya at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks, that will help me. Now what I would like to have is the
>>>>>>>>>>>> following: if I have two processors and ten edges, the partitioning results
>>>>>>>>>>>> in the first processor having the edges 0-4 and the second processor, the
>>>>>>>>>>>> edges 5-9. I also have a global vector with as many components as edges,
>>>>>>>>>>>> 10. How can I partition it so the first processor also has the 0-4
>>>>>>>>>>>> components and the second, the 5-9 components of the vector?
>>>>>>>>>>>>
>>>>>>>>>>> I think it would help to know what you want to accomplish.
>>>>>>>>>>> This is how you are proposing to do it.'
>>>>>>>>>>>
>>>>>>>>>>> If you just want to put data on edges, DMNetwork has a
>>>>>>>>>>> facility for that already.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> Matt
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Miguel
>>>>>>>>>>>> On Feb 23, 2015 8:08 AM, "Abhyankar, Shrirang G." <
>>>>>>>>>>>> abhyshr at mcs.anl.gov> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Miguel,
>>>>>>>>>>>>> One possible way is to store the global numbering of any
>>>>>>>>>>>>> edge/vertex in the "component" attached to it. Once the mesh gets
>>>>>>>>>>>>> partitioned, the components are also distributed so you can easily retrieve
>>>>>>>>>>>>> the global number of any edge/vertex by accessing its component. This is
>>>>>>>>>>>>> what is done in the DMNetwork example pf.c although the global numbering is
>>>>>>>>>>>>> not used for anything.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Shri
>>>>>>>>>>>>> From: Matthew Knepley <knepley at gmail.com>
>>>>>>>>>>>>> Date: Mon, 23 Feb 2015 07:54:34 -0600
>>>>>>>>>>>>> To: Miguel Angel Salazar de Troya <salazardetroya at gmail.com>
>>>>>>>>>>>>> Cc: "petsc-users at mcs.anl.gov" <petsc-users at mcs.anl.gov>
>>>>>>>>>>>>> Subject: Re: [petsc-users] DMNetworkGetEdgeRange() in parallel
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Feb 22, 2015 at 3:59 PM, Miguel Angel Salazar de
>>>>>>>>>>>>> Troya <salazardetroya at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks. Once I obtain that Index Set with the routine DMPlexCreateCellNumbering()
>>>>>>>>>>>>>> (I assume that the edges in DMNetwork correspond to cells in DMPlex) can I
>>>>>>>>>>>>>> use it to partition a vector with as many components as edges I have in my
>>>>>>>>>>>>>> network?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I do not completely understand the question.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you want a partition of the edges, you can use
>>>>>>>>>>>>> DMPlexCreatePartition() and its friend DMPlexDistribute(). What
>>>>>>>>>>>>> are you trying to do?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Matt
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>> Miguel
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Feb 22, 2015 at 12:15 PM, Matthew Knepley <
>>>>>>>>>>>>>> knepley at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sun, Feb 22, 2015 at 11:01 AM, Miguel Angel Salazar de
>>>>>>>>>>>>>>> Troya <salazardetroya at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I noticed that the routine DMNetworkGetEdgeRange()
>>>>>>>>>>>>>>>> returns the local indices for the edge range. Is there any way to obtain
>>>>>>>>>>>>>>>> the global indices? So if my network has 10 edges, the processor 1 has the
>>>>>>>>>>>>>>>> 0-4 edges and the processor 2, the 5-9 edges, how can I obtain this
>>>>>>>>>>>>>>>> information?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> One of the points of DMPlex is we do not require a global
>>>>>>>>>>>>>>> numbering. Everything is numbered
>>>>>>>>>>>>>>> locally, and the PetscSF maps local numbers to local numbers
>>>>>>>>>>>>>>> in order to determine ownership.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If you want to create a global numbering for some reason,
>>>>>>>>>>>>>>> you can using DMPlexCreatePointNumbering().
>>>>>>>>>>>>>>> There are also cell and vertex versions that we use for
>>>>>>>>>>>>>>> output, so you could do it just for edges as well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Matt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>> Miguel
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> *Miguel Angel Salazar de Troya*
>>>>>>>>>>>>>>>> Graduate Research Assistant
>>>>>>>>>>>>>>>> Department of Mechanical Science and Engineering
>>>>>>>>>>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>>>>>>>>>>> (217) 550-2360
>>>>>>>>>>>>>>>> salaza11 at illinois.edu
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> What most experimenters take for granted before they begin
>>>>>>>>>>>>>>> their experiments is infinitely more interesting than any results to which
>>>>>>>>>>>>>>> their experiments lead.
>>>>>>>>>>>>>>> -- Norbert Wiener
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> *Miguel Angel Salazar de Troya*
>>>>>>>>>>>>>> Graduate Research Assistant
>>>>>>>>>>>>>> Department of Mechanical Science and Engineering
>>>>>>>>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>>>>>>>>> (217) 550-2360
>>>>>>>>>>>>>> salaza11 at illinois.edu
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> What most experimenters take for granted before they begin
>>>>>>>>>>>>> their experiments is infinitely more interesting than any results to which
>>>>>>>>>>>>> their experiments lead.
>>>>>>>>>>>>> -- Norbert Wiener
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>>>>>> experiments lead.
>>>>>>>>>>> -- Norbert Wiener
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Miguel Angel Salazar de Troya*
>>>>>>>>>> Graduate Research Assistant
>>>>>>>>>> Department of Mechanical Science and Engineering
>>>>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>>>>> (217) 550-2360
>>>>>>>>>> salaza11 at illinois.edu
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>>>> experiments lead.
>>>>>>>>> -- Norbert Wiener
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Miguel Angel Salazar de Troya*
>>>>>>>> Graduate Research Assistant
>>>>>>>> Department of Mechanical Science and Engineering
>>>>>>>> University of Illinois at Urbana-Champaign
>>>>>>>> (217) 550-2360
>>>>>>>> salaza11 at illinois.edu
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> What most experimenters take for granted before they begin their
>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>> experiments lead.
>>>>>>> -- Norbert Wiener
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Miguel Angel Salazar de Troya*
>>>>>> Graduate Research Assistant
>>>>>> Department of Mechanical Science and Engineering
>>>>>> University of Illinois at Urbana-Champaign
>>>>>> (217) 550-2360
>>>>>> salaza11 at illinois.edu
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> What most experimenters take for granted before they begin their
>>>>> experiments is infinitely more interesting than any results to which their
>>>>> experiments lead.
>>>>> -- Norbert Wiener
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Miguel Angel Salazar de Troya*
>>>> Graduate Research Assistant
>>>> Department of Mechanical Science and Engineering
>>>> University of Illinois at Urbana-Champaign
>>>> (217) 550-2360
>>>> salaza11 at illinois.edu
>>>>
>>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>
>>
>>
>> --
>> *Miguel Angel Salazar de Troya*
>> Graduate Research Assistant
>> Department of Mechanical Science and Engineering
>> University of Illinois at Urbana-Champaign
>> (217) 550-2360
>> salaza11 at illinois.edu
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
--
*Miguel Angel Salazar de Troya*
Graduate Research Assistant
Department of Mechanical Science and Engineering
University of Illinois at Urbana-Champaign
(217) 550-2360
salaza11 at illinois.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150225/8657aa31/attachment-0001.html>
More information about the petsc-users
mailing list