[petsc-users] Petsc Section in DMPlex
Nicholas Arnold-Medabalimi
narnoldm at umich.edu
Thu Dec 8 20:32:48 CST 2022
Hi
Thank you for your help with this barrage of questions. The AddField, while
nice for visualization, isn't the critical path for my development.
I've gotten the Filtered DM with its corresponding state vector sorted out.
Now I'm moving on to the mesh distribution. From your 2nd email in this
thread I've added
call DMPlexDistribute(dmplex_filtered, 1, distrib_sf, dmplex_distrib,ierr)
! adds section to dmplex_filtered and allocates vec_distrib using
DMCreateGlobalVector
call addSectionToDMPlex(dmplex_distrib,vec_distrib)
call DMGetGlobalSection(dmplex_distrib,distributedSection,ierr)
call DMPlexDistributeField(dmplex_disb, distrib_sf, filteredfieldSection,
vec_filtered, distributedSection, vec_distrib,ierr)
I'm not entirely clear if the DM I should feed into Distribute field should
be the starting or end DM but based on doc I think it should be the
destination. But I'm getting this error on run.
[1]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[1]PETSC ERROR: No support for this operation for this object type
[1]PETSC ERROR: PetscSectionSetUp is currently unsupported for
includesConstraints = PETSC_TRUE
[1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
I suspect this is occurring because I am incorrectly handling the 3
sets(full, filtered, distributed) of DMPLEX, Section, and Vec's.
Thanks
Nicholas
On Thu, Dec 8, 2022 at 12:50 PM Nicholas Arnold-Medabalimi <
narnoldm at umich.edu> wrote:
> I think I've figured out the issue. In previous efforts, I used
> DMAddField, which I think was key for the output to work properly.
>
>
> On Thu, Dec 8, 2022 at 10:48 AM Nicholas Arnold-Medabalimi <
> narnoldm at umich.edu> wrote:
>
>> Hi Matt
>>
>> Thanks. Found the issue just messed up the Fortran to C indexing.
>>
>> Another question. I have been using the Petsc VTK output to view things.
>> In some previous efforts, I used the PetscFVM object to set up my section
>> data. When I output vectors using that method in ParaView, I could view the
>> rank information and the state vector to visualize. As far as I can tell
>> when I do the same with Vectors that were created with my manually created
>> section the VecView using PETSCVIEWERVTK only mesh is output with the Rank
>> distribution is viewable (basically as if I had done a DM output instead of
>> a Vec output). My guess is this is because I'm not setting something in my
>> section Field setup properly for the VTK viewer to output it?
>>
>> For my section set up, I am calling
>> PetscSectionCreate
>> PetscSectionSetChart
>> PetscSectionSetFieldName
>> and then the appropriate PetscSectionSetDof
>> PetscSectionSetup
>> DMSetLocalSection
>>
>> Thanks again for all the help.
>>
>> Sincerely
>> Nicholas
>>
>>
>> On Thu, Dec 8, 2022 at 7:27 AM Matthew Knepley <knepley at gmail.com> wrote:
>>
>>> On Thu, Dec 8, 2022 at 3:04 AM Nicholas Arnold-Medabalimi <
>>> narnoldm at umich.edu> wrote:
>>>
>>>> Hi Matt
>>>>
>>>> I think I've gotten it just about there. I'm just having an issue with
>>>> the VecISCopy. I have an IS built that matches size correctly to map from
>>>> the full state to the filtered state. The core issue I think, is should the
>>>> expanded IS the ownership range of the vector subtracted out. Looking at
>>>> the implementation, it looks like VecISCopy takes care of that for me.
>>>> (Line 573 in src/vec/vec/utils/projection.c) But I could be mistaken.
>>>>
>>>
>>> It is a good question. We have tried to give guidance on the manpage:
>>>
>>> The index set identifies entries in the global vector. Negative
>>> indices are skipped; indices outside the ownership range of vfull will
>>> raise an error.
>>>
>>> which means that it expects _global_ indices, and you have retrieved the
>>> global section, so that matches.
>>> The calculation of the index size looks right to me, and so does the
>>> index calculation.
>>>
>>> I would put a check in the loop, making sure that the calculated indices
>>> lie within [oStart, oEnd). The global
>>> section is designed to ensure that. It is not clear why one would lie
>>> outside.
>>>
>>> When I am debugging, I run a very small problem, and print out all the
>>> sections.
>>>
>>> Thanks,
>>>
>>> Matt
>>>
>>>
>>>> The error I am getting is:
>>>> [0]PETSC ERROR: --------------------- Error Message
>>>> --------------------------------------------------------------
>>>> [0]PETSC ERROR: No support for this operation for this object type
>>>> [0]PETSC ERROR: Only owned values supported
>>>>
>>>>
>>>> Here is what I am currently doing.
>>>>
>>>> call DMPlexFilter(dmplex_full, iBlankLabel, 1, dmplex_filtered,ierr)
>>>> call DMPlexGetSubpointIS(dmplex_filtered, subpointsIS,ierr)
>>>>
>>>> ! adds section to dmplex_filtered and allocates vec_filtered using
>>>> DMCreateGlobalVector
>>>> call addSectionToDMPlex(dmplex_filtered,vec_filtered)
>>>>
>>>> ! Get Sections for dmplex_filtered and dmplex_full
>>>> call DMGetGlobalSection(dmplex_filtered,filteredfieldSection,ierr)
>>>> call DMGetGlobalSection(dmplex_full,fullfieldSection,ierr)
>>>>
>>>>
>>>>
>>>> call ISGetIndicesF90(subpointsIS, subPointKey,ierr)
>>>> ExpandedIndexSize = 0
>>>> do i = 1, size(subPointKey)
>>>> call PetscSectionGetDof(fullfieldSection, subPointKey(i), dof,ierr)
>>>> ExpandedIndexSize = ExpandedIndexSize + dof
>>>> enddo
>>>>
>>>>
>>>> !Create expandedIS from offset sections of full and filtered sections
>>>> allocate(ExpandedIndex(ExpandedIndexSize))
>>>> call VecGetOwnershipRange(vec_full,oStart,oEnd,ierr)
>>>> do i = 1, size(subPointKey)
>>>> call PetscSectionGetOffset(fullfieldSection, subPointKey(i),
>>>> offset,ierr)
>>>> call PetscSectionGetDof(fullfieldSection, subPointKey(i), dof,ierr)
>>>> !offset=offset-oStart !looking at VecIScopy it takes care of
>>>> this subtraction (not sure)
>>>> do j = 1, (dof)
>>>> ExpandedIndex((i-1)*dof+j) = offset+j
>>>> end do
>>>> enddo
>>>>
>>>> call ISCreateGeneral(PETSC_COMM_WORLD, ExpandedIndexSize,
>>>> ExpandedIndex, PETSC_COPY_VALUES, expandedIS,ierr)
>>>> call ISRestoreIndicesF90(subpointsIS, subPointKey,ierr)
>>>> deallocate(ExpandedIndex)
>>>>
>>>>
>>>> call VecGetLocalSize(vec_full,sizeVec,ierr)
>>>> write(*,*) sizeVec
>>>> call VecGetLocalSize(vec_filtered,sizeVec,ierr)
>>>> write(*,*) sizeVec
>>>> call ISGetLocalSize(expandedIS,sizeVec,ierr)
>>>> write(*,*) sizeVec
>>>> call PetscSynchronizedFlush(PETSC_COMM_WORLD,ierr)
>>>>
>>>>
>>>> call VecISCopy(vec_full,expandedIS,SCATTER_REVERSE,vec_filtered,ierr)
>>>>
>>>>
>>>> Thanks again for the great help.
>>>>
>>>> Sincerely
>>>> Nicholas
>>>>
>>>>
>>>> On Wed, Dec 7, 2022 at 9:29 PM Matthew Knepley <knepley at gmail.com>
>>>> wrote:
>>>>
>>>>> On Wed, Dec 7, 2022 at 9:21 PM Nicholas Arnold-Medabalimi <
>>>>> narnoldm at umich.edu> wrote:
>>>>>
>>>>>> Thank you for the help.
>>>>>>
>>>>>> I think the last piece of the puzzle is how do I create the "expanded
>>>>>> IS" from the subpoint IS using the section?
>>>>>>
>>>>>
>>>>> Loop over the points in the IS. For each point, get the dof and offset
>>>>> from the Section. Make a new IS that has all the
>>>>> dogs, namely each run [offset, offset+dof).
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Matt
>>>>>
>>>>>
>>>>>> Sincerely
>>>>>> Nicholas
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 7, 2022 at 7:06 AM Matthew Knepley <knepley at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> On Wed, Dec 7, 2022 at 6:51 AM Nicholas Arnold-Medabalimi <
>>>>>>> narnoldm at umich.edu> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Thank you so much for your patience. One thing to note: I don't
>>>>>>>> have any need to go back from the filtered distributed mapping back to the
>>>>>>>> full but it is good to know.
>>>>>>>>
>>>>>>>> One aside question.
>>>>>>>> 1) Is natural and global ordering the same in this context?
>>>>>>>>
>>>>>>>
>>>>>>> No.
>>>>>>>
>>>>>>>
>>>>>>>> As far as implementing what you have described.
>>>>>>>>
>>>>>>>> When I call ISView on the generated SubpointIS, I get an unusual
>>>>>>>> error which I'm not sure how to interpret. (this case is running on 2 ranks
>>>>>>>> and the filter label has points located on both ranks of the original DM.
>>>>>>>> However, if I manually get the indices (the commented lines), it seems to
>>>>>>>> not have any issues.
>>>>>>>> call DMPlexFilter(dmplex_full, iBlankLabel, 1,
>>>>>>>> dmplex_filtered,ierr)
>>>>>>>> call DMPlexGetSubpointIS(dmplex_filtered, subpointsIS,ierr)
>>>>>>>> !call ISGetIndicesF90(subpointsIS, subPointKey,ierr)
>>>>>>>> !write(*,*) subPointKey
>>>>>>>> !call ISRestoreIndicesF90(subpointsIS, subPointKey,ierr)
>>>>>>>> call ISView(subpointsIS,PETSC_VIEWER_STDOUT_WORLD,ierr)
>>>>>>>>
>>>>>>>> [1]PETSC ERROR: --------------------- Error Message
>>>>>>>> --------------------------------------------------------------
>>>>>>>> [1]PETSC ERROR: Arguments must have same communicators
>>>>>>>> [1]PETSC ERROR: Different communicators in the two objects:
>>>>>>>> Argument # 1 and 2 flag 3
>>>>>>>> [1]PETSC ERROR: See https://petsc.org/release/faq/ for trouble
>>>>>>>> shooting.
>>>>>>>> [1]PETSC ERROR: Petsc Development GIT revision:
>>>>>>>> v3.18.1-320-g7810d690132 GIT Date: 2022-11-20 20:25:41 -0600
>>>>>>>> [1]PETSC ERROR: Configure options with-fc=mpiifort
>>>>>>>> with-mpi-f90=mpiifort --download-triangle --download-parmetis
>>>>>>>> --download-metis --with-debugging=1 --download-hdf5
>>>>>>>> --prefix=/home/narnoldm/packages/petsc_install
>>>>>>>> [1]PETSC ERROR: #1 ISView() at
>>>>>>>> /home/narnoldm/packages/petsc/src/vec/is/is/interface/index.c:1629
>>>>>>>>
>>>>>>>
>>>>>>> The problem here is the subpointsIS is a _serial_ object, and you
>>>>>>> are using a parallel viewer. You can use PETSC_VIEWER_STDOUT_SELF,
>>>>>>> or you can pull out the singleton viewer from STDOUT_WORLD if you
>>>>>>> want them all to print in order.
>>>>>>>
>>>>>>>
>>>>>>>> As far as the overall process you have described my question on
>>>>>>>> first glance is do I have to allocate/create the vector that is output by
>>>>>>>> VecISCopy before calling it, or does it create the vector automatically?
>>>>>>>>
>>>>>>>
>>>>>>> You create both vectors. I would do it using DMCreateGlobalVector()
>>>>>>> from both DMs.
>>>>>>>
>>>>>>>
>>>>>>>> I think I would need to create it first using a section and Setting
>>>>>>>> the Vec in the filtered DM?
>>>>>>>>
>>>>>>>
>>>>>>> Setting the Section in the filtered DM.
>>>>>>>
>>>>>>>
>>>>>>>> And I presume in this case I would be using the scatter reverse
>>>>>>>> option to go from the full set to the reduced set?
>>>>>>>>
>>>>>>>
>>>>>>> Yes
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Matt
>>>>>>>
>>>>>>>
>>>>>>>> Sincerely
>>>>>>>> Nicholas
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Sincerely
>>>>>>>> Nick
>>>>>>>>
>>>>>>>> On Wed, Dec 7, 2022 at 6:00 AM Matthew Knepley <knepley at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Wed, Dec 7, 2022 at 3:35 AM Nicholas Arnold-Medabalimi <
>>>>>>>>> narnoldm at umich.edu> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Matthew
>>>>>>>>>>
>>>>>>>>>> Thank you for the help. This clarified a great deal.
>>>>>>>>>>
>>>>>>>>>> I have a follow-up question related to DMPlexFilter. It may be
>>>>>>>>>> better to describe what I'm trying to achieve.
>>>>>>>>>>
>>>>>>>>>> I have a general mesh I am solving which has a section with cell
>>>>>>>>>> center finite volume states, as described in my initial email. After
>>>>>>>>>> calculating some metrics, I tag a bunch of cells with an identifying Label
>>>>>>>>>> and use DMFilter to generate a new DM which is only that subset of cells.
>>>>>>>>>> Generally, this leads to a pretty unbalanced DM so I then plan to use
>>>>>>>>>> DMPlexDIstribute to balance that DM across the processors. The coordinates
>>>>>>>>>> pass along fine, but the state(or I should say Section) does not at least
>>>>>>>>>> as far as I can tell.
>>>>>>>>>>
>>>>>>>>>> Assuming I can get a filtered DM I then distribute the DM and
>>>>>>>>>> state using the method you described above and it seems to be working ok.
>>>>>>>>>>
>>>>>>>>>> The last connection I have to make is the transfer of information
>>>>>>>>>> from the full mesh to the "sampled" filtered mesh. From what I can gather I
>>>>>>>>>> would need to get the mapping of points using DMPlexGetSubpointIS and then
>>>>>>>>>> manually copy the values from the full DM section to the filtered DM? I
>>>>>>>>>> have the process from full->filtered->distributed all working for the
>>>>>>>>>> coordinates so its just a matter of transferring the section correctly.
>>>>>>>>>>
>>>>>>>>>> I appreciate all the help you have provided.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Let's do this in two steps, which makes it easier to debug. First,
>>>>>>>>> do not redistribute the submesh. Just use DMPlexGetSubpointIS()
>>>>>>>>> to get the mapping of filtered points to points in the original
>>>>>>>>> mesh. Then create an expanded IS using the Section which makes
>>>>>>>>> dofs in the filtered mesh to dofs in the original mesh. From this
>>>>>>>>> use
>>>>>>>>>
>>>>>>>>> https://petsc.org/main/docs/manualpages/Vec/VecISCopy/
>>>>>>>>>
>>>>>>>>> to move values between the original vector and the filtered vector.
>>>>>>>>>
>>>>>>>>> Once that works, you can try redistributing the filtered mesh.
>>>>>>>>> Before calling DMPlexDistribute() on the filtered mesh, you need to call
>>>>>>>>>
>>>>>>>>> https://petsc.org/main/docs/manualpages/DM/DMSetUseNatural/
>>>>>>>>>
>>>>>>>>> When you redistribute, it will compute a mapping back to the
>>>>>>>>> original layout. Now when you want to transfer values, you
>>>>>>>>>
>>>>>>>>> 1) Create a natural vector with DMCreateNaturalVec()
>>>>>>>>>
>>>>>>>>> 2) Use DMGlobalToNaturalBegin/End() to move values from the
>>>>>>>>> filtered vector to the natural vector
>>>>>>>>>
>>>>>>>>> 3) Use VecISCopy() to move values from the natural vector to the
>>>>>>>>> original vector
>>>>>>>>>
>>>>>>>>> Let me know if you have any problems.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Matt
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Sincerely
>>>>>>>>>> Nicholas
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Nov 28, 2022 at 6:19 AM Matthew Knepley <
>>>>>>>>>> knepley at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Sun, Nov 27, 2022 at 10:22 PM Nicholas Arnold-Medabalimi <
>>>>>>>>>>> narnoldm at umich.edu> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Petsc Users
>>>>>>>>>>>>
>>>>>>>>>>>> I have a question about properly using PetscSection to
>>>>>>>>>>>> assign state variables to a DM. I have an existing DMPlex mesh distributed
>>>>>>>>>>>> on 2 processors. My goal is to have state variables set to the cell
>>>>>>>>>>>> centers. I then want to call DMPlexDistribute, which I hope will balance
>>>>>>>>>>>> the mesh elements and hopefully transport the state variables to the
>>>>>>>>>>>> hosting processors as the cells are distributed to a different processor
>>>>>>>>>>>> count or simply just redistributing after doing mesh adaption.
>>>>>>>>>>>>
>>>>>>>>>>>> Looking at the DMPlex User guide, I should be able to achieve
>>>>>>>>>>>> this with a single field section using SetDof and assigning the DOF to the
>>>>>>>>>>>> points corresponding to cells.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Note that if you want several different fields, you can
>>>>>>>>>>> clone the DM first for this field
>>>>>>>>>>>
>>>>>>>>>>> call DMClone(dm,dmState,ierr)
>>>>>>>>>>>
>>>>>>>>>>> and use dmState in your calls below.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> call DMPlexGetHeightStratum(dm,0,c0,c1,ierr)
>>>>>>>>>>>> call DMPlexGetChart(dm,p0,p1,ierr)
>>>>>>>>>>>> call PetscSectionCreate(PETSC_COMM_WORLD,section,ierr)
>>>>>>>>>>>> call PetscSectionSetNumFields(section,1,ierr) call
>>>>>>>>>>>> PetscSectionSetChart(section,p0,p1,ierr)
>>>>>>>>>>>> do i = c0, (c1-1)
>>>>>>>>>>>> call PetscSectionSetDof(section,i,nvar,ierr)
>>>>>>>>>>>> end do
>>>>>>>>>>>> call PetscSectionSetup(section,ierr)
>>>>>>>>>>>> call DMSetLocalSection(dm,section,ierr)
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> In the loop, I would add a call to
>>>>>>>>>>>
>>>>>>>>>>> call PetscSectionSetFieldDof(section,i,0,nvar,ierr)
>>>>>>>>>>>
>>>>>>>>>>> This also puts in the field breakdown. It is not essential, but
>>>>>>>>>>> nicer.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> From here, it looks like I can access and set the state vars
>>>>>>>>>>>> using
>>>>>>>>>>>>
>>>>>>>>>>>> call DMGetGlobalVector(dmplex,state,ierr)
>>>>>>>>>>>> call DMGetGlobalSection(dmplex,section,ierr)
>>>>>>>>>>>> call VecGetArrayF90(state,stateVec,ierr)
>>>>>>>>>>>> do i = c0, (c1-1)
>>>>>>>>>>>> call PetscSectionGetOffset(section,i,offset,ierr)
>>>>>>>>>>>> stateVec(offset:(offset+nvar))=state_i(:) !simplified
>>>>>>>>>>>> assignment
>>>>>>>>>>>> end do
>>>>>>>>>>>> call VecRestoreArrayF90(state,stateVec,ierr)
>>>>>>>>>>>> call DMRestoreGlobalVector(dmplex,state,ierr)
>>>>>>>>>>>>
>>>>>>>>>>>> To my understanding, I should be using Global vector since this
>>>>>>>>>>>> is a pure assignment operation and I don't need the ghost cells.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Yes.
>>>>>>>>>>>
>>>>>>>>>>> But the behavior I am seeing isn't exactly what I'd expect.
>>>>>>>>>>>>
>>>>>>>>>>>> To be honest, I'm somewhat unclear on a few things
>>>>>>>>>>>>
>>>>>>>>>>>> 1) Should be using nvar fields with 1 DOF each or 1 field with
>>>>>>>>>>>> nvar DOFs or what the distinction between the two methods are?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> We have two divisions in a Section. A field can have a number of
>>>>>>>>>>> components. This is intended to model a vector or tensor field.
>>>>>>>>>>> Then a Section can have a number of fields, such as velocity and
>>>>>>>>>>> pressure for a Stokes problem. The division is mainly to help the
>>>>>>>>>>> user, so I would use the most natural one.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> 2) Adding a print statement after the offset assignment I get
>>>>>>>>>>>> (on rank 0 of 2)
>>>>>>>>>>>> cell 1 offset 0
>>>>>>>>>>>> cell 2 offset 18
>>>>>>>>>>>> cell 3 offset 36
>>>>>>>>>>>> which is expected and works but on rank 1 I get
>>>>>>>>>>>> cell 1 offset 9000
>>>>>>>>>>>> cell 2 offset 9018
>>>>>>>>>>>> cell 3 offset 9036
>>>>>>>>>>>>
>>>>>>>>>>>> which isn't exactly what I would expect. Shouldn't the offsets
>>>>>>>>>>>> reset at 0 for the next rank?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The local and global sections hold different information. This
>>>>>>>>>>> is the source of the confusion. The local section does describe a local
>>>>>>>>>>> vector, and thus includes overlap or "ghost" dofs. The global
>>>>>>>>>>> section describes a global vector. However, it is intended to deliver
>>>>>>>>>>> global indices, and thus the offsets give back global indices.
>>>>>>>>>>> When you use VecGetArray*() you are getting out the local array, and
>>>>>>>>>>> thus you have to subtract the first index on this process. You
>>>>>>>>>>> can get that from
>>>>>>>>>>>
>>>>>>>>>>> VecGetOwnershipRange(v, &rstart, &rEnd);
>>>>>>>>>>>
>>>>>>>>>>> This is the same whether you are using DMDA or DMPlex or any
>>>>>>>>>>> other DM.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> 3) Does calling DMPlexDistribute also distribute the section
>>>>>>>>>>>> data associated with the DOF, based on the description in DMPlexDistribute
>>>>>>>>>>>> it looks like it should?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> No. By default, DMPlexDistribute() only distributes coordinate
>>>>>>>>>>> data. I you want to distribute your field, it would look something like
>>>>>>>>>>> this:
>>>>>>>>>>>
>>>>>>>>>>> DMPlexDistribute(dm, 0, &sfDist, &dmDist);
>>>>>>>>>>> VecCreate(comm, &stateDist);
>>>>>>>>>>> VecSetDM(sateDist, dmDist);
>>>>>>>>>>> PetscSectionCreate(comm §ionDist);
>>>>>>>>>>> DMSetLocalSection(dmDist, sectionDist);
>>>>>>>>>>> DMPlexDistributeField(dmDist, sfDist, section, state,
>>>>>>>>>>> sectionDist, stateDist);
>>>>>>>>>>>
>>>>>>>>>>> We do this in src/dm/impls/plex/tests/ex36.c
>>>>>>>>>>>
>>>>>>>>>>> THanks,
>>>>>>>>>>>
>>>>>>>>>>> Matt
>>>>>>>>>>>
>>>>>>>>>>> I'd appreciate any insight into the specifics of this usage. I
>>>>>>>>>>>> expect I have a misconception on the local vs global section. Thank you.
>>>>>>>>>>>>
>>>>>>>>>>>> Sincerely
>>>>>>>>>>>> Nicholas
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Nicholas Arnold-Medabalimi
>>>>>>>>>>>>
>>>>>>>>>>>> Ph.D. Candidate
>>>>>>>>>>>> Computational Aeroscience Lab
>>>>>>>>>>>> University of Michigan
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>>>>>> experiments lead.
>>>>>>>>>>> -- Norbert Wiener
>>>>>>>>>>>
>>>>>>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>>>>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Nicholas Arnold-Medabalimi
>>>>>>>>>>
>>>>>>>>>> Ph.D. Candidate
>>>>>>>>>> Computational Aeroscience Lab
>>>>>>>>>> University of Michigan
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> What most experimenters take for granted before they begin their
>>>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>>>> experiments lead.
>>>>>>>>> -- Norbert Wiener
>>>>>>>>>
>>>>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Nicholas Arnold-Medabalimi
>>>>>>>>
>>>>>>>> Ph.D. Candidate
>>>>>>>> Computational Aeroscience Lab
>>>>>>>> University of Michigan
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> What most experimenters take for granted before they begin their
>>>>>>> experiments is infinitely more interesting than any results to which their
>>>>>>> experiments lead.
>>>>>>> -- Norbert Wiener
>>>>>>>
>>>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Nicholas Arnold-Medabalimi
>>>>>>
>>>>>> Ph.D. Candidate
>>>>>> Computational Aeroscience Lab
>>>>>> University of Michigan
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> What most experimenters take for granted before they begin their
>>>>> experiments is infinitely more interesting than any results to which their
>>>>> experiments lead.
>>>>> -- Norbert Wiener
>>>>>
>>>>> https://www.cse.buffalo.edu/~knepley/
>>>>> <http://www.cse.buffalo.edu/~knepley/>
>>>>>
>>>>
>>>>
>>>> --
>>>> Nicholas Arnold-Medabalimi
>>>>
>>>> Ph.D. Candidate
>>>> Computational Aeroscience Lab
>>>> University of Michigan
>>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>> https://www.cse.buffalo.edu/~knepley/
>>> <http://www.cse.buffalo.edu/~knepley/>
>>>
>>
>>
>> --
>> Nicholas Arnold-Medabalimi
>>
>> Ph.D. Candidate
>> Computational Aeroscience Lab
>> University of Michigan
>>
>
>
> --
> Nicholas Arnold-Medabalimi
>
> Ph.D. Candidate
> Computational Aeroscience Lab
> University of Michigan
>
--
Nicholas Arnold-Medabalimi
Ph.D. Candidate
Computational Aeroscience Lab
University of Michigan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20221208/3b9d99f3/attachment-0001.html>
More information about the petsc-users
mailing list