<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Mon, Feb 19, 2018 at 4:33 PM, Adrian Croucher <span dir="ltr"><<a href="mailto:a.croucher@auckland.ac.nz" target="_blank">a.croucher@auckland.ac.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>hi Matt</p>
<p>I tried what you suggested and modified
DMPlexGetFieldType_Internal() so that all processes have the same
field type (ft):</p>
<p><tt></tt></p></div></blockquote><div>Yep, this is going to take specifying in the interface that it is collective. It looks like it might be used as if it is not. I will get to it</div><div>as soon as I can.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><p><tt>PetscErrorCode DMPlexGetFieldType_Internal(DM dm,
PetscSection section, PetscInt field, PetscInt *sStart, PetscInt
*sEnd, PetscViewerVTKFieldType *ft)</tt><tt><br>
</tt><tt>{</tt><tt><br>
</tt><tt> PetscInt dim, pStart, pEnd, vStart, vEnd, cStart,
cEnd, cEndInterior;</tt><tt><br>
</tt><tt> PetscInt vcdof[2] = {0,0}, globalvcdof[2];</tt><tt><br>
</tt><tt> PetscErrorCode ierr;</tt><tt><br>
</tt><tt><br>
</tt><tt> PetscFunctionBegin;</tt><tt><br>
</tt><tt> *ft = PETSC_VTK_POINT_FIELD;</tt><tt><br>
</tt><tt> ierr = DMGetDimension(dm, &dim);CHKERRQ(ierr);</tt><tt><br>
</tt><tt> ierr = DMPlexGetDepthStratum(dm, 0, &vStart,
&vEnd);CHKERRQ(ierr);</tt><tt><br>
</tt><tt> ierr = DMPlexGetHeightStratum(dm, 0, &cStart,
&cEnd);CHKERRQ(ierr);</tt><tt><br>
</tt><tt> ierr = DMPlexGetHybridBounds(dm, &cEndInterior,
NULL, NULL, NULL);CHKERRQ(ierr);</tt><tt><br>
</tt><tt> cEnd = cEndInterior < 0 ? cEnd : cEndInterior;</tt><tt><br>
</tt><tt> ierr = PetscSectionGetChart(section, &pStart,
&pEnd);CHKERRQ(ierr);</tt><tt><br>
</tt><tt> if (field >= 0) {</tt><tt><br>
</tt><tt> if ((vStart >= pStart) && (vStart <
pEnd)) {ierr = PetscSectionGetFieldDof(<wbr>section, vStart, field,
&vcdof[0]);CHKERRQ(ierr);}</tt><tt><br>
</tt><tt> if ((cStart >= pStart) && (cStart <
pEnd)) {ierr = PetscSectionGetFieldDof(<wbr>section, cStart, field,
&vcdof[1]);CHKERRQ(ierr);}</tt><tt><br>
</tt><tt> } else {</tt><tt><br>
</tt><tt> if ((vStart >= pStart) && (vStart <
pEnd)) {ierr = PetscSectionGetDof(section, vStart,
&vcdof[0]);CHKERRQ(ierr);}</tt><tt><br>
</tt><tt> if ((cStart >= pStart) && (cStart <
pEnd)) {ierr = PetscSectionGetDof(section, cStart,
&vcdof[1]);CHKERRQ(ierr);}</tt><tt><br>
</tt><tt> }</tt><tt><br>
</tt><tt> ierr = MPI_Allreduce(&vcdof, &globalvcdof, 2,
MPIU_INT, MPI_MAX,
PetscObjectComm((PetscObject)<wbr>dm));CHKERRQ(ierr);</tt><tt><br>
</tt><tt> if (globalvcdof[0]) {</tt><tt><br>
</tt><tt> *sStart = vStart;</tt><tt><br>
</tt><tt> *sEnd = vEnd;</tt><tt><br>
</tt><tt> if (globalvcdof[0] == dim) *ft =
PETSC_VTK_POINT_VECTOR_FIELD;</tt><tt><br>
</tt><tt> else *ft = PETSC_VTK_POINT_FIELD;</tt><tt><br>
</tt><tt> } else if (globalvcdof[1]) {</tt><tt><br>
</tt><tt> *sStart = cStart;</tt><tt><br>
</tt><tt> *sEnd = cEnd;</tt><tt><br>
</tt><tt> if (globalvcdof[1] == dim) *ft =
PETSC_VTK_CELL_VECTOR_FIELD;</tt><tt><br>
</tt><tt> else *ft = PETSC_VTK_CELL_FIELD;</tt><tt><br>
</tt><tt> } else SETERRQ(PetscObjectComm((<wbr>PetscObject) dm),
PETSC_ERR_ARG_WRONG, "Could not classify input Vec for VTK");</tt><tt><br>
</tt><tt> PetscFunctionReturn(0);</tt><tt><br>
</tt><tt>}</tt><tt><br>
</tt><br>
</p>
<p>This succeeded in preventing the error I was originally getting.
However, it just crashes further down the line. The error message
is below- any suggestions on what else might need changing to fix
this?</p>
<p>Cheers, Adrian</p>
<p><br>
</p>
<p><tt>[0]PETSC ERROR: --------------------- Error Message
------------------------------<wbr>------------------------------<wbr>--</tt><tt><br>
</tt><tt>[0]PETSC ERROR: Petsc has generated inconsistent data</tt><tt><br>
</tt><tt>[0]PETSC ERROR: MPI_Allreduce() called in different
locations (code lines) on different processors</tt><tt><br>
</tt><tt>[0]PETSC ERROR: See
<a class="m_7969656230746099817moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">http://www.mcs.anl.gov/petsc/<wbr>documentation/faq.html</a> for trouble
shooting.</tt><tt><br>
</tt><tt>[0]PETSC ERROR: Petsc Development GIT revision:
v3.8.3-2236-gcb4b83bf76 GIT Date: 2018-02-13 12:48:33 -0700</tt><tt><br>
</tt><tt>[0]PETSC ERROR: waiwera on a linux-gnu-c-opt named
en-354401 by acro018 Tue Feb 20 10:18:55 2018</tt><tt><br>
</tt><tt>[0]PETSC ERROR: Configure options --with-x
--download-hdf5 --download-netcdf --download-pnetcdf
--download-exodusii --download-triangle --download-ptscotch
--download-chaco --download-hypre</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #1 VecSetBlockSize() line 1351 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>vector.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #2 VecSetBlockSize() line 1351 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>vector.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #3 VecCreateMPIWithArray() line 610 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/impls/mpi/<wbr>pbvec.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: [1]PETSC ERROR: ---------------------
Error Message
------------------------------<wbr>------------------------------<wbr>--</tt><tt><br>
</tt><tt>[1]PETSC ERROR: Petsc has generated inconsistent data</tt><tt><br>
</tt><tt>[1]PETSC ERROR: MPI_Allreduce() called in different
locations (code lines) on different processors</tt><tt><br>
</tt><tt>[1]PETSC ERROR: See
<a class="m_7969656230746099817moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">http://www.mcs.anl.gov/petsc/<wbr>documentation/faq.html</a> for trouble
shooting.</tt><tt><br>
</tt><tt>[1]PETSC ERROR: Petsc Development GIT revision:
v3.8.3-2236-gcb4b83bf76 GIT Date: 2018-02-13 12:48:33 -0700</tt><tt><br>
</tt><tt>[1]PETSC ERROR: waiwera on a linux-gnu-c-opt named
en-354401 by acro018 Tue Feb 20 10:18:55 2018</tt><tt><br>
</tt><tt>[1]PETSC ERROR: Configure options --with-x
--download-hdf5 --download-netcdf --download-pnetcdf
--download-exodusii --download-triangle --download-ptscotch
--download-chaco --download-hypre</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #1 PetscSplitOwnership() line 88 in
/home/acro018/software/PETSc/<wbr>code/src/sys/utils/psplit.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #2 PetscSplitOwnership() line 88 in
/home/acro018/software/PETSc/<wbr>code/src/sys/utils/psplit.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #3 PetscLayoutSetUp() line 137 in
/home/acro018/software/PETSc/<wbr>code/src/vec/is/utils/pmap.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: --------------------- Error Message
------------------------------<wbr>------------------------------<wbr>--</tt><tt><br>
</tt><tt>[2]PETSC ERROR: Petsc has generated inconsistent data</tt><tt><br>
</tt><tt>[2]PETSC ERROR: MPI_Allreduce() called in different
locations (code lines) on different processors</tt><tt><br>
</tt><tt>[2]PETSC ERROR: See
<a class="m_7969656230746099817moz-txt-link-freetext" href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">http://www.mcs.anl.gov/petsc/<wbr>documentation/faq.html</a> for trouble
shooting.</tt><tt><br>
</tt><tt>[2]PETSC ERROR: Petsc Development GIT revision:
v3.8.3-2236-gcb4b83bf76 GIT Date: 2018-02-13 12:48:33 -0700</tt><tt><br>
</tt><tt>[2]PETSC ERROR: waiwera on a linux-gnu-c-opt named
en-354401 by acro018 Tue Feb 20 10:18:55 2018</tt><tt><br>
</tt><tt>[2]PETSC ERROR: Configure options --with-x
--download-hdf5 --download-netcdf --download-pnetcdf
--download-exodusii --download-triangle --download-ptscotch
--download-chaco --download-hypre</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #1 PetscSplitOwnership() line 88 in
/home/acro018/software/PETSc/<wbr>code/src/sys/utils/psplit.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #2 PetscSplitOwnership() line 88 in
/home/acro018/software/PETSc/<wbr>code/src/sys/utils/psplit.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #3 PetscLayoutSetUp() line 137 in
/home/acro018/software/PETSc/<wbr>code/src/vec/is/utils/pmap.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #4 VecCreate_MPI_Private() line 489 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/impls/mpi/<wbr>pbvec.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #5 VecCreateMPIWithArray() line 611 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/impls/mpi/<wbr>pbvec.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #6 VecGetSubVector() line 1301 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>rvector.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #7 PetscSectionGetField_Internal(<wbr>) line
276 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/utils/<wbr>vsection.c</tt><tt><br>
</tt><tt>#4 VecGetSubVector() line 1301 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>rvector.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #5 PetscSectionGetField_Internal(<wbr>) line
276 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/utils/<wbr>vsection.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #6 VecView_Plex_Local_HDF5_<wbr>Internal()
line 169 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/<wbr>plexhdf5.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #7 VecView_Plex_Local() line 306 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/plex.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #8 VecView_Plex_HDF5_Internal() line 214
in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/<wbr>plexhdf5.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #9 VecView_Plex() line 354 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/plex.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: #10 VecView() line 586 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>vector.c</tt><tt><br>
</tt><tt>[0]PETSC ERROR: [1]PETSC ERROR: #4
VecCreate_MPI_Private() line 489 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/impls/mpi/<wbr>pbvec.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #5 VecCreateMPIWithArray() line 611 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/impls/mpi/<wbr>pbvec.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #6 VecGetSubVector() line 1301 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>rvector.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #7 PetscSectionGetField_Internal(<wbr>) line
276 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/utils/<wbr>vsection.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #8 VecView_Plex_Local_HDF5_<wbr>Internal()
line 169 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/<wbr>plexhdf5.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #9 VecView_Plex_Local() line 306 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/plex.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #8 VecView_Plex_Local_HDF5_<wbr>Internal()
line 169 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/<wbr>plexhdf5.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #9 VecView_Plex_Local() line 306 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/plex.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #10 VecView_Plex_HDF5_Internal() line 214
in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/<wbr>plexhdf5.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #11 VecView_Plex() line 354 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/plex.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #12 VecView() line 586 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>vector.c</tt><tt><br>
</tt><tt>[2]PETSC ERROR: #13 User provided function() line 0 in
User file</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #10 VecView_Plex_HDF5_Internal() line 214
in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/<wbr>plexhdf5.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #11 VecView_Plex() line 354 in
/home/acro018/software/PETSc/<wbr>code/src/dm/impls/plex/plex.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #12 VecView() line 586 in
/home/acro018/software/PETSc/<wbr>code/src/vec/vec/interface/<wbr>vector.c</tt><tt><br>
</tt><tt>[1]PETSC ERROR: #13 User provided function() line 0 in
User file</tt><tt><br>
</tt><tt>#11 User provided function() line 0 in User file</tt><br>
<br>
</p><span class="">
<br>
<div class="m_7969656230746099817moz-cite-prefix">On 13/02/18 17:13, Matthew Knepley
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Mon, Feb 12, 2018 at 10:57 PM,
Adrian Croucher <span dir="ltr"><<a href="mailto:a.croucher@auckland.ac.nz" target="_blank">a.croucher@auckland.ac.nz</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">hi,<br>
<br>
I have a DMPlex which I pass in to DMCreateGlobalVector()
to create a vector, and then use VecView() to view the
vector.<br>
<br>
If the DM happens to contain no points on one process,
this still works fine if I view the vector using
PETSC_VIEWER_STDOUT_WORLD.<br>
<br>
However if I create an HDF5 viewer using
PetscViewerHDF5Open() and try to view the vector using
that, it crashes with the error message "Could not
classify input Vec for VTK".<br>
<br>
It goes into VecView_Plex_Local_HDF5_Intern<wbr>al(),
which calls DMPlexGetFieldType_Internal() (plex.c:93).
This has some conditionals testing if vStart < pEnd or
cStart < pEnd, which are never satisfied because pEnd =
0. This is what leads to it raising the error.<br>
<br>
This function returns the point range sStart, sEnd and
field type ft. To handle the case where there are no
points on the process, should it perhaps just return
sStart, sEnd = 0,0 and a default (or maybe null) value for
ft?</blockquote>
<div><br>
</div>
<div>Crap, this is a problem. I will have to put some
parallel code in there because everyone must agree on the
ft.</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
</div>
</div>
</div>
</blockquote>
<br>
</span><span class=""><pre class="m_7969656230746099817moz-signature" cols="72">--
Dr Adrian Croucher
Senior Research Fellow
Department of Engineering Science
University of Auckland, New Zealand
email: <a class="m_7969656230746099817moz-txt-link-abbreviated" href="mailto:a.croucher@auckland.ac.nz" target="_blank">a.croucher@auckland.ac.nz</a>
tel: <a href="tel:+64%209-923%204611" value="+6499234611" target="_blank">+64 (0)9 923 4611</a>
</pre>
</span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.caam.rice.edu/~mk51/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div>
</div></div>