[petsc-users] Uninterpolating DMLabels?
Justin Chang
jchang27 at uh.edu
Thu Apr 9 19:21:28 CDT 2015
1) I did what you had suggested and it works very well thanks.
2) In my case, calling uninterpolate does make a difference, just not
necessarily in the solver step (I am using Tao's BLMVM). I attached two
output files, one containing the timings and summaries with uninterpolating
and one without uninterpolating. And the source code if you wanted to
verify the "correctness" of my implementation. Namely, the biggest
difference I see is in setting up the PetscSection. When uninterpolating
the data this steps takes 8.64039 seconds but when I do not uninterpolate
the data it takes 61.2977 seconds. Is this "significant" enough, or is it
unimportant given the fact that this step occurs only once during any
simulation?
3) Also, about DMPlexDistribute(...). Right now it still seems extremely
slow, hence why I am distributing a coarse mesh and having each processor
refine its local portion of the mesh. In my current implementation, I have
rank 0 distribute the mesh via ParMETIS. Will there eventually be a
methodology to do parallel I/O then redistribution for load balancing or
does it already exist?
Thanks,
On Tue, Apr 7, 2015 at 8:37 PM, Matthew Knepley <knepley at gmail.com> wrote:
> On Tue, Apr 7, 2015 at 7:18 PM, Justin Chang <jchang27 at uh.edu> wrote:
>
>> Hi all,
>>
>> So I am interpolating/uninterpolating my DMPlex mesh. This is the mesh
>> information for a cell-vertex mesh only:
>>
>> DM Object: 1 MPI processes
>> type: plex
>> DM_0x84000000_0 in 3 dimensions:
>> 0-cells: 159
>> 3-cells: 592
>> Labels:
>> marker: 1 strata of sizes (100)
>> depth: 2 strata of sizes (159, 592)
>>
>> I am interpolating the mesh with the following lines:
>>
>> ierr = DMPlexInterpolate(*dm, &idm);CHKERRQ(ierr);
>> ierr = DMPlexCopyCoordinates(*dm, idm);CHKERRQ(ierr);
>> ierr = DMPlexCopyLabels(*dm, idm);CHKERRQ(ierr);
>> ierr = DMPlexGetLabel(idm, "marker", &label);CHKERRQ(ierr);
>> ierr = DMPlexMarkBoundaryFaces(idm,label);CHKERRQ(ierr);
>> ierr = DMDestroy(dm);CHKERRQ(ierr);
>> *dm = idm;
>>
>> And the resulting mesh:
>>
>> DM Object: 1 MPI processes
>> type: plex
>> DM_0x84000000_1 in 3 dimensions:
>> 0-cells: 159
>> 1-cells: 845
>> 2-cells: 1280
>> 3-cells: 592
>> Labels:
>> marker: 1 strata of sizes (292)
>> depth: 4 strata of sizes (159, 845, 1280, 592)
>>
>> The reason I did this is so that incase I decide to refine the mesh with
>> "-dm_refine <number>" the DM will be sure to include the "marker" points in
>> the refinement process. Now, if I decide to uninterpolate the mesh with the
>> following lines:
>>
>> ierr = DMPlexUninterpolate(*dm, &idm);CHKERRQ(ierr);
>> ierr = DMPlexCopyCoordinates(*dm, idm);CHKERRQ(ierr);
>> ierr = DMPlexCopyLabels(*dm, idm);CHKERRQ(ierr);
>> ierr = DMDestroy(dm);CHKERRQ(ierr);
>> *dm = idm;
>>
>> I end up with this mesh:
>>
>> DM Object: 1 MPI processes
>> type: plex
>> DM_0x84000000_2 in 3 dimensions:
>> 0-cells: 159
>> 3-cells: 592
>> Labels:
>> marker: 1 strata of sizes (292)
>> depth: 2 strata of sizes (159, 592)
>>
>> The problem is that although the cells and vertices have gone back to the
>> original number, the "marker" label still includes face/edge points. This
>> gives me an error once I invoke DMCreateGobalVector(...).
>>
>> [0]PETSC ERROR: --------------------- Error Message
>> --------------------------------------------------------------
>> [0]PETSC ERROR: Argument out of range
>> [0]PETSC ERROR: Section point 755 should be in [0, 751)
>> [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
>> for trouble shooting.
>> [0]PETSC ERROR: Petsc Development GIT revision: v3.5.3-2528-gbee642f GIT
>> Date: 2015-03-29 20:36:38 -0500
>> [0]PETSC ERROR: ./cube_with_hole on a arch-linux2-c-debug named pacotaco
>> by justin Tue Apr 7 19:09:12 2015
>> [0]PETSC ERROR: Configure options --download-chaco --download-exodusii
>> --download-fblaslapack --download-hdf5 --download-metis --download-mpich
>> --download-mumps --download-netcdf --download-parmetis --download-scalapack
>> --download-triangle --with-cc=gcc --with-cmake=cmake --with-cxx=g++
>> --with-debugging=1 --with-fc=gfortran --with-valgrind=1
>> PETSC_ARCH=arch-linux2-c-debug
>> [0]PETSC ERROR: #1 PetscSectionGetDof() line 499 in
>> /home/justin/petsc-dev/src/vec/is/utils/vsectionis.c
>> [0]PETSC ERROR: #2 DMPlexGetConeSize() line 841 in
>> /home/justin/petsc-dev/src/dm/impls/plex/plex.c
>> [0]PETSC ERROR: #3 DMPlexGetTransitiveClosure() line 1342 in
>> /home/justin/petsc-dev/src/dm/impls/plex/plex.c
>> [0]PETSC ERROR: #4 DMPlexLabelComplete() line 88 in
>> /home/justin/petsc-dev/src/dm/impls/plex/plexsubmesh.c
>> [0]PETSC ERROR: #5 DMCreateDefaultSection_Plex() line 5605 in
>> /home/justin/petsc-dev/src/dm/impls/plex/plex.c
>> [0]PETSC ERROR: #6 DMGetDefaultSection() line 3035 in
>> /home/justin/petsc-dev/src/dm/interface/dm.c
>> [0]PETSC ERROR: #7 DMGetDefaultGlobalSection() line 3266 in
>> /home/justin/petsc-dev/src/dm/interface/dm.c
>> [0]PETSC ERROR: #8 DMCreateGlobalVector_Section_Private() line 13 in
>> /home/justin/petsc-dev/src/dm/interface/dmi.c
>> [0]PETSC ERROR: #9 DMCreateGlobalVector_Plex() line 1170 in
>> /home/justin/petsc-dev/src/dm/impls/plex/plexcreate.c
>> [0]PETSC ERROR: #10 DMCreateGlobalVector() line 698 in
>> /home/justin/petsc-dev/src/dm/interface/dm.c
>> [0]PETSC ERROR: #11 main() line 363 in
>> /home/justin/Dropbox/Research_Topics/Code_PETSc/Nonneg/cube_with_hole.c
>>
>>
>> From this error, it seems it's trying to access sieve points that non
>> longer exist. And I think it has to do with the label "marker" still
>> contains data from the interpolated mesh. Is there a way to "uninterpolate"
>> or remove such points? Or is there a better way of approaching this whole
>> thing?
>>
>
> 1) It would be easy to write a small function to throw out the points in
> the label that do not exist in the DM. You
> would just extract each stratumIS and then Clear each invalid point.
>
> 2) However, I do not understand the rationale for this above. You could
> just call MarkBoundaryFaces on the final mesh.
> If you are really trying to preserve a label across regular refinement
> AND you really do not want an interpolated mesh,
> then code up 1). I have yet to see a case where the extra memory for
> edges and faces makes a difference, but it may exist.
>
> Thanks,
>
> Matt
>
>
>> Thanks,
>>
>>
>>
>> --
>> Justin Chang
>> PhD Candidate, Civil Engineering - Computational Sciences
>> University of Houston, Department of Civil and Environmental Engineering
>> Houston, TX 77004
>> (512) 963-3262
>>
>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
--
Justin Chang
PhD Candidate, Civil Engineering - Computational Sciences
University of Houston, Department of Civil and Environmental Engineering
Houston, TX 77004
(512) 963-3262
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150409/ea331c22/attachment-0001.html>
-------------- next part --------------
========================
1 processors:
========================
Creating DMPlex... done
DM before interpolating:
DM Object: 1 MPI processes
type: plex
DM_0x7fa87381c0a0_0 in 3 dimensions:
0-cells: 159
3-cells: 592
Labels:
marker: 1 strata of sizes (100)
depth: 2 strata of sizes (159, 592)
DM after interpolation:
DM Object: 1 MPI processes
type: plex
DM_0x7fa87381c0a0_1 in 3 dimensions:
0-cells: 159
1-cells: 845
2-cells: 1280
3-cells: 592
Labels:
marker: 1 strata of sizes (580)
depth: 4 strata of sizes (159, 845, 1280, 592)
Distributing mesh... done
Refining mesh... done
DM Object:DM_0x7fa87381c0a0_1 1 MPI processes
type: plex
DM_0x7fa87381c0a0_1 in 3 dimensions:
0-cells: 416594
1-cells: 2866000
2-cells: 4874240
3-cells: 2424832
Labels:
marker: 1 strata of sizes (147460)
depth: 4 strata of sizes (416594, 2866000, 4874240, 2424832)
DM after uninterpolation:
DM Object: 1 MPI processes
type: plex
DM_0x7fa87381c0a0_2 in 3 dimensions:
0-cells: 416594
3-cells: 2424832
Labels:
marker: 1 strata of sizes (24580)
depth: 2 strata of sizes (416594, 2424832)
Setting up and initializing problem... done
========================
Solving non-negative...
========================
Tao converged with reason 2
========================
Time summary:
========================
Create DMPlex: 0.00444102s
Distribute and refine DMPlex: 8.95675s
Set up problem: 8.64039s
Solver: 30.9317s
Postprocess into VTU: 0.662271s
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./cube_with_hole on a arch-darwin-c-debug named CAML-03.CIVE.UH.EDU with 1 processor, by jychang48 Thu Apr 9 18:22:03 2015
Using Petsc Development GIT revision: v3.5.3-2528-gbee642f GIT Date: 2015-03-29 20:36:38 -0500
Max Max/Min Avg Total
Time (sec): 4.934e+01 1.00000 4.934e+01
Objects: 2.950e+02 1.00000 2.950e+02
Flops: 2.389e+10 1.00000 2.389e+10 2.389e+10
Flops/sec: 4.841e+08 1.00000 4.841e+08 4.841e+08
MPI Messages: 5.500e+00 1.00000 5.500e+00 5.500e+00
MPI Message Lengths: 1.411e+07 1.00000 2.566e+06 1.411e+07
MPI Reductions: 2.100e+01 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flops
and VecAXPY() for complex vectors of length N --> 8N flops
Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 4.9336e+01 100.0% 2.3886e+10 100.0% 5.500e+00 100.0% 2.566e+06 100.0% 2.100e+01 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flops: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flops in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flops --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
ThreadCommRunKer 494 1.0 5.0792e+00 1.0 6.67e+09 1.0 0.0e+00 0.0e+00 0.0e+00 10 28 0 0 0 10 28 0 0 0 1314
CreateMesh 1 1.0 8.9612e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 2.0e+01 18 0 0 0 95 18 0 0 0 95 0
VecView 1 1.0 2.3220e-02 1.0 6.47e+05 1.0 1.0e+00 3.1e+06 0.0e+00 0 0 18 22 0 0 0 18 22 0 28
VecDot 10740 1.0 4.3375e+00 1.0 8.42e+09 1.0 0.0e+00 0.0e+00 0.0e+00 9 35 0 0 0 9 35 0 0 0 1941
VecNorm 490 1.0 1.7819e-01 1.0 3.84e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 2156
VecScale 1955 1.0 5.8523e-01 1.0 7.66e+08 1.0 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 1 3 0 0 0 1310
VecCopy 6356 1.0 2.6006e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 5 0 0 0 0 5 0 0 0 0 0
VecSet 47 1.0 2.1615e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 8287 1.0 3.3792e+00 1.0 6.69e+09 1.0 0.0e+00 0.0e+00 0.0e+00 7 28 0 0 0 7 28 0 0 0 1979
VecAYPX 976 1.0 4.0512e-01 1.0 3.83e+08 1.0 0.0e+00 0.0e+00 0.0e+00 1 2 0 0 0 1 2 0 0 0 944
VecWAXPY 1 1.0 5.2404e-04 1.0 3.92e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 748
VecPointwiseMult 2929 1.0 1.6945e+00 1.0 1.15e+09 1.0 0.0e+00 0.0e+00 0.0e+00 3 5 0 0 0 3 5 0 0 0 678
MatMult 495 1.0 4.4913e+00 1.0 5.52e+09 1.0 0.0e+00 0.0e+00 0.0e+00 9 23 0 0 0 9 23 0 0 0 1229
MatAssemblyBegin 2 1.0 2.1458e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 2 1.0 1.4330e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatZeroEntries 1 1.0 4.7700e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DMPlexInterp 2 1.0 3.0179e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DMPlexStratify 11 1.0 2.0543e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 4 0 0 0 0 4 0 0 0 0 0
DMPlexPrealloc 1 1.0 3.2820e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 7 0 0 0 0 7 0 0 0 0 0
DMPlexResidualFE 1 1.0 1.7055e+00 1.0 1.63e+08 1.0 0.0e+00 0.0e+00 0.0e+00 3 1 0 0 0 3 1 0 0 0 96
DMPlexJacobianFE 1 1.0 4.0298e+00 1.0 6.47e+05 1.0 0.0e+00 0.0e+00 0.0e+00 8 0 0 0 0 8 0 0 0 0 0
SFSetGraph 1 1.0 7.3969e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastBegin 3 1.0 8.4453e-03 1.0 0.00e+00 0.0 4.5e+00 2.4e+06 0.0e+00 0 0 82 78 0 0 0 82 78 0 0
SFBcastEnd 3 1.0 3.5551e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFReduceBegin 1 1.0 1.4312e-03 1.0 0.00e+00 0.0 1.0e+00 3.1e+06 0.0e+00 0 0 18 22 0 0 0 18 22 0 0
SFReduceEnd 1 1.0 1.3900e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SNESFunctionEval 1 1.0 1.7262e+00 1.0 1.63e+08 1.0 3.5e+00 2.2e+06 0.0e+00 3 1 64 56 0 3 1 64 56 0 94
SNESJacobianEval 1 1.0 4.0329e+00 1.0 6.47e+05 1.0 1.0e+00 3.1e+06 0.0e+00 8 0 18 22 0 8 0 18 22 0 0
TaoSolve 1 1.0 2.5136e+01 1.0 2.35e+10 1.0 0.0e+00 0.0e+00 0.0e+00 51 98 0 0 0 51 98 0 0 0 935
TaoLineSearchApply 489 1.0 9.2746e+00 1.0 8.59e+09 1.0 0.0e+00 0.0e+00 0.0e+00 19 36 0 0 0 19 36 0 0 0 926
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Viewer 4 3 2312 0
Object 5 5 2980 0
Container 11 11 6380 0
Vector 42 42 626666400 0
Matrix 2 2 75555348 0
Distributed Mesh 23 23 107520 0
GraphPartitioner 11 11 6820 0
Star Forest Bipartite Graph 48 48 39336 0
Discrete System 23 23 19136 0
Index Set 59 59 50389936 0
IS L to G Mapping 1 1 1666972 0
Section 56 56 37856 0
SNES 1 1 1340 0
SNESLineSearch 1 1 880 0
DMSNES 1 1 680 0
Krylov Solver 1 1 1232 0
Preconditioner 1 1 864 0
Linear Space 1 1 656 0
Dual Space 1 1 672 0
FE Space 1 1 764 0
Tao 1 1 1720 0
TaoLineSearch 1 1 896 0
========================================================================================================================
Average time to get PetscTime(): 9.53674e-08
#PETSc Option Table entries:
-concentration 1,0
-dm_refine 4
-f_al 1e3
-f_at 1
-f_boundary cube_with_hole_test_bd.dat
-f_mesh cube_with_hole_test_mesh.dat
-f_out 1
-f_outname 3D_cube_orig.vtu
-ksp_rtol 1.0e-9
-ksp_type cg
-log_summary
-nonnegative 1
-pc_type jacobi
-petscpartitioner_type parmetis
-petscspace_order 1
-snes_atol 1.0e-6
-tao_type blmvm
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --download-chaco --download-exodusii --download-fblaslapack --download-hdf5 --download-metis --download-mumps --download-netcdf --download-openmpi --download-parmetis --download-scalapack --download-triangle --with-cc=gcc --with-cmake=cmake --with-cxx=g++ --with-debugging=0 --with-fc=gfortran --with-valgrind=1
-----------------------------------------
Libraries compiled on Mon Apr 6 20:58:57 2015 on CAML-03.CIVE.UH.EDU
Machine characteristics: Darwin-14.1.0-x86_64-i386-64bit
Using PETSc directory: /Users/jychang48/Software/petsc-dev
Using PETSc arch: arch-darwin-c-debug
-----------------------------------------
Using C compiler: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpicc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -O ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpif90 -fPIC -Wall -Wno-unused-variable -ffree-line-length-0 -Wno-unused-dummy-argument -O ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/include -I/Users/jychang48/Software/petsc-dev/include -I/Users/jychang48/Software/petsc-dev/include -I/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/include -I/opt/local/include -I/opt/X11/include
-----------------------------------------
Using C linker: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpicc
Using Fortran linker: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpif90
Using libraries: -Wl,-rpath,/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -L/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -lpetsc -Wl,-rpath,/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -L/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lscalapack -lexoIIv2for -lexodus -lflapack -lfblas -lparmetis -lnetcdf -ltriangle -lmetis -lssl -lcrypto -Wl,-rpath,/opt/local/lib -L/opt/local/lib -lhwloc -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -lchaco -Wl,-rpath,/opt/X11/lib -L/opt/X11/lib -lX11 -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.0/lib/darwin -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lgfortran -Wl,-rpath,/usr/local/Cellar/gcc/4.9.2/lib/gcc/x86_64-apple-darwin14.0.0/4.9.2 -L/usr/local/Cellar/gcc/4.9.2/lib/gcc/x86_64-apple-darwin14.0.0/4.9.2 -Wl,-rpath,/usr/local/Cellar/gcc/4.9.2/lib -L/usr/local/Cellar/gcc/4.9.2/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm -lclang_rt.osx -lmpi_cxx -lc++ -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -lclang_rt.osx -Wl,-rpath,/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -L/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -ldl -lmpi -lSystem -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -lclang_rt.osx -ldl
-----------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cube_with_hole.c
Type: text/x-csrc
Size: 21809 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150409/ea331c22/attachment-0001.c>
-------------- next part --------------
========================
1 processors:
========================
Creating DMPlex... done
DM before interpolating:
DM Object: 1 MPI processes
type: plex
DM_0x7fe27ae3d3b0_0 in 3 dimensions:
0-cells: 159
3-cells: 592
Labels:
marker: 1 strata of sizes (100)
depth: 2 strata of sizes (159, 592)
DM after interpolation:
DM Object: 1 MPI processes
type: plex
DM_0x7fe27ae3d3b0_1 in 3 dimensions:
0-cells: 159
1-cells: 845
2-cells: 1280
3-cells: 592
Labels:
marker: 1 strata of sizes (580)
depth: 4 strata of sizes (159, 845, 1280, 592)
Distributing mesh... done
Refining mesh... done
Setting up and initializing problem... done
========================
Solving non-negative...
========================
Tao converged with reason 2
========================
Time summary:
========================
Create DMPlex: 0.00416207s
Distribute and refine DMPlex: 4.0888s
Set up problem: 61.2977s
Solver: 42.2464s
Postprocess into VTU: 3.73179s
************************************************************************************************************************
*** WIDEN YOUR WINDOW TO 120 CHARACTERS. Use 'enscript -r -fCourier9' to print this document ***
************************************************************************************************************************
---------------------------------------------- PETSc Performance Summary: ----------------------------------------------
./cube_with_hole on a arch-darwin-c-debug named CAML-03.CIVE.UH.EDU with 1 processor, by jychang48 Thu Apr 9 18:26:21 2015
Using Petsc Development GIT revision: v3.5.3-2528-gbee642f GIT Date: 2015-03-29 20:36:38 -0500
Max Max/Min Avg Total
Time (sec): 1.116e+02 1.00000 1.116e+02
Objects: 2.760e+02 1.00000 2.760e+02
Flops: 2.389e+10 1.00000 2.389e+10 2.389e+10
Flops/sec: 2.142e+08 1.00000 2.142e+08 2.142e+08
MPI Messages: 5.500e+00 1.00000 5.500e+00 5.500e+00
MPI Message Lengths: 1.411e+07 1.00000 2.566e+06 1.411e+07
MPI Reductions: 1.100e+01 1.00000
Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
e.g., VecAXPY() for real vectors of length N --> 2N flops
and VecAXPY() for complex vectors of length N --> 8N flops
Summary of Stages: ----- Time ------ ----- Flops ----- --- Messages --- -- Message Lengths -- -- Reductions --
Avg %Total Avg %Total counts %Total Avg %Total counts %Total
0: Main Stage: 1.1157e+02 100.0% 2.3895e+10 100.0% 5.500e+00 100.0% 2.566e+06 100.0% 1.100e+01 100.0%
------------------------------------------------------------------------------------------------------------------------
See the 'Profiling' chapter of the users' manual for details on interpreting output.
Phase summary info:
Count: number of times phase was executed
Time and Flops: Max - maximum over all processors
Ratio - ratio of maximum to minimum over all processors
Mess: number of messages sent
Avg. len: average message length (bytes)
Reduct: number of global reductions
Global: entire computation
Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
%T - percent time in this phase %F - percent flops in this phase
%M - percent messages in this phase %L - percent message lengths in this phase
%R - percent reductions in this phase
Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
------------------------------------------------------------------------------------------------------------------------
Event Count Time (sec) Flops --- Global --- --- Stage --- Total
Max Ratio Max Ratio Max Ratio Mess Avg len Reduct %T %F %M %L %R %T %F %M %L %R Mflop/s
------------------------------------------------------------------------------------------------------------------------
--- Event Stage 0: Main Stage
ThreadCommRunKer 494 1.0 5.0596e+00 1.0 6.67e+09 1.0 0.0e+00 0.0e+00 0.0e+00 5 28 0 0 0 5 28 0 0 0 1319
CreateMesh 1 1.0 4.0930e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 1.0e+01 4 0 0 0 91 4 0 0 0 91 0
VecView 1 1.0 3.2374e-01 1.0 3.47e+06 1.0 1.0e+00 3.1e+06 0.0e+00 0 0 18 22 0 0 0 18 22 0 11
VecDot 10740 1.0 4.3332e+00 1.0 8.42e+09 1.0 0.0e+00 0.0e+00 0.0e+00 4 35 0 0 0 4 35 0 0 0 1943
VecNorm 490 1.0 1.7682e-01 1.0 3.84e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 2173
VecScale 1955 1.0 5.8404e-01 1.0 7.66e+08 1.0 0.0e+00 0.0e+00 0.0e+00 1 3 0 0 0 1 3 0 0 0 1312
VecCopy 6356 1.0 2.5955e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0
VecSet 46 1.0 2.1418e-01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
VecAXPY 8287 1.0 3.3791e+00 1.0 6.69e+09 1.0 0.0e+00 0.0e+00 0.0e+00 3 28 0 0 0 3 28 0 0 0 1979
VecAYPX 976 1.0 4.0522e-01 1.0 3.83e+08 1.0 0.0e+00 0.0e+00 0.0e+00 0 2 0 0 0 0 2 0 0 0 944
VecWAXPY 1 1.0 5.5599e-04 1.0 3.92e+05 1.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 705
VecPointwiseMult 2929 1.0 1.6930e+00 1.0 1.15e+09 1.0 0.0e+00 0.0e+00 0.0e+00 2 5 0 0 0 2 5 0 0 0 678
MatMult 495 1.0 4.4760e+00 1.0 5.52e+09 1.0 0.0e+00 0.0e+00 0.0e+00 4 23 0 0 0 4 23 0 0 0 1234
MatAssemblyBegin 2 1.0 2.1458e-06 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatAssemblyEnd 2 1.0 1.4720e-02 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
MatZeroEntries 1 1.0 4.8032e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DMPlexInterp 2 1.0 2.8389e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
DMPlexStratify 10 1.0 1.8760e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 2 0 0 0 0 2 0 0 0 0 0
DMPlexPrealloc 1 1.0 4.1800e+01 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 37 0 0 0 0 37 0 0 0 0 0
DMPlexResidualFE 1 1.0 8.6615e+00 1.0 1.66e+08 1.0 0.0e+00 0.0e+00 0.0e+00 8 1 0 0 0 8 1 0 0 0 19
DMPlexJacobianFE 1 1.0 8.3987e+00 1.0 3.47e+06 1.0 0.0e+00 0.0e+00 0.0e+00 8 0 0 0 0 8 0 0 0 0 0
SFSetGraph 1 1.0 6.8889e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFBcastBegin 3 1.0 8.0659e-03 1.0 0.00e+00 0.0 4.5e+00 2.4e+06 0.0e+00 0 0 82 78 0 0 0 82 78 0 0
SFBcastEnd 3 1.0 3.3667e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SFReduceBegin 1 1.0 1.4300e-03 1.0 0.00e+00 0.0 1.0e+00 3.1e+06 0.0e+00 0 0 18 22 0 0 0 18 22 0 0
SFReduceEnd 1 1.0 1.2019e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 0 0 0 0 0 0 0 0 0 0 0
SNESFunctionEval 1 1.0 8.7082e+00 1.0 1.66e+08 1.0 3.5e+00 2.2e+06 0.0e+00 8 1 64 56 0 8 1 64 56 0 19
SNESJacobianEval 1 1.0 8.4015e+00 1.0 3.47e+06 1.0 1.0e+00 3.1e+06 0.0e+00 8 0 18 22 0 8 0 18 22 0 0
TaoSolve 1 1.0 2.5100e+01 1.0 2.35e+10 1.0 0.0e+00 0.0e+00 0.0e+00 22 98 0 0 0 22 98 0 0 0 936
TaoLineSearchApply 489 1.0 9.2474e+00 1.0 8.59e+09 1.0 0.0e+00 0.0e+00 0.0e+00 8 36 0 0 0 8 36 0 0 0 928
------------------------------------------------------------------------------------------------------------------------
Memory usage is given in bytes:
Object Type Creations Destructions Memory Descendants' Mem.
Reports information only for process 0.
--- Event Stage 0: Main Stage
Viewer 4 3 2312 0
Object 5 5 2980 0
Container 11 11 6380 0
Vector 41 41 616666600 0
Matrix 2 2 75555348 0
Distributed Mesh 21 21 98176 0
GraphPartitioner 10 10 6200 0
Star Forest Bipartite Graph 44 44 36016 0
Discrete System 21 21 17472 0
Index Set 54 54 55120896 0
IS L to G Mapping 1 1 1666972 0
Section 52 52 35152 0
SNES 1 1 1340 0
SNESLineSearch 1 1 880 0
DMSNES 1 1 680 0
Krylov Solver 1 1 1232 0
Preconditioner 1 1 864 0
Linear Space 1 1 656 0
Dual Space 1 1 672 0
FE Space 1 1 764 0
Tao 1 1 1720 0
TaoLineSearch 1 1 896 0
========================================================================================================================
Average time to get PetscTime(): 0
#PETSc Option Table entries:
-concentration 1,0
-dm_refine 4
-f_al 1e3
-f_at 1
-f_boundary cube_with_hole_test_bd.dat
-f_mesh cube_with_hole_test_mesh.dat
-f_out 1
-f_outname 3D_cube_orig.vtu
-ksp_rtol 1.0e-9
-ksp_type cg
-log_summary
-nonnegative 1
-pc_type jacobi
-petscpartitioner_type parmetis
-petscspace_order 1
-snes_atol 1.0e-6
-tao_type blmvm
#End of PETSc Option Table entries
Compiled without FORTRAN kernels
Compiled with full precision matrices (default)
sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
Configure options: --download-chaco --download-exodusii --download-fblaslapack --download-hdf5 --download-metis --download-mumps --download-netcdf --download-openmpi --download-parmetis --download-scalapack --download-triangle --with-cc=gcc --with-cmake=cmake --with-cxx=g++ --with-debugging=0 --with-fc=gfortran --with-valgrind=1
-----------------------------------------
Libraries compiled on Mon Apr 6 20:58:57 2015 on CAML-03.CIVE.UH.EDU
Machine characteristics: Darwin-14.1.0-x86_64-i386-64bit
Using PETSc directory: /Users/jychang48/Software/petsc-dev
Using PETSc arch: arch-darwin-c-debug
-----------------------------------------
Using C compiler: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpicc -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -O ${COPTFLAGS} ${CFLAGS}
Using Fortran compiler: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpif90 -fPIC -Wall -Wno-unused-variable -ffree-line-length-0 -Wno-unused-dummy-argument -O ${FOPTFLAGS} ${FFLAGS}
-----------------------------------------
Using include paths: -I/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/include -I/Users/jychang48/Software/petsc-dev/include -I/Users/jychang48/Software/petsc-dev/include -I/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/include -I/opt/local/include -I/opt/X11/include
-----------------------------------------
Using C linker: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpicc
Using Fortran linker: /Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/bin/mpif90
Using libraries: -Wl,-rpath,/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -L/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -lpetsc -Wl,-rpath,/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -L/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lscalapack -lexoIIv2for -lexodus -lflapack -lfblas -lparmetis -lnetcdf -ltriangle -lmetis -lssl -lcrypto -Wl,-rpath,/opt/local/lib -L/opt/local/lib -lhwloc -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -lchaco -Wl,-rpath,/opt/X11/lib -L/opt/X11/lib -lX11 -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.0/lib/darwin -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lgfortran -Wl,-rpath,/usr/local/Cellar/gcc/4.9.2/lib/gcc/x86_64-apple-darwin14.0.0/4.9.2 -L/usr/local/Cellar/gcc/4.9.2/lib/gcc/x86_64-apple-darwin14.0.0/4.9.2 -Wl,-rpath,/usr/local/Cellar/gcc/4.9.2/lib -L/usr/local/Cellar/gcc/4.9.2/lib -lgfortran -lgcc_ext.10.5 -lquadmath -lm -lclang_rt.osx -lmpi_cxx -lc++ -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -lclang_rt.osx -Wl,-rpath,/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -L/Users/jychang48/Software/petsc-dev/arch-darwin-c-debug/lib -ldl -lmpi -lSystem -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin -lclang_rt.osx -ldl
-----------------------------------------
More information about the petsc-users
mailing list