<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Oct 16, 2014 at 3:02 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Actually, I have some related issues:<br><br></div>In a previous thread, I saw that to output as hdf5/mxf you basically need the following two runtime options:<br><br>-dm_view hdf5:sol.h5 -snes_view_solution hdf5:sol.h5::append<br></div></div></blockquote><div><br></div><div>Can you send the code and we will get it to output. Its easiest to have something to work from,</div><div>and we have not tried this out yet. You can send it to <a href="mailto:petsc-maint@mcs.anl.gov">petsc-maint@mcs.anl.gov</a> and its private.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>1) For the code that I had shown earlier in this thread (test.c), it works serially if I used the SNES solver instead of the TaoSolver. However, when I run it on multiple processors I get the following error:<br><br>HDF5-DIAG: Error detected in HDF5 (1.8.12) MPI-process 0:<br> #000: H5F.c line 2061 in H5Fclose(): decrementing file ID failed<br> major: Object atom<br> minor: Unable to close file<br> #001: H5I.c line 1479 in H5I_dec_app_ref(): can't decrement ID ref count<br> major: Object atom<br> minor: Unable to decrement reference count<br> #002: H5F.c line 1838 in H5F_close(): can't close file<br> major: File accessibilty<br> minor: Unable to close file<br> #003: H5F.c line 2000 in H5F_try_close(): problems closing file<br> major: File accessibilty<br> minor: Unable to close file<br> #004: H5F.c line 1145 in H5F_dest(): low level truncate failed<br> major: File accessibilty<br> minor: Write failed<br> #005: H5FD.c line 1897 in H5FD_truncate(): driver truncate request failed<br> major: Virtual File Layer<br> minor: Can't update object<br> #006: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): MPI_File_set_size failed<br> major: Internal error (too specific to document in detail)<br> minor: Some MPI function failed<br> #007: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): Invalid argument, error stack:<br>MPI_FILE_SET_SIZE(74): Inconsistent arguments to collective routine <br> major: Internal error (too specific to document in detail)<br> minor: MPI Error String<br>HDF5-DIAG: Error detected in HDF5 (1.8.12) MPI-process 1:<br> #000: H5F.c line 2061 in H5Fclose(): decrementing file ID failed<br> major: Object atom<br> minor: Unable to close file<br> #001: H5I.c line 1479 in H5I_dec_app_ref(): can't decrement ID ref count<br> major: Object atom<br> minor: Unable to decrement reference count<br> #002: H5F.c line 1838 in H5F_close(): can't close file<br> major: File accessibilty<br> minor: Unable to close file<br> #003: H5F.c line 2000 in H5F_try_close(): problems closing file<br> major: File accessibilty<br> minor: Unable to close file<br> #004: H5F.c line 1145 in H5F_dest(): low level truncate failed<br> major: File accessibilty<br> minor: Write failed<br> #005: H5FD.c line 1897 in H5FD_truncate(): driver truncate request failed<br> major: Virtual File Layer<br> minor: Can't update object<br> #006: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): MPI_File_set_size failed<br> major: Internal error (too specific to document in detail)<br> minor: Some MPI function failed<br> #007: H5FDmpio.c line 1989 in H5FD_mpio_truncate(): Invalid argument, error stack:<br>MPI_FILE_SET_SIZE(74): Inconsistent arguments to collective routine <br> major: Internal error (too specific to document in detail)<br> minor: MPI Error String<br><br><br></div><div>I compared my code with what was in SNES ex12 and I don't seem to understand why I am getting this error.<br><br></div><div>2) Is there a way to output as HDF5/xmf via the Tao Solver? I tried using -ksp_view_solution hdf5:sol.h5::append but it gave me no solution. The multiple processor issue also applies to this<br><br><br>Any help appreciated. Thanks<br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 14, 2014 at 11:58 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Hi Jason,<br><br></div>Both of those algorithms resolved my problem. Thanks!<span><font color="#888888"><br><br></font></span></div><span><font color="#888888">Justin<br></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Oct 14, 2014 at 10:12 PM, Jason Sarich <span dir="ltr"><<a href="mailto:jason.sarich@gmail.com" target="_blank">jason.sarich@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Justin,<div><br></div><div>If the only constraints you have are bounds on the variables, then you are much better off using the TAO algorithms tron or blmvm. IPM builds its own KKT matrix for a ksp solve that is undoubtably much less efficient than what will be used in tron (blmvm avoids ksp solves all together).</div><div><br></div><div>That begin said, it looks like there's a bug in IPM and I'll try to track it down, thanks for the report.</div><span><font color="#888888"><div><br></div><div>Jason</div><div><br></div></font></span></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Tue, Oct 14, 2014 at 9:34 PM, Justin Chang <span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><span>
<div dir="ltr">
<div>
<div>Attached is the source code and the makefile used to compile/run the code.<br>
<br>
</div>
The source code is basically a dumbed down version of SNES ex12 plus necessary TAO routines.<br>
<br>
</div>
Thanks<br>
</div></span><div><div>
<div class="gmail_extra"><br>
<div class="gmail_quote"><span>On Tue, Oct 14, 2014 at 8:39 PM, Matthew Knepley <span dir="ltr">
<<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>></span> wrote:<br>
</span><div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"><span>On Tue, Oct 14, 2014 at 8:04 PM, Justin Chang
<span dir="ltr"><<a href="mailto:jychang48@gmail.com" target="_blank">jychang48@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>Hi all,<br>
<br>
</div>
So I am writing a non-negative diffusion equation using DMPlex's FEM and Tao's SetVariableBounds functions. My code works really perfectly when I run it with one processor. However, once I use 2 or more processors, I get this error:<br>
</div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>It looks like the problem is in the TAO definition, but you can check by just solving your problem with, for instance, BJacobi-LU in parallel.</div>
<div>
<div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>
[0]PETSC ERROR: Nonconforming object sizes<br>
[0]PETSC ERROR: Vector wrong size 89 for scatter 88 (scatter reverse and vector to != ctx from size)<br>
[1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>
[1]PETSC ERROR: Nonconforming object sizes<br>
[1]PETSC ERROR: Vector wrong size 87 for scatter 88 (scatter reverse and vector to != ctx from size)<br>
[1]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">
http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
[1]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT Date: 2014-10-04 20:10:35 -0500<br>
[1]PETSC ERROR: [0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">
http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<br>
[0]PETSC ERROR: Petsc Development GIT revision: v3.5.2-526-gfaecc80 GIT Date: 2014-10-04 20:10:35 -0500<br>
[0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by justin Tue Oct 14 19:48:50 2014<br>
[0]PETSC ERROR: ./bin/diff2D on a arch-linux2-c-debug named pacotaco by justin Tue Oct 14 19:48:50 2014<br>
[1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-fblaslapack --download-mpich --with-debugging=1 --download-metis --download-parmetis --download-triangle --with-cmake=cmake --download-ctetgen --download-superlu --download-scalapack
--download-mumps --download-hdf5 --with-valgrind=1 -with-cmake=cmake<br>
[1]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-fblaslapack --download-mpich --with-debugging=1 --download-metis --download-parmetis --download-triangle --with-cmake=cmake --download-ctetgen --download-superlu --download-scalapack
--download-mumps --download-hdf5 --with-valgrind=1 -with-cmake=cmake<br>
[0]PETSC ERROR: #1 VecScatterBegin() line 1713 in /home/justin/petsc-master/src/vec/vec/utils/vscat.c<br>
#1 VecScatterBegin() line 1713 in /home/justin/petsc-master/src/vec/vec/utils/vscat.c<br>
[1]PETSC ERROR: [0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c<br>
[0]PETSC ERROR: #2 MatMultTranspose_MPIAIJ() line 1010 in /home/justin/petsc-master/src/mat/impls/aij/mpi/mpiaij.c<br>
[1]PETSC ERROR: #3 MatMultTranspose() line 2242 in /home/justin/petsc-master/src/mat/interface/matrix.c<br>
#3 MatMultTranspose() line 2242 in /home/justin/petsc-master/src/mat/interface/matrix.c<br>
[0]PETSC ERROR: #4 IPMComputeKKT() line 616 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c<br>
[1]PETSC ERROR: #4 IPMComputeKKT() line 616 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c<br>
[1]PETSC ERROR: [0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c<br>
[0]PETSC ERROR: #5 TaoSolve_IPM() line 50 in /home/justin/petsc-master/src/tao/constrained/impls/ipm/ipm.c<br>
[1]PETSC ERROR: #6 TaoSolve() line 190 in /home/justin/petsc-master/src/tao/interface/taosolver.c<br>
#6 TaoSolve() line 190 in /home/justin/petsc-master/src/tao/interface/taosolver.c<br>
[0]PETSC ERROR: #7 main() line 341 in /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c<br>
[1]PETSC ERROR: #7 main() line 341 in /home/justin/Dropbox/Research_Topics/Petsc_Nonneg_diffusion/src/diff2D.c<br>
[1]PETSC ERROR: [0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------<br>
----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------<br>
application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0<br>
application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1<br>
[cli_1]: aborting job:<br>
application called MPI_Abort(MPI_COMM_WORLD, 60) - process 1<br>
[cli_0]: aborting job:<br>
application called MPI_Abort(MPI_COMM_WORLD, 60) - process 0<br>
<br>
<br>
</div>
<div>I have no idea how or why I am getting this error. What does this mean?<br>
</div>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
<div>It looks like one dof is given to proc 0 which should live on proc 1. We have to look at the divisions</div>
<div>made in the KKT solver. Can you send a small example?</div>
<div><br>
</div>
<div> Matt</div>
<span>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div></div>
My code is essentially built off of SNES ex12.c. The Jacobian matrix, residual vector, and solution vector were created using DMPlex and the built-in FEM functions within. The Hessian matrix and gradient vector were created by simple matmult() functions of
the jacobian and residual. The lower bounds vector was created by duplicating the solution vector (initial guess set to zero). My FormFunctionGradient() is basically the same thing as in the maros.c example. can give more information if needed.<br>
<br>
Thanks,<br>
Justin<br>
</div>
</blockquote>
</span></div>
<span><font color="#888888"><br>
<br clear="all"><span class="HOEnZb"><font color="#888888">
<div><br>
</div>
-- <br>
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener </font></span></font></span></div><span class="HOEnZb"><font color="#888888">
</font></span></div><span class="HOEnZb"><font color="#888888">
</font></span></blockquote><span class="HOEnZb"><font color="#888888">
</font></span></div></div></div><span class="HOEnZb"><font color="#888888">
<br>
</font></span></div><span class="HOEnZb"><font color="#888888">
</font></span></div></div></div><span class="HOEnZb"><font color="#888888">
</font></span></blockquote></div><span class="HOEnZb"><font color="#888888"><br></font></span></div><span class="HOEnZb"><font color="#888888">
</font></span></blockquote></div><span class="HOEnZb"><font color="#888888"><br></font></span></div>
</div></div></blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener
</div></div>