<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <br>
    On 18/6/2012 9:52 PM, Matthew Knepley wrote:
    <blockquote
cite="mid:CAMYG4GkstZNkbP7smud0kbV7fnWDUbGop1myKp2C=cz4xxcehw@mail.gmail.com"
      type="cite">On Mon, Jun 18, 2012 at 7:30 PM, TAY wee-beng <span
        dir="ltr"><<a moz-do-not-send="true"
          href="mailto:zonexo@gmail.com" target="_blank">zonexo@gmail.com</a>></span>
      wrote:<br>
      <div class="gmail_quote">
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div bgcolor="#FFFFFF" text="#000000"> <br>
            On 17/6/2012 2:33 PM, Jed Brown wrote:
            <blockquote type="cite">
              <div class="gmail_quote">On Sun, Jun 17, 2012 at 7:26 AM,
                TAY wee-beng <span dir="ltr"><<a
                    moz-do-not-send="true"
                    href="mailto:zonexo@gmail.com" target="_blank">zonexo@gmail.com</a>></span>
                wrote:<br>
                <blockquote class="gmail_quote" style="margin:0 0 0
                  .8ex;border-left:1px #ccc solid;padding-left:1ex">
                  <div bgcolor="#FFFFFF" text="#000000">
                    <div> <br>
                      On 16/6/2012 9:24 AM, Jed Brown wrote:
                      <blockquote type="cite">
                        <p>It depends how you want to solve the problem.
                          I usually group all dofs together. There is a
                          2D Stokes+Thermodynamics example in SNES ex30
                          (or 31?).</p>
                      </blockquote>
                      <br>
                    </div>
                    I tried to understand ex30. I have some questions
                    since it's in C and I'm using to Fortran
                    programming.<br>
                  </div>
                </blockquote>
                <div><br>
                </div>
                <div>This looks about right, see
                  src/dm/examples/tutorials/ex11f90.F.</div>
              </div>
            </blockquote>
            <br>
            I tried to build and run ex11f90 in linux. However, it's not
            working:<br>
            <br>
            I have attached the run and valgrind output:<br>
          </div>
        </blockquote>
        <div><br>
        </div>
        <div>It appears to be a problem with your MPI.</div>
        <div><br>
        </div>
        <div>   Matt</div>
      </div>
    </blockquote>
    <br>
    Is there anyway to solve this problem? I'm running on a cluster
    using openmpi 1.5.3 and I've no admin rights.<br>
    <blockquote
cite="mid:CAMYG4GkstZNkbP7smud0kbV7fnWDUbGop1myKp2C=cz4xxcehw@mail.gmail.com"
      type="cite">
      <div class="gmail_quote">
        <div> </div>
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">
          <div bgcolor="#FFFFFF" text="#000000"> run output:<br>
            <i><br>
              [wtay@hpc12:tutorials]$ ./ex11f90 <br>
              [0]PETSC ERROR:
              ------------------------------------------------------------------------<br>
              [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation
              Violation, probably memory access out of range<br>
              [0]PETSC ERROR: Try option -start_in_debugger or
              -on_error_attach_debugger<br>
              [0]PETSC ERROR: or see <a moz-do-not-send="true"
                href="http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind"
                target="_blank">http://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind</a>[0]PETSC

              ERROR: or try <a moz-do-not-send="true"
                href="http://valgrind.org" target="_blank">http://valgrind.org</a>
              on GNU/linux and Apple Mac OS X to find memory corruption
              errors<br>
              [0]PETSC ERROR: likely location of problem given in stack
              below<br>
              [0]PETSC ERROR: ---------------------  Stack Frames
              ------------------------------------<br>
              [0]PETSC ERROR: Note: The EXACT line numbers in the stack
              are not available,<br>
              [0]PETSC ERROR:       INSTEAD the line number of the start
              of the function<br>
              [0]PETSC ERROR:       is given.<br>
              [0]PETSC ERROR: --------------------- Error Message
              ------------------------------------<br>
              [0]PETSC ERROR: Signal received!<br>
              [0]PETSC ERROR:
              ------------------------------------------------------------------------<br>
              [0]PETSC ERROR: Petsc Development HG revision:
              f3b998b41b349e16d47fe42b0e223d3462737e05  HG Date: Fri Jun
              15 17:50:32 2012 -0500<br>
              [0]PETSC ERROR: See docs/changes/index.html for recent
              updates.<br>
              [0]PETSC ERROR: See docs/faq.html for hints about trouble
              shooting.<br>
              [0]PETSC ERROR: See docs/index.html for manual pages.<br>
              [0]PETSC ERROR:
              ------------------------------------------------------------------------<br>
              [0]PETSC ERROR: ./ex11f90 on a petsc-3.3 named hpc12 by
              wtay Tue Jun 19 03:22:52 2012<br>
              [0]PETSC ERROR: Libraries linked from
              /home/wtay/Lib/petsc-3.3-dev_shared_debug/lib<br>
              [0]PETSC ERROR: Configure run at Sun Jun 17 16:51:29 2012<br>
              [0]PETSC ERROR: Configure options
              --with-mpi-dir=/opt/openmpi-1.5.3/
              --with-blas-lapack-dir=/opt/intelcpro-11.1.059/mkl/lib/em64t/
              --with-debugging=1 --download-hypre=1
              --prefix=/home/wtay/Lib/petsc-3.3-dev_shared_debug
              --known-mpi-shared=1 --with-shared-libraries<br>
              [0]PETSC ERROR:
              ------------------------------------------------------------------------<br>
              [0]PETSC ERROR: User provided function() line 0 in unknown
              directory unknown file<br>
--------------------------------------------------------------------------<br>
              MPI_ABORT was invoked on rank 0 in communicator
              MPI_COMM_WORLD <br>
              with errorcode 59.<br>
              <br>
              NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI
              processes.<br>
              You may or may not see output from other processes,
              depending on<br>
              exactly when Open MPI kills them.<br>
--------------------------------------------------------------------------</i><br>
            <br>
            valgrind output:<br>
            <i><br>
              ==6027== Memcheck, a memory error detector<br>
              ==6027== Copyright (C) 2002-2011, and GNU GPL'd, by Julian
              Seward et al.<br>
              ==6027== Using Valgrind-3.7.0 and LibVEX; rerun with -h
              for copyright info<br>
              ==6027== Command: ex11f90<br>
              ==6027== <br>
              ==6027== Invalid read of size 8<br>
              ==6027==    at 0xA60148D: _wordcopy_fwd_dest_aligned (in
              /lib64/<a moz-do-not-send="true"
                href="http://libc-2.12.so" target="_blank">libc-2.12.so</a>)<br>
              ==6027==    by 0xA5FB11D: __GI_memmove (in /lib64/<a
                moz-do-not-send="true" href="http://libc-2.12.so"
                target="_blank">libc-2.12.so</a>)<br>
              ==6027==    by 0xA6027DB: argz_insert (in /lib64/<a
                moz-do-not-send="true" href="http://libc-2.12.so"
                target="_blank">libc-2.12.so</a>)<br>
              ==6027==    by 0x95CAF25: lt_argz_insert (ltdl.c:1679)<br>
              ==6027==    by 0x95CB7D0: foreachfile_callback
              (ltdl.c:1718)<br>
              ==6027==    by 0x95CB4F1: foreach_dirinpath (ltdl.c:710)<br>
              ==6027==    by 0x95CB580: lt_dlforeachfile (ltdl.c:1865)<br>
              ==6027==    by 0x95DB999: mca_base_component_find
              (mca_base_component_find.c:301)<br>
              ==6027==    by 0x95DC4B0: mca_base_components_open
              (mca_base_components_open.c:128)<br>
              ==6027==    by 0x95F7CE7: opal_paffinity_base_open
              (paffinity_base_open.c:112)<br>
              ==6027==    by 0x95C39FE: opal_init (opal_init.c:307)<br>
              ==6027==    by 0x95815A4: orte_init (orte_init.c:78)<br>
              ==6027==  Address 0xb39d9a8 is 40 bytes inside a block of
              size 43 alloc'd<br>
              ==6027==    at 0x4C267BA: malloc (vg_replace_malloc.c:263)<br>
              ==6027==    by 0x95CA358: lt__malloc (lt__alloc.c:54)<br>
              ==6027==    by 0x95CB751: foreachfile_callback
              (ltdl.c:1764)<br>
              ==6027==    by 0x95CB4F1: foreach_dirinpath (ltdl.c:710)<br>
              ==6027==    by 0x95CB580: lt_dlforeachfile (ltdl.c:1865)<br>
              ==6027==    by 0x95DB999: mca_base_component_find
              (mca_base_component_find.c:301)<br>
              ==6027==    by 0x95DC4B0: mca_base_components_open
              (mca_base_components_open.c:128)<br>
              ==6027==    by 0x95F7CE7: opal_paffinity_base_open
              (paffinity_base_open.c:112)<br>
              ==6027==    by 0x95C39FE: opal_init (opal_init.c:307)<br>
              ==6027==    by 0x95815A4: orte_init (orte_init.c:78)<br>
              ==6027==    by 0x95432AE: ompi_mpi_init
              (ompi_mpi_init.c:350)<br>
              ==6027==    by 0x955938F: PMPI_Init (pinit.c:84)<br>
              ==6027== <br>
              ==6027== Syscall param writev(vector[...]) points to
              uninitialised byte(s)<br>
              ==6027==    at 0xA65692B: writev (in /lib64/<a
                moz-do-not-send="true" href="http://libc-2.12.so"
                target="_blank">libc-2.12.so</a>)<br>
              ==6027==    by 0xC9A2C56: mca_oob_tcp_msg_send_handler
              (oob_tcp_msg.c:249)<br>
              ==6027==    by 0xC9A417C: mca_oob_tcp_peer_send
              (oob_tcp_peer.c:204)<br>
              ==6027==    by 0xC9A67FC: mca_oob_tcp_send_nb
              (oob_tcp_send.c:167)<br>
              ==6027==    by 0xC3953B5: orte_rml_oob_send
              (rml_oob_send.c:136)<br>
              ==6027==    by 0xC3955FF: orte_rml_oob_send_buffer
              (rml_oob_send.c:270)<br>
              ==6027==    by 0xCDB1E87: modex (grpcomm_bad_module.c:573)<br>
              ==6027==    by 0x95436F1: ompi_mpi_init
              (ompi_mpi_init.c:682)<br>
              ==6027==    by 0x955938F: PMPI_Init (pinit.c:84)<br>
              ==6027==    by 0x8AB4FF4: MPI_INIT (pinit_f.c:75)<br>
              ==6027==    by 0x50ED97E: petscinitialize_ (zstart.c:299)<br>
              ==6027==    by 0x40881C: MAIN__ (ex11f90.F:43)<br>
              ==6027==  Address 0xed30cd1 is 161 bytes inside a block of
              size 256 alloc'd<br>
              ==6027==    at 0x4C268B2: realloc
              (vg_replace_malloc.c:632)<br>
              ==6027==    by 0x95C4F22: opal_dss_buffer_extend
              (dss_internal_functions.c:63)<br>
              ==6027==    by 0x95C5A64: opal_dss_copy_payload
              (dss_load_unload.c:164)<br>
              ==6027==    by 0x95A1246:
              orte_grpcomm_base_pack_modex_entries
              (grpcomm_base_modex.c:861)<br>
              ==6027==    by 0xCDB1E3C: modex (grpcomm_bad_module.c:563)<br>
              ==6027==    by 0x95436F1: ompi_mpi_init
              (ompi_mpi_init.c:682)<br>
              ==6027==    by 0x955938F: PMPI_Init (pinit.c:84)<br>
              ==6027==    by 0x8AB4FF4: MPI_INIT (pinit_f.c:75)<br>
              ==6027==    by 0x50ED97E: petscinitialize_ (zstart.c:299)<br>
              ==6027==    by 0x40881C: MAIN__ (ex11f90.F:43)<br>
              ==6027==    by 0x4087AB: main (in
              /home/wtay/Codes/petsc-dev/src/dm/examples/tutorials/ex11f90)<br>
              ==6027== <br>
              ==6027== Invalid read of size 8<br>
              ==6027==    at 0x50FC6D8: vecview_ (zvectorf.c:56)<br>
              ==6027==    by 0x408A05: MAIN__ (ex11f90.F:56)<br>
              ==6027==    by 0xEE1CC2F: ???<br>
              ==6027==    by 0xEE5A0AF: ???<br>
              ==6027==    by 0x6F5C9F: ??? (in
              /home/wtay/Codes/petsc-dev/src/dm/examples/tutorials/ex11f90)<br>
              ==6027==    by 0x4962FF: ??? (in
              /home/wtay/Codes/petsc-dev/src/dm/examples/tutorials/ex11f90)<br>
              ==6027==    by 0x6F5C9F: ??? (in
              /home/wtay/Codes/petsc-dev/src/dm/examples/tutorials/ex11f90)<br>
              ==6027==    by 0x7FEFFFC13: ???<br>
              ==6027==  Address 0xfffffffffffffeb8 is not stack'd,
              malloc'd or (recently) free'd<br>
              ==6027== </i><br>
            <br>
            <br>
             <br>
            <blockquote type="cite">
              <div class="gmail_quote">
                <div> </div>
                <blockquote class="gmail_quote" style="margin:0 0 0
                  .8ex;border-left:1px #ccc solid;padding-left:1ex">
                  <div bgcolor="#FFFFFF" text="#000000"> <br>
                    Supposed I have a field - u,v,w,p, so in order to
                    use them, I do the following:<br>
                    <i><br>
                      type field<br>
                      <br>
                      real u,v,w,p<br>
                      <br>
                      end type field<br>
                      <br>
                      type(field), pointer :: field1(:,:,:)    ->
                      make a derived variable</i><br>
                    <br>
                    Also:<br>
                    <i><br>
                      Vec field_local,field_global<br>
                      <br>
                      call DMDACreate3d with dof = 4<br>
                      <br>
                      call DMCreateGlobalVector(da, field_local,ierr)<br>
                      <br>
                      call DMCreateLocalVector(da,field_global,ierr)<br>
                      <br>
                      call DMGetLocalVector(da, field_local,ierr)   
                      ->    To insert values<br>
                      <br>
                      call DMDAVecGetArrayF90(da,
                      field_local,field1,ierr)<br>
                      <br>
                      do k = zs, zs + zm - 1<br>
                      <br>
                      do j = ys, ys + ym -1 <br>
                      <br>
                          do i = xs, xs + xm - 1<br>
                      <br>
                              field1(i,j,k)%u = ...     ->   
                      evaluate u,v,w,p etc<br>
                      <br>
                          end do<br>
                      <br>
                      end do<br>
                      <br>
                      call
                      DMDAVecRestoreArrayF90(da,field_local,field1,ierr)<br>
                      <br>
                      call
                      DMLocalToGlobalBegin(da,field_local,INSERT_VALUES,field_global,ierr)<br>
                      <br>
                      call
                      DMLocalToGlobalEnd(da,field_local,INSERT_VALUES,field_global,ierr)<br>
                      <br>
                      call DMRestoreLocalVector(da,field_local,ierr)</i><br>
                    <br>
                    Is this the correct way?<br>
                    <br>
                    Also, supposed I now want to solve my u,v,w momentum
                    eqns. Although they're not coupled together, I
                    believe it's faster if I assemble them into 1 big
                    matrix. <br>
                    <br>
                    So for Ax = b, x =
                    (field(1,1,1)%u,field(1,1,1)%v,field(1,1,1)%w,field(2,1,1)%u....
                    ) <br>
                    <br>
                    For b, do I duplicate a Vec similar to field_local?<br>
                    <br>
                    What about matrix A? Do I use the
                    MatSetValuesStencil?<br>
                  </div>
                </blockquote>
                <div><br>
                </div>
                <div>Yes</div>
                <div> </div>
                <blockquote class="gmail_quote" style="margin:0 0 0
                  .8ex;border-left:1px #ccc solid;padding-left:1ex">
                  <div bgcolor="#FFFFFF" text="#000000"> <br>
                    Lastly, the type field contains u,v,w and p.
                    However, I'm only solving u,v,w. Do I have to skip
                    some values or use identity matrix to solve it?<br>
                  </div>
                </blockquote>
                <div><br>
                </div>
                <div>Why not make a field that contains only u,v,w. I
                  don't see what you're trying to do.</div>
              </div>
            </blockquote>
          </div>
        </blockquote>
      </div>
      <br>
      <br clear="all">
      <div><br>
      </div>
      -- <br>
      What most experimenters take for granted before they begin their
      experiments is infinitely more interesting than any results to
      which their experiments lead.<br>
      -- Norbert Wiener<br>
    </blockquote>
  </body>
</html>