<div dir="ltr">Hello devs,<div><br></div><div>Ive been creating the framework to distribute the arrays from my model using the global/local vectors and the DMDAs, for this i created a small subroutine that is responsible for making the necessary calls:</div><div><br></div><pre style="margin-top:0px;margin-bottom:0px;padding:0px;font-family:consolas,menlo,"liberation mono",courier,monospace;font-size:12px;line-height:1.4;color:rgb(51,51,51)"> <span class="gmail-k" style="font-weight:700">SUBROUTINE </span><span class="gmail-n">DAs</span><span class="gmail-p">(</span><span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">globalv</span><span class="gmail-p">,</span><span class="gmail-n">localv</span><span class="gmail-p">)</span>
<a name="PetscObjs.f90-394" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-395" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! use PetscObjects, only :: ierr</span>
<a name="PetscObjs.f90-396" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-397" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-398" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! Umbrella program to update and communicate the arrays in a</span>
<a name="PetscObjs.f90-399" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! distributed fashion using the DMDA objects from PETSc.</span>
<a name="PetscObjs.f90-400" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! Manuel Valera 1/20/17</span>
<a name="PetscObjs.f90-401" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-402" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! Arguments:</span>
<a name="PetscObjs.f90-403" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! da = DMDA array either 1d,2d or 3d, already created and setup</span>
<a name="PetscObjs.f90-404" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! globalv = global vector to be operated into</span>
<a name="PetscObjs.f90-405" style="color:rgb(53,114,176)"></a> <span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! localv = local chunk each processor works in.</span>
<a name="PetscObjs.f90-406" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-408" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-409" style="color:rgb(53,114,176)"></a> <span class="gmail-n">Vec</span><span class="gmail-p">,</span><span class="gmail-k" style="font-weight:700">intent</span><span class="gmail-p">(</span><span class="gmail-n">inout</span><span class="gmail-p">)</span> <span class="gmail-kd" style="font-weight:700">::</span> <span class="gmail-n">globalv</span><span class="gmail-p">,</span><span class="gmail-n">localv</span>
<a name="PetscObjs.f90-410" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-411" style="color:rgb(53,114,176)"></a> <span class="gmail-n">DM</span><span class="gmail-p">,</span><span class="gmail-k" style="font-weight:700">intent</span><span class="gmail-p">(</span><span class="gmail-n">inout</span><span class="gmail-p">)</span> <span class="gmail-kd" style="font-weight:700">::</span> <span class="gmail-n">da</span>
<a name="PetscObjs.f90-412" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-413" style="color:rgb(53,114,176)"></a> <span class="gmail-k" style="font-weight:700">call </span><span class="gmail-n">DMCreateGlobalVector</span><span class="gmail-p">(</span><span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">globalv</span><span class="gmail-p">,</span><span class="gmail-n">ierr</span><span class="gmail-p">)</span>
<a name="PetscObjs.f90-414" style="color:rgb(53,114,176)"></a> <span class="gmail-k" style="font-weight:700">call </span><span class="gmail-n">DMCreateLocalVector</span><span class="gmail-p">(</span><span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">localv</span><span class="gmail-p">,</span><span class="gmail-n">ierr</span><span class="gmail-p">)</span>
<a name="PetscObjs.f90-415" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-416" style="color:rgb(53,114,176)"></a> <span class="gmail-k" style="font-weight:700">call </span><span class="gmail-n">DMGlobalToLocalBegin</span><span class="gmail-p">(</span><span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">globalv</span><span class="gmail-p">,</span><span class="gmail-n">INSERT_VALUES</span><span class="gmail-p">,</span><span class="gmail-n">localv</span><span class="gmail-p">,</span><span class="gmail-n">ierr</span><span class="gmail-p">)</span>
<a name="PetscObjs.f90-417" style="color:rgb(53,114,176)"></a> <span class="gmail-k" style="font-weight:700">call </span><span class="gmail-n">DMGlobalToLocalEnd</span><span class="gmail-p">(</span><span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">globalv</span><span class="gmail-p">,</span><span class="gmail-n">INSERT_VALUES</span><span class="gmail-p">,</span><span class="gmail-n">localv</span><span class="gmail-p">,</span><span class="gmail-n">ierr</span><span class="gmail-p">)</span>
<a name="PetscObjs.f90-418" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-419" style="color:rgb(53,114,176)"></a>
<a name="PetscObjs.f90-420" style="color:rgb(53,114,176)"></a> </pre><div><span style="color:rgb(51,51,51);font-family:consolas,menlo,"liberation mono",courier,monospace;font-size:12px"> </span><span class="gmail-k" style="color:rgb(51,51,51);font-family:consolas,menlo,"liberation mono",courier,monospace;font-size:12px;font-weight:700">END SUBROUTINE </span><span class="gmail-n" style="color:rgb(51,51,51);font-family:consolas,menlo,"liberation mono",courier,monospace;font-size:12px">DAs</span> </div><div><br></div><div><br></div><div>I call this from another program where i have declared a vector as:</div><div><br></div><div><pre style="margin-top:0px;margin-bottom:0px;padding:0px;font-family:consolas,menlo,"liberation mono",courier,monospace;font-size:12px;line-height:1.4;color:rgb(51,51,51)"><span class="gmail-cp" style="font-weight:700;color:rgb(153,153,153)">#include "petsc/finclude/petscdm.h"</span>
<a name="GridP.f90-3" style="color:rgb(53,114,176)"></a><span class="gmail-cp" style="font-weight:700;color:rgb(153,153,153)">#include "petsc/finclude/petscdmda.h"</span>
<a name="GridP.f90-4" style="color:rgb(53,114,176)"></a>
<a name="GridP.f90-5" style="color:rgb(53,114,176)"></a><span class="gmail-k" style="font-weight:700">use </span><span class="gmail-n">petsc</span>
<a name="GridP.f90-6" style="color:rgb(53,114,176)"></a><span class="gmail-k" style="font-weight:700">use </span><span class="gmail-n">petscdm</span>
<a name="GridP.f90-7" style="color:rgb(53,114,176)"></a><span class="gmail-k" style="font-weight:700">use </span><span class="gmail-n">petscdmda</span>
</pre></div><div><span class="gmail-n"><br></span></div><div><span class="gmail-n"><pre style="margin-top:0px;margin-bottom:0px;padding:0px;font-family:consolas,menlo,"liberation mono",courier,monospace;font-size:12px;line-height:1.4;color:rgb(51,51,51)"><span class="gmail-k" style="font-weight:700">IMPLICIT NONE</span></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;font-family:consolas,menlo,"liberation mono",courier,monospace;font-size:12px;line-height:1.4;color:rgb(51,51,51)"><span class="gmail-k" style="font-weight:700"><br></span></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-k"><pre style="color:rgb(51,51,51);font-size:12px;font-weight:normal;font-family:consolas,menlo,"liberation mono",courier,monospace;margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-n">Vec</span> <span class="gmail-kd" style="font-weight:700">::</span> <span class="gmail-n">xpdmda</span><span class="gmail-p">,</span><span class="gmail-n">yp</span><span class="gmail-p">,</span><span class="gmail-n">zp</span><span class="gmail-p">,</span><span class="gmail-n">xcp</span><span class="gmail-p">,</span><span class="gmail-n">ycp</span><span class="gmail-p">,</span><span class="gmail-n">zcp</span></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><pre style="color:rgb(51,51,51);font-size:12px;font-weight:normal;font-family:consolas,menlo,"liberation mono",courier,monospace;margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-n">DM</span> <span class="gmail-kd" style="font-weight:700">::</span> <span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">dac</span><span class="gmail-p">,</span><span class="gmail-n">dau</span><span class="gmail-p">,</span><span class="gmail-n">dav</span><span class="gmail-p">,</span><span class="gmail-n">daw</span></pre><pre style="color:rgb(51,51,51);font-size:12px;font-weight:normal;font-family:consolas,menlo,"liberation mono",courier,monospace;margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-n"><br></span></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-n"><pre style="color:rgb(51,51,51);font-size:12px;font-weight:normal;font-family:consolas,menlo,"liberation mono",courier,monospace;margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-k" style="font-weight:700">CONTAINS</span>
<a name="GridP.f90-45" style="color:rgb(53,114,176)"></a><span class="gmail-k" style="font-weight:700">SUBROUTINE </span><span class="gmail-n">InitGridP</span>
<a name="GridP.f90-46" style="color:rgb(53,114,176)"></a>
<a name="GridP.f90-47" style="color:rgb(53,114,176)"></a><span class="gmail-k" style="font-weight:700">IMPLICIT NONE</span>
<a name="GridP.f90-48" style="color:rgb(53,114,176)"></a><span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">!PetscInt :: ierr</span>
<a name="GridP.f90-49" style="color:rgb(53,114,176)"></a>
<a name="GridP.f90-50" style="color:rgb(53,114,176)"></a><span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">! CREATE DMDA:</span>
<a name="GridP.f90-51" style="color:rgb(53,114,176)"></a>
<a name="GridP.f90-52" style="color:rgb(53,114,176)"></a><span class="gmail-k" style="font-weight:700">call </span><span class="gmail-n">CDA3</span><span class="gmail-p">(</span><span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">IMax</span><span class="gmail-o" style="font-weight:700">-</span><span class="gmail-mi" style="color:rgb(0,153,153)">1</span><span class="gmail-p">,</span><span class="gmail-n">JMax</span><span class="gmail-o" style="font-weight:700">-</span><span class="gmail-mi" style="color:rgb(0,153,153)">1</span><span class="gmail-p">,</span><span class="gmail-n">KMax</span><span class="gmail-o" style="font-weight:700">-</span><span class="gmail-mi" style="color:rgb(0,153,153)">1</span><span class="gmail-p">)</span>
<a name="GridP.f90-53" style="color:rgb(53,114,176)"></a>
<a name="GridP.f90-54" style="color:rgb(53,114,176)"></a><span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">!xpdmda,yp,zp have the same dimensions than x,y,z</span>
<a name="GridP.f90-55" style="color:rgb(53,114,176)"></a><span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">!xpl,ypl,zpl are the local versions of xp,yp,zp</span>
<a name="GridP.f90-56" style="color:rgb(53,114,176)"></a>
<a name="GridP.f90-57" style="color:rgb(53,114,176)"></a><span class="gmail-c" style="font-style:italic;color:rgb(153,153,136)">!We create the global and local vectors and communicate their values:</span>
<a name="GridP.f90-58" style="color:rgb(53,114,176)"></a><span class="gmail-k" style="font-weight:700">call </span><span class="gmail-n">DAs</span><span class="gmail-p">(</span><span class="gmail-n">da</span><span class="gmail-p">,</span><span class="gmail-n">xpdmda</span><span class="gmail-p">,</span><span class="gmail-n">xpl</span><span class="gmail-p">,</span><span class="gmail-n">x</span><span class="gmail-p">)</span></pre><pre style="color:rgb(51,51,51);font-size:12px;font-weight:normal;font-family:consolas,menlo,"liberation mono",courier,monospace;margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-p"><br></span></pre><pre style="color:rgb(51,51,51);font-weight:normal;margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-p"><font face="arial, helvetica, sans-serif">And at this call it crashes with the following message:</font></span></pre><pre style="color:rgb(51,51,51);font-size:12px;font-weight:normal;font-family:consolas,menlo,"liberation mono",courier,monospace;margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-p"><br></span></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;line-height:1.4"><span class="gmail-p"><font color="#333333" face="consolas, menlo, liberation mono, courier, monospace">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Null argument, when expecting valid pointer
[0]PETSC ERROR: Null Object: Parameter # 2
[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html">http://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.
[0]PETSC ERROR: Petsc Development GIT revision: v3.7.5-2949-gc3c607c GIT Date: 2017-01-20 17:34:16 -0600
[0]PETSC ERROR: ./ucmsLEP on a arch-linux2-c-debug named ocean by valera Sat Feb 4 15:53:53 2017
[0]PETSC ERROR: Configure options --with-cc=gcc --with-cxx=g++ --with-fc=gfortran --download-fblaslapack --download-mpich --download-hdf5 --download-netcdf --download-hypre --download-metis --download-parmetis --download-trillinos --with-debugging=1
[0]PETSC ERROR: #3 VecSetLocalToGlobalMapping() line 79 in /usr/dataC/home/valera/petsc/src/vec/vec/interface/vector.c
[0]PETSC ERROR: #4 DMCreateGlobalVector_DA() line 41 in /usr/dataC/home/valera/petsc/src/dm/impls/da/dadist.c
[0]PETSC ERROR: #5 DMCreateGlobalVector() line 840 in /usr/dataC/home/valera/petsc/src/dm/interface/dm.c
</font></span></pre><div style="color:rgb(51,51,51);font-size:12px;font-weight:normal"><span class="gmail-p"><br></span></div><div style="color:rgb(51,51,51);font-size:12px;font-weight:normal"><span class="gmail-p"><br></span></div><div style="color:rgb(51,51,51);font-weight:normal"><font face="arial, helvetica, sans-serif">So my question is, do i have to setup the vectors in any additional way? i followed the fortran DMDA examples closely and i didnt notice anything extra,</font></div><div style="color:rgb(51,51,51);font-weight:normal"><font face="arial, helvetica, sans-serif"><br></font></div><div style="color:rgb(51,51,51);font-weight:normal"><font face="arial, helvetica, sans-serif">Thanks,</font></div><div style="color:rgb(51,51,51);font-weight:normal"><font face="arial, helvetica, sans-serif"><br></font></div><div style="color:rgb(51,51,51);font-weight:normal"><font face="arial, helvetica, sans-serif">Manuel</font></div></span></pre></pre></span></pre></span></div></div>