<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 22/05/2014 12:01 PM, Matthew Knepley
wrote:<br>
</div>
<blockquote
cite="mid:CAMYG4Gmyvo88x5A7AOEFptgYwdHRc885PDhjM5nStfJ96Gfe7Q@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Thu, May 22, 2014 at 1:58 PM,
Danyang Su <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:danyang.su@gmail.com" target="_blank">danyang.su@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Hi All,<br>
<br>
I have a 1D transient flow problem (1 dof) coupled with
energy balance (1 dof), so the total dof per node is 2.<br>
<br>
The whole domain has 10 nodes in z direction. <br>
<br>
The program runs well with 1 processor but failed in 2
processors. The matrix is the same for 1 processor and 2
processor but the rhs are different. <br>
<br>
The following is used to set the rhs value.<br>
<br>
call VecGetArrayF90(x_vec_loc, vecpointer, ierr)<br>
vecpointer = (calculate the rhs value here)<br>
call VecRestoreArrayF90(x_vec_loc,vecpointer,ierr)<br>
call DMLocalToGlobalBegin(da,x_vec_loc,INSERT_VALUES,
x_vec_gbl,ierr)<br>
call DMLocalToGlobalEnd(da,x_vec_loc,INSERT_VALUES,
x_vec_gbl,ierr)<br>
<br>
<b>Vecview Correct </b> <b><font
color="#ff0000">Vecview Wrong</font></b><br>
dof local node Process [0]
<u>Process [0] </u>
<i> <u>Process [0] </u></i><br>
1 1
1.395982780116148E-021
1.39598e-021 1.39598e-021<br>
1 2
0.000000000000000E+000
0 0<br>
1 3
0.000000000000000E+000
0 0<br>
1 4
5.642372883946980E-037
5.64237e-037 5.64237e-037<br>
1 5
0.000000000000000E+000
0 0<br>
<font color="#ff0000">1 6
-1.395982780116148E-021 </font> <font
color="#3366ff"> -7.52316e-037 </font>
-1.39598e-021 Line A<br>
2 1 <font color="#33ff33">0.000000000000000E+000</font>
<font color="#3366ff">7.52316e-037</font>
0<br>
2 2 <font color="#33ff33">0.000000000000000E+000</font>
<font color="#3366ff">0 </font>
0<br>
2 3 <font color="#33ff33">
0.000000000000000E+000</font> <font
color="#3366ff">1.68459e-016 </font> 0<br>
2 4 <font color="#33ff33">4.814824860968090E-035</font>
<font color="#3366ff">0.1296 </font>
4.81482e-035<br>
2 5 <font color="#33ff33">0.000000000000000E+000</font>
<u><i>Process [1]</i></u>
Line B<br>
<font color="#ff0000">2 6
-1.371273884908092E-019</font> <font
color="#33ff33">0</font>
7.52316e-037 Line C<br>
<font
color="#33ff33">0</font>
0<br>
Process [1]
<font color="#33ff33">0</font>
1.68459e-016<br>
<font color="#ff0000">1 1
1.395982780116148E-021</font> <font
color="#33ff33">4.81482e-035</font>
0.1296 Line D<br>
1 2 <font color="#3366ff">-7.523163845262640E-037</font>
<font color="#33ff33">0</font>
1.37127e-019 Line E
<br>
1 3 <font color="#3366ff">7.523163845262640E-037</font>
-7.22224e-035 -7.22224e-035<br>
1 4 <font color="#3366ff">0.000000000000000E+000</font>
7.22224e-035 7.22224e-035<br>
1 5 <font color="#3366ff">1.684590875336239E-016</font>
0 0<br>
1 6 <font color="#3366ff">0.129600000000000</font>
128623 128623<br>
<font color="#ff0000">2 1
1.371273884908092E-019</font>
0
0 Line F<br>
2 2
-7.222237291452134E-035 <br>
2 3
7.222237291452134E-035 <br>
2 4
0.000000000000000E+000 <br>
2 5
128623.169844761 <br>
2 6 0.000000000000000E+000 <br>
<br>
The red line (Line A, C, D and F) is the ghost values
for 2 subdomains, but when run with 2 processor, the
program treates Line B, C, D, and E as ghost values.<br>
<b>How can I handle this kind of local vector to global
vector assembly?</b></div>
</blockquote>
<div><br>
</div>
<div>Why are you not using DMDAVecGetArrayF90()? This is
exactly what it is for.</div>
</div>
</div>
</div>
</blockquote>
Thanks, Matthew. <br>
<br>
I tried the following codes but still cannot get the correct global
rhs vector<br>
<br>
call DMDAVecGetArrayF90(da,x_vec_loc,vecpointer1d,ierr)<br>
do i = 1,nvz
!nvz is local node amount, here is 6<br>
vecpointer1d(0,i-1) = x_array_loc(i)
!assume x_array_loc is the local rhs (the third column in the above
mentioned data)<br>
vecpointer1d(1,i-1) = x_array_loc(i+nvz)<br>
end do<br>
call DMDAVecRestoreArrayF90(da,x_vec_loc,vecpointer1d,ierr)<br>
call DMLocalToGlobalBegin(da,x_vec_loc,INSERT_VALUES,
x_vec_gbl,ierr)<br>
call DMLocalToGlobalEnd(da,x_vec_loc,INSERT_VALUES, x_vec_gbl,ierr)<br>
<br>
<br>
Now the rhs for 1 processor is as follows. It is not what I want.<br>
<br>
1.39598e-021<br>
0<br>
-0<br>
-0<br>
-0<br>
-0<br>
5.64237e-037<br>
4.81482e-035<br>
-0<br>
-0<br>
-7.52316e-037<br>
-7.22224e-035<br>
7.52316e-037<br>
7.22224e-035<br>
-0<br>
-0<br>
1.68459e-016<br>
128623<br>
0.1296<br>
0<br>
<blockquote
cite="mid:CAMYG4Gmyvo88x5A7AOEFptgYwdHRc885PDhjM5nStfJ96Gfe7Q@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><br>
<b>In fact, the codes can work if the dof and local node
is as follows.</b><br>
dof local node <br>
1 1 <br>
2 1 <br>
1 2 <br>
2 2 <br>
1 3 <br>
2 3 <br>
<br>
Thanks and regards,<br>
<br>
Danyang<br>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
What most experimenters take for granted before they begin
their experiments is infinitely more interesting than any
results to which their experiments lead.<br>
-- Norbert Wiener
</div>
</div>
</blockquote>
<br>
</body>
</html>