[petsc-users] Question on local vec to global vec for dof > 1
Barry Smith
bsmith at mcs.anl.gov
Thu May 22 19:51:38 CDT 2014
VecStrideScatter,Gather<All> can be used to go from an interlaced vector to a separate vector for each component. You can also write code as below where you have a non-interlated ARRAY and you put/take the values into the array obtained with the DMDAVecGetArrayF90. In other words PETSc vectors remained interlaced but you work with other arrays that are not interlaced.
Barry
On May 22, 2014, at 7:42 PM, Danyang Su <danyang.su at gmail.com> wrote:
> On 22/05/2014 5:34 PM, Barry Smith wrote:
>> DMDA does not work that way. Local and global vectors associated with DA’s are always “interlaced”.
> Then, is there any routine convert matrix to be "interlaced"?
>
> Thanks,
>
> Danyang
>>
>> Barry
>>
>> On May 22, 2014, at 6:33 PM, Danyang Su <danyang.su at gmail.com> wrote:
>>
>>> Hi Barry,
>>>
>>> I use the following routine to reorder from the local rhs to global rhs.
>>>
>>> PetscScalar, pointer :: vecpointer1d(:,:)
>>>
>>> call DMDAVecGetArrayF90(da,x_vec_gbl,vecpointer1d,ierr) !x_vec_gbl is a global vector created by DMCreateGlobalVector
>>> do i = nvzls,nvzle !local node number without ghost node
>>> vecpointer1d(0,i-1) = x_array_loc(i-nvzgls+1) !x_array_loc is local rhs
>>> vecpointer1d(1,i-1) = x_array_loc(i-nvzgls+1+nvz) !nvz = 6 for the present 1d example
>>> end do
>>> call DMDAVecRestoreArrayF90(da,x_vec_gbl,vecpointer1d,ierr)
>>>
>>> Now Vecview gives the same rhs for 1 processor and 2 processors, but rhs order is not what I expected. I want global rhs vector hold the values of dof=1 first and then dof=2 as the local matrix and rhs hold value in this order.
>>>
>>> x_vec_gbl x_vec_gbl
>>> dof node VecView(Current) dof node VecView (Expected)
>>> 1 1 1.39598e-021 1 1 1.39598e-021
>>> 2 1 0 1 2 0
>>> 1 2 -0 1 3 0
>>> 2 2 -0 1 4 5.64237e-037
>>> 1 3 -0 1 5 0
>>> 2 3 -0 1 6 -7.52316e-037
>>> 1 4 5.64237e-037 1 7 7.52316e-037
>>> 2 4 4.81482e-035 1 8 0
>>> 1 5 -0 1 9 1.68459e-016
>>> 2 5 -0 1 10 0.1296
>>> 1 6 -7.52316e-037 2 1 0
>>> 2 6 -7.22224e-035 2 2 0
>>> 1 7 7.52316e-037 2 3 0
>>> 2 7 7.22224e-035 2 4 4.81482e-035
>>> 1 8 -0 2 5 0
>>> 2 8 -0 2 6 -7.22224e-035
>>> 1 9 1.68459e-016 2 7 7.22224e-035
>>> 2 9 128623 2 8 0
>>> 1 10 0.1296 2 9 128623
>>> 2 10 0 2 10 0
>>>
>>> Thanks and regards,
>>>
>>> Danyang
>>>
>>>
>>> On 22/05/2014 4:03 PM, Barry Smith wrote:
>>>> Always do things one small step at at time. On one process what is x_vec_loc (use VecView again on it). Is it what you expect? Then run on two processes but call VecView on
>>>> x_vec_loc only on the first process. Is it what you expect?
>>>>
>>>> Also what is vecpointer1d declared to be?
>>>>
>>>>
>>>> Barry
>>>>
>>>> On May 22, 2014, at 4:44 PM, Danyang Su
>>>> <danyang.su at gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>> On 22/05/2014 12:01 PM, Matthew Knepley wrote:
>>>>>
>>>>>> On Thu, May 22, 2014 at 1:58 PM, Danyang Su <danyang.su at gmail.com>
>>>>>> wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> I have a 1D transient flow problem (1 dof) coupled with energy balance (1 dof), so the total dof per node is 2.
>>>>>>
>>>>>> The whole domain has 10 nodes in z direction.
>>>>>>
>>>>>> The program runs well with 1 processor but failed in 2 processors. The matrix is the same for 1 processor and 2 processor but the rhs are different.
>>>>>>
>>>>>> The following is used to set the rhs value.
>>>>>>
>>>>>> call VecGetArrayF90(x_vec_loc, vecpointer, ierr)
>>>>>> vecpointer = (calculate the rhs value here)
>>>>>> call VecRestoreArrayF90(x_vec_loc,vecpointer,ierr)
>>>>>> call DMLocalToGlobalBegin(da,x_vec_loc,INSERT_VALUES, x_vec_gbl,ierr)
>>>>>> call DMLocalToGlobalEnd(da,x_vec_loc,INSERT_VALUES, x_vec_gbl,ierr)
>>>>>>
>>>>>> Vecview Correct Vecview Wrong
>>>>>> dof local node Process [0] Process [0] Process [0]
>>>>>> 1 1 1.395982780116148E-021 1.39598e-021 1.39598e-021
>>>>>> 1 2 0.000000000000000E+000 0 0
>>>>>> 1 3 0.000000000000000E+000 0 0
>>>>>> 1 4 5.642372883946980E-037 5.64237e-037 5.64237e-037
>>>>>> 1 5 0.000000000000000E+000 0 0
>>>>>> 1 6 -1.395982780116148E-021 -7.52316e-037 -1.39598e-021 Line A
>>>>>> 2 1 0.000000000000000E+000 7.52316e-037 0
>>>>>> 2 2 0.000000000000000E+000 0 0
>>>>>> 2 3 0.000000000000000E+000 1.68459e-016 0
>>>>>> 2 4 4.814824860968090E-035 0.1296 4.81482e-035
>>>>>> 2 5 0.000000000000000E+000 Process [1] Line B
>>>>>> 2 6 -1.371273884908092E-019 0 7.52316e-037 Line C
>>>>>> 0 0
>>>>>> Process [1] 0 1.68459e-016
>>>>>> 1 1 1.395982780116148E-021 4.81482e-035 0.1296 Line D
>>>>>> 1 2 -7.523163845262640E-037 0 1.37127e-019 Line E
>>>>>> 1 3 7.523163845262640E-037 -7.22224e-035 -7.22224e-035
>>>>>> 1 4 0.000000000000000E+000 7.22224e-035 7.22224e-035
>>>>>> 1 5 1.684590875336239E-016 0 0
>>>>>> 1 6 0.129600000000000 128623 128623
>>>>>> 2 1 1.371273884908092E-019 0 0 Line F
>>>>>> 2 2 -7.222237291452134E-035
>>>>>> 2 3 7.222237291452134E-035
>>>>>> 2 4 0.000000000000000E+000
>>>>>> 2 5 128623.169844761
>>>>>> 2 6 0.000000000000000E+000
>>>>>>
>>>>>> The red line (Line A, C, D and F) is the ghost values for 2 subdomains, but when run with 2 processor, the program treates Line B, C, D, and E as ghost values.
>>>>>> How can I handle this kind of local vector to global vector assembly?
>>>>>>
>>>>>> Why are you not using DMDAVecGetArrayF90()? This is exactly what it is for.
>>>>>>
>>>>> Thanks, Matthew.
>>>>>
>>>>> I tried the following codes but still cannot get the correct global rhs vector
>>>>> call DMDAVecGetArrayF90(da,x_vec_loc,vecpointer1d,ierr)
>>>>> do i = 1,nvz !nvz is local node amount, here is 6
>>>>> vecpointer1d(0,i-1) = x_array_loc(i) !assume x_array_loc is the local rhs (the third column in the above mentioned data)
>>>>> vecpointer1d(1,i-1) = x_array_loc(i+nvz)
>>>>> end do
>>>>> call DMDAVecRestoreArrayF90(da,x_vec_loc,vecpointer1d,ierr)
>>>>> call DMLocalToGlobalBegin(da,x_vec_loc,INSERT_VALUES, x_vec_gbl,ierr)
>>>>> call DMLocalToGlobalEnd(da,x_vec_loc,INSERT_VALUES, x_vec_gbl,ierr)
>>>>>
>>>>>
>>>>> Now the rhs for 1 processor is as follows. It is not what I want.
>>>>>
>>>>> 1.39598e-021
>>>>> 0
>>>>> -0
>>>>> -0
>>>>> -0
>>>>> -0
>>>>> 5.64237e-037
>>>>> 4.81482e-035
>>>>> -0
>>>>> -0
>>>>> -7.52316e-037
>>>>> -7.22224e-035
>>>>> 7.52316e-037
>>>>> 7.22224e-035
>>>>> -0
>>>>> -0
>>>>> 1.68459e-016
>>>>> 128623
>>>>> 0.1296
>>>>> 0
>>>>>
>>>>>> Matt
>>>>>>
>>>>>> In fact, the codes can work if the dof and local node is as follows.
>>>>>> dof local node
>>>>>> 1 1
>>>>>> 2 1
>>>>>> 1 2
>>>>>> 2 2
>>>>>> 1 3
>>>>>> 2 3
>>>>>>
>>>>>> Thanks and regards,
>>>>>>
>>>>>> Danyang
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.
>>>>>> -- Norbert Wiener
>>>>>>
>
More information about the petsc-users
mailing list