[petsc-users] How to have a local copy (sequential) of a parallel matrix

[ehsan sadrfaridpour]

Thanks, the IS problem is solved.
But now I have another problem to compile the code.

*I use below code:*

> Mat m_WA_nt_local;
>     MatCreateSeqAIJ(PETSC_COMM_SELF,num_points,num_points,
> Config_params::getInstance()->get_pre_init_loader_matrix(), NULL,
> &m_WA_nt_local);
>     IS set;
>     if(rank ==0){
>         ISCreateStride(PETSC_COMM_SELF, num_points, 0, 1, &set);
>         ISView(set, PETSC_VIEWER_STDOUT_SELF);
>     }
>     MatGetSubMatrices(m_WA_norm_T, 1, &set, &set, MAT_INITIAL_MATRIX,
> &m_WA_nt_local);
>

*The error I get is :*

> error: cannot convert ‘_p_Mat**’ to ‘_p_Mat***’ for argument ‘6’ to
> ‘PetscErrorCode MatGetSubMatrices(Mat, PetscInt, _p_IS* const*, _p_IS*
> const*, MatReuse, _p_Mat***)’
>      MatGetSubMatrices(m_WA_norm_T, 1, &set, &set, MAT_INITIAL_MATRIX,
> &m_WA_nt_local);
>


*I tried to go around it by define a array of Matrices using "Mat *
m_WA_nt_local"*
So, the first 2 lines changed to below and I can compile the code.

> Mat * m_WA_nt_local;
>     MatCreateSeqAIJ(PETSC_COMM_SELF,num_points,num_points,
> Config_params::getInstance()->get_pre_init_loader_matrix(), NULL,
> m_WA_nt_local);
>



*However, I get errors like below when I run the code with 2 mpi process.*

>  --------------------- Error Message
> --------------------------------------------------------------
> [1]PETSC ERROR: Invalid argument
> [1]PETSC ERROR: Wrong type of object: Parameter # 3
> [1]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
> for trouble shooting.
> [1]PETSC ERROR: Petsc Release Version 3.6.3, unknown
> [1]PETSC ERROR: ut_main on a linux-cxx-debug named grappelli by esfp Wed
> Jun 29 16:21:04 2016
> [1]PETSC ERROR: Configure options PETSC_ARCH=linux-cxx-debug --with-cc=gcc
> --with-cxx=g++ --with-clanguage=c++ --with-gnu-compilers=1
> --with-mpi-compilers=1 --with-debugging=1 --with-shared-libraries=1
> --download-openmpi=1 --download-f2cblaslapack --download-superlu_dist=1
> --download-superlu=1 --download-metis=1 --download-parmetis=1
> --download-blacs=1 --with-hdf5 --with-hdf5-dir=/usr/local/hdf5/
> [1]PETSC ERROR: #1 MatGetSubMatrices() line 6605 in
> /home/esfp/tools/libraries/petsc/src/mat/interface/matrix.c
>


*I think I need to do something for other processes, but I don't know what
I need to do.*

Best,
Ehsan



On Wed, Jun 29, 2016 at 4:03 PM, Dave May <dave.mayhem23 at gmail.com> wrote:

>
>
> On Wednesday, 29 June 2016, ehsan sadrfaridpour <it.sadr at gmail.com> wrote:
>
>> I faced the below error during compiling my code for using
>> MatGetSubMatrices.
>>
>> error: cannot convert ‘IS {aka _p_IS*}’ to ‘_p_IS* const*’ for argument
>>> ‘3’ to ‘PetscErrorCode MatGetSubMatrices(Mat, PetscInt, _p_IS* const*,
>>> _p_IS* const*, MatReuse, _p_Mat***)’
>>>          MatGetSubMatrices(m_WA_norm_T, 1, set, set, MAT_INITIAL_MATRIX,
>>> &m_local_W);
>>>
>>
>> My code :
>>
>>> PetscMPIInt    rank;
>>> MPI_Comm_rank(PETSC_COMM_WORLD, &rank);
>>>
>>> if(rank ==0){
>>>         Mat m_local_W;
>>>         MatCreateSeqAIJ(PETSC_COMM_SELF,num_points,num_points, num_nz,
>>> NULL,&m_local_W);// try to reserve space for only number of final non zero
>>> entries for each fine node (e.g. 4)
>>>         IS set;
>>>         ISCreateStride(PETSC_COMM_SELF, num_points, 0, 1, &set_row);
>>>         MatGetSubMatrices(m_WA_norm_T, 1, set_row, set_col,
>>> MAT_INITIAL_MATRIX, &m_local_W);
>>>
>>>     }
>>>
>>
>> I followed below example:
>>
>> http://www.mcs.anl.gov/petsc/petsc-current/src/vec/is/is/examples/tutorials/ex2.c.html
>>
>
> This code won't work in parallel.
> The man page says this function is collective on Mat. You need to move the
> call to MatGetSubMatrices outside of the if(rank==0) loop.
>
>
>
>>
>>
>>
>>
>>
>> On Wed, Jun 29, 2016 at 3:19 PM, ehsan sadrfaridpour <it.sadr at gmail.com>
>> wrote:
>>
>>> Thanks a lot for great support.
>>>
>>> On Wed, Jun 29, 2016 at 3:11 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>
>>>>
>>>>    MatGetSubmatrices() just have the first process request all the rows
>>>> and columns and the others request none. You can use ISCreateStride() to
>>>> create the ISs without having to make an array of all the indices.
>>>>
>>>>
>>>> > On Jun 29, 2016, at 1:43 PM, ehsan sadrfaridpour <it.sadr at gmail.com>
>>>> wrote:
>>>> >
>>>> > Hi,
>>>> >
>>>> > I need to have access to most of elements of a parallel MPIAIJ matrix
>>>> only from 1 process (rank 0).
>>>> > I tried to copy or duplicate it to SEQAIJ, but I faced problems.
>>>> >
>>>> > How can I have a local copy of a matrix which is distributed on
>>>> multiple process? I don't want to update the matrix, and the read-only
>>>> version of it would be enough.
>>>> >
>>>> > Best,
>>>> > Ehsan
>>>> >
>>>> >
>>>>
>>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160629/75f9838a/attachment.html>