[petsc-users] Get wrong answer when use multi-process

[Jed Brown]

张国熙 <altriaex86 at gmail.com> writes:

> Hi, all
>
> I am confused about my code, for it could return right answer when I use 1
> process, but return totally wrong answer when more than 1 process.
>
> This is how I feed data to it.
>
> I have a CSR matrix, represented by Ap(pointer),Ai(index),and temp(data).

This matrix is stored redundantly on each process?

You should run with valgrind and confirm that you assemble the same
matrix in parallel before worrying about solvers.

> First determine local matrix for each process. Then feed data to them.
>
>   int temprank,localsize,line_pos;
>     line_pos = 0;
>     if(rank == 0)
>     {
>         localsize   = size/pro + ((size % pro) > rank);
>     }
>     else
>     {
>         for (temprank = 0;temprank <rank;temprank++)
>         {
>             localsize   = size/pro + ((size % pro) > temprank);
>             line_pos += localsize;
>         }
>     }
>
>     Lin_index = new int [localsize+1];
>     for(i=0;i<localsize+1;i++)
>     {
>         Lin_index [i] = Ap[line_pos+i]-Ap[line_pos];
>     }
>     std::cerr<<"line_pos    "<<line_pos<<"\n";
>     MatMPIAIJSetPreallocationCSR(A,Lin_index,Ai+line_pos,temp+line_pos);
>
> I use spectral transform with MATSOLVERMUMPS to calculate eigenvalue.
>
>
> The strange thing is, when I run it with one process, the eigenvalue is
> what I want, typically,
>  (8.39485e+13,5.3263)  (3.93842e+13,-82.6948) first two.
> But for 2 process:
> eigenvalue    (2.76523e+13,7.62222e+12)
> eigenvalue    (2.76523e+13,-7.62222e+12)
>
> 3 process:
> eigenvalue    (6.81292e+13,-3071.82)
> eigenvalue    (3.49533e+13,2.48858e+13)
>
> 4
> eigenvalue    (9.7562e+13,5012.4)
> eigenvalue    (7.2019e+13,8.28561e+13)
>
> However, it could pass simple test like
>     int n = 12;
>     int nz = 12;
>     int Ap[13] = {0,1,2,3,4,5,6,7,8,9,10,11,12};
>     int Ai[12] = { 0,1,2,3,4,5,6,7,8,9,10,11};
>     double Ax[12] = {-1,-2,-3,-4,-5,6,7,8,9,10,11};
>     double Az[12] = {0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0};
>
>
> Do you have any idea about it?
>
> Thanks a lot!!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140513/eca84ec5/attachment.pgp>