[petsc-users] MatMult() returning different values depending on # of processors?
Peter Brune
prbrune at gmail.com
Thu Jun 26 21:05:03 CDT 2014
Yep! Looks good. One other thing is that you can pre/postface all PETSc
calls with ierr = ... CHKERRQ(ierr); and it will properly trace other
problems if they arise.
- Peter
On Thu, Jun 26, 2014 at 8:56 PM, Arthur Kurlej <akurlej at gmail.com> wrote:
> Hello,
>
> I'm sorry, but this is a bit new for me, and I'm still not quite sure I
> follow.
>
> Are you recommending that opposed to doing this:
> if(procs!=1){
> for(i=0;i<procs;i++){
> if(rank==i){
> VecGetLocalSize(*x,&length);
> VecGetOwnershipRange(*x,&div,NULL);
> ISCreateStride(PETSC_COMM_WORLD,length,div+begin,1,&iscol);
> ISCreateStride(PETSC_COMM_WORLD,length,div+begin,1,&isrow);
> ierr = MatGetSubMatrix(*A,isrow,iscol,MAT_INITIAL_MATRIX,AA);
> CHKERRQ(ierr);
> }
> }
> }
> else{
> ISCreateStride(PETSC_COMM_SELF,final_size,begin,1,&iscol);
> ISCreateStride(PETSC_COMM_SELF,final_size,begin,1,&isrow);
> ierr = MatGetSubMatrix(*A,isrow,iscol,MAT_INITIAL_MATRIX,AA);
> CHKERRQ(ierr);
> }
>
>
> The proper implementation would instead just be the following:
> VecGetLocalSize(*x,&length);
> VecGetOwnershipRange(*x,&div,NULL);
> ISCreateStride(PETSC_COMM_WORLD,length,div+begin,1,&iscol);
> ISCreateStride(PETSC_COMM_WORLD,length,div+begin,1,&isrow);
> ierr = MatGetSubMatrix(*A,isrow,iscol,MAT_INITIAL_MATRIX,AA);
> CHKERRQ(ierr);
> ?
>
>
>
>
>
> On Thu, Jun 26, 2014 at 5:32 PM, Peter Brune <prbrune at gmail.com> wrote:
>
>> MatGetSubMatrix() is collective on Mat and ISCreateXXX is collective on
>> the provided comm, so the logic you have built to call it on one proc at a
>> time is unnecessary at best and most likely incorrect and likely to produce
>> strange results. You can forgo the if statement and loop over processors,
>> create the ISes on the same comm as x, and then call MatGetSubMatrix() once.
>>
>> - Peter
>>
>>
>> On Thu, Jun 26, 2014 at 4:26 PM, Arthur Kurlej <akurlej at gmail.com> wrote:
>>
>>> I cannot send the original code, but I reproduced the problem in another
>>> code. I have attached a makefile the code, and the data for the x vector
>>> and A matrix.
>>>
>>> I think the problem may be with my ShortenMatrix function, but it's not
>>> clear to me what exactly is going wrong and how to fix it. So I would
>>> appreciate some assistance there.
>>>
>>>
>>> Thanks,
>>> Arthur
>>>
>>>
>>>
>>> On Wed, Jun 25, 2014 at 6:24 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>
>>>>
>>>> Can you send the code that reproduces this behavior?
>>>>
>>>> Barry
>>>>
>>>> On Jun 25, 2014, at 4:37 PM, Arthur Kurlej <akurlej at gmail.com> wrote:
>>>>
>>>> > Hi Barry,
>>>> >
>>>> > So for the matrix C that I am currently testing (size 162x162), the
>>>> condition number is roughly 10^4.
>>>> >
>>>> > For reference, I'm porting MATLAB code into PETSc, and for one
>>>> processor, the PETSc b vector is roughly equivalent to the MATLAB b vector.
>>>> So I know that for one processor, my program is performing as expected.
>>>> >
>>>> > I've included examples below of values for b (also of size 162),
>>>> ranging from indices 131 to 141.
>>>> >
>>>> > #processors=1:
>>>> > 0
>>>> > 1.315217173959314e-20
>>>> > 1.315217173959314e-20
>>>> > 4.843201487740107e-17
>>>> > 4.843201487740107e-17
>>>> > 8.166104700666665e-14
>>>> > 8.166104700666665e-14
>>>> > 6.303834267553249e-11
>>>> > 6.303834267553249e-11
>>>> > 2.227932688485483e-08
>>>> > 2.227932688485483e-08
>>>> >
>>>> > # processors=2:
>>>> > 5.480410831461926e-22
>>>> > 2.892553944350444e-22
>>>> > 2.892553944350444e-22
>>>> > 7.524038923310717e-24
>>>> > 7.524038923214420e-24
>>>> > -3.340766769043093e-26
>>>> > -7.558372155761972e-27
>>>> > 5.551561288838557e-25
>>>> > 5.550551546879874e-25
>>>> > -1.579397982093437e-22
>>>> > 2.655766754178065e-22
>>>> >
>>>> > # processors = 4:
>>>> > 5.480410831461926e-22
>>>> > 2.892553944351728e-22
>>>> > 2.892553944351728e-22
>>>> > 7.524092205125593e-24
>>>> > 7.524092205125593e-24
>>>> > -2.584939414228212e-26
>>>> > -2.584939414228212e-26
>>>> > 0
>>>> > 0
>>>> > -1.245940797657998e-23
>>>> > -1.245940797657998e-23
>>>> >
>>>> > # processors = 8:
>>>> > 5.480410831461926e-22
>>>> > 2.892553944023035e-22
>>>> > 2.892553944023035e-22
>>>> > 7.524065744581494e-24
>>>> > 7.524065744581494e-24
>>>> > -2.250265175188197e-26
>>>> > -2.250265175188197e-26
>>>> > -6.543127892265160e-26
>>>> > 1.544288143499193e-317
>>>> > 8.788794008375919e-25
>>>> > 8.788794008375919e-25
>>>> >
>>>> >
>>>> > Thanks,
>>>> > Arthur
>>>> >
>>>> >
>>>> >
>>>> > On Wed, Jun 25, 2014 at 4:06 PM, Barry Smith <bsmith at mcs.anl.gov>
>>>> wrote:
>>>> >
>>>> > How different are the values in b? Can you send back a few
>>>> examples of the different b’s? Any idea of the condition number of C?
>>>> >
>>>> > Barry
>>>> >
>>>> > On Jun 25, 2014, at 3:10 PM, Arthur Kurlej <akurlej at gmail.com> wrote:
>>>> >
>>>> > > Hi all,
>>>> > >
>>>> > > While running my code, I have found that MatMult() returns
>>>> different values depending on the number of processors I use (and there is
>>>> quite the variance in the values).
>>>> > >
>>>> > > The setup of my code is as follows (I can go into more
>>>> depth/background if needed):
>>>> > > -Generate parallel AIJ matrix of size NxN, denoted as A
>>>> > > -Retrieve parallel AIJ submatrix from the last N-1 rows&columns
>>>> from A, denoted as C
>>>> > > -Generate vector of length N-1, denoted as x
>>>> > > -Find C*x=b
>>>> > >
>>>> > > I have already checked that A, C, and x are all equivalent when ran
>>>> for any number of processors, it is only the values of vector b that varies.
>>>> > >
>>>> > > Does anyone have an idea about what's going on?
>>>> > >
>>>> > >
>>>> > > Thanks,
>>>> > > Arthur
>>>> > >
>>>> >
>>>> >
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140626/dacdaee5/attachment-0001.html>
More information about the petsc-users
mailing list