<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Mon, Mar 30, 2015 at 5:18 AM, Klaus Kaiser <span dir="ltr"><<a href="mailto:kaiser@igpm.rwth-aachen.de" target="_blank">kaiser@igpm.rwth-aachen.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hallo Metthew,<br>
<br>
at the weekend I updated petsc to the newest release version 3.5.2
and I still got this strange behavior.<br></div></div></blockquote><div><br></div><div>Okay, I will check it out. The first thing I am going to do is try with AIJ matrices instead of BAIJ, which</div><div>you can probably do before me. If you get the right result, then its a bug in the BAIJ implementation,</div><div>which I am confident has not been tested as thoroughly.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div>
Best<span class="HOEnZb"><font color="#888888"><br>
<br>
Klaus</font></span><div><div class="h5"><br>
<br>
On 03/27/2015 05:07 PM, Klaus Kaiser wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<div>Hallo Matthew,<br>
<br>
here is a short example I used for 8 cores:<br>
<br>
int d_nnz[3];<br>
int o_nnz[3];<br>
d_nnz[0] = 3; d_nnz[1] = 3; d_nnz[2] = 3;<br>
o_nnz[0] = 6; o_nnz[1] = 6; o_nnz[2] = 6;<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0,
d_nnz, 0, o_nnz,&A);<br>
ierr =
MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr = MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0,
d_nnz, 0, o_nnz,&Ah);<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0,
d_nnz, 0, o_nnz,&At);<br>
<br>
std::vector<double> insert(3*3*3*3, 1.0);<br>
for(int i=0;i<8;++i)<br>
{<br>
int rows[3] = {i,i+1,i+3};<br>
int cols[3] = {i,i+1,i+3};<br>
<br>
MatSetValuesBlocked(A, 3, rows, 3, cols, &insert[0],
ADD_VALUES);<br>
}<br>
<br>
MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
MatDuplicate(A,MAT_COPY_VALUES,&Ah);<br>
MatDuplicate(A,MAT_COPY_VALUES,&At);<br>
MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY);<br>
<br>
MatAXPY(Ah,1.,At,SAME_NONZERO_PATTERN);<br>
MatAXPY(A,1.,Ah,SAME_NONZERO_PATTERN);<br>
<br>
MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
<br>
The result is, that only some values are computed correctly.<br>
<br>
Best and Thank you a lot<br>
<br>
Klaus<br>
<br>
<br>
<br>
On 03/27/2015 04:26 PM, Matthew Knepley wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Mar 27, 2015 at 10:16 AM,
Klaus Kaiser <span dir="ltr"><<a href="mailto:kaiser@igpm.rwth-aachen.de" target="_blank">kaiser@igpm.rwth-aachen.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hallo Matthew,<br>
<br>
thanks for your fast response. With "no beside the
nonzero structure" I meant, that I do not create a
different non-zero structure while I'm adding values
to my matrices.<br>
<br>
I also tried MAT_COPY_VALUES and the result was the
same.<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Please send the small example and we will figure out
what is going wrong.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div> I do not get a error message but the resulting
matrix is wrong. Here I have a short example. The
first 9 rows and first 3 colomn of the matrices Ah,
At and Ah+At on the first processor (of 8):<br>
<br>
Ah: <br>
0: 60.3553
-0.249975 2.77556e-17<br>
1: 0
60.3553 0<br>
2: 0
0 60.3553<br>
3: 17.6777
0.374962 0.124987<br>
4: 0
17.6777 0<br>
5: 0
0 17.6777<br>
6: -7.32233
-0.124987 0.374962<br>
7: 0
-7.32233 0<br>
8: 0
0 -7.32233<br>
<br>
<br>
At:<br>
0: 0
0 0<br>
1: 2500
0 0<br>
2: -4.54747e-13
0 0<br>
3: 0
0 0<br>
4: 1250
0 0<br>
5: 1250
0 0<br>
6: 0
0 0<br>
7: 1250
0 0<br>
8: 3750
0 0<br>
<br>
Ah+At<br>
0: 60.3553
-0.249975 2.77556e-17<br>
1: 2500
60.3553 0<br>
2: -4.54747e-13
0 60.3553<br>
3: 17.6777
0.374962 0.124987<br>
4: 0
17.6777 0<br>
5: 0
0 17.6777<br>
6: -7.32233
-0.124987 0.374962<br>
7: 0
-7.32233 0<br>
8: 0
0 -7.32233<br>
<br>
you can see the first 3 rows of the resulting matrix
looks exactly like what I would expect, but the last
6 rows only consists of the values of Ah. When you
would also look on the matrix A and A+Ah+At you
would also see, that values where both matrices have
an nonzero entry are not sum correctly.<br>
<br>
Best<br>
<br>
Klaus<br>
<br>
On 03/27/2015 03:59 PM, Matthew Knepley wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Mar 27, 2015 at
9:48 AM, Klaus Kaiser <span dir="ltr"><<a href="mailto:kaiser@igpm.rwth-aachen.de" target="_blank">kaiser@igpm.rwth-aachen.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hallo,<br>
<br>
I have a strange behavior in my code
concerning the function MatAXPY. I create 3
different Matrices<br>
<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD,
block_size, local_size, local_size,
system_size, system_size, 0, d_nnz, 0,
o_nnz,&A);<br>
ierr =
MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr =
MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD,
block_size, local_size, local_size,
system_size, system_size, 0, d_nnz, 0,
o_nnz,&At);<br>
ierr =
MatSetOption(At,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr =
MatSetOption(At,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(At,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD,
block_size, local_size, local_size,
system_size, system_size, 0, d_nnz, 0,
o_nnz,&Ah);<br>
ierr =
MatSetOption(Ah,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr =
MatSetOption(Ah,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(Ah,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
</blockquote>
<div><br>
</div>
<div>These creations are superfluous since you
use MatDuplicate() below.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> and want to
sum these three matrixes with different
factors. First I fill the Matrix A with some
values, and duplicate the structure of A to
the other two matrices:<br>
<br>
MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&Ah);<br>
MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&At);<br>
MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY);<br>
</blockquote>
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> After this I
fill the matrices At and Ah with some other
values, which are not beside the non zero
structure (I also tried with just copying
the Matrix A). Now after another MatAssembly</blockquote>
<div><br>
</div>
<div>I do not understand "no beside the
nonzero structure". Do you mean that the
nonzero structure is the same?</div>
<div><br>
</div>
<div>Can you first test with MAT_COPY_VALUES?</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I want to add
these Matrices in the form A+c*(Ah+d*At): </blockquote>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br>
MatAXPY(Ah,c,At,SAME_NONZERO_PATTERN);<br>
MatAXPY(A,d,Ah,SAME_NONZERO_PATTERN);<br>
<br>
When I run the method with mpi and one core
everything works fine. Starting the same
method with more cores, the sum of the
matrices fails. It seems like some values
are added</blockquote>
<div><br>
</div>
<div>Please send the full output of a failure
when you use MAT_COPY_VALUES on 2 procs.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> correctly and
many values are missed. Using
DIFFERENT_NONZERO_STRUCTURE leads to the
right behavior in the multi-core case, but
is very slow. I checked with a viewer if all
matrices have the same nonzero structure and
this is the case.<br>
<br>
Does anyone know why this fails, or do I
have made any wrong thoughts?<br>
<br>
I'm corrently working with a petsc version
(Petsc Release Version 3.3.0, Patch 5, Sat
Dec 1 15:10:41 CST 2012), I looked into the
changelogs up to the current version and did
not find any note about MatAXPY or MatAYPX.<br>
<br>
<br>
Best and Thanks a lot for your help<span><font color="#888888"><br>
<br>
Klaus<br>
<br>
</font></span></blockquote>
</div>
<br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
<div>What most experimenters take for
granted before they begin their
experiments is infinitely more interesting
than any results to which their
experiments lead.<br>
-- Norbert Wiener</div>
</font></span></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>What most experimenters take
for granted before they begin their experiments is
infinitely more interesting than any results to which
their experiments lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</blockquote>
<br>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>