<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Of course this is not my real world
example, and not the best way to fill a matrix. I just wanted to
quickly fill the matrices and give an example for my problem.
Therefore it does not matter how the matrices look like, the
problem is that the resulting sum of these three matrices is
wrong.<br>
<br>
(e.g. all matricies have values beside the diagonal block which
have the value a, therefore the resulting sum should have the
value 3*a, but in my computation the result is a. Some values are
computed correctly, the entry (1,1) for example, but most are
wrong.)<br>
<br>
Overall the interessting part is:<br>
<br>
MatAXPY(Ah,1.,At,SAME_NONZERO_PATTERN);<br>
MatAXPY(A,1.,Ah,SAME_NONZERO_PATTERN);<br>
<br>
I made an matrix output for the Matrix A before, after this
calculation and at the end of the code. With the result, that the
matrices after computing the sum are wrong.<br>
<br>
Best<br>
<br>
Klaus<br>
<br>
On 04/09/2015 09:11 PM, Matthew Knepley wrote:<br>
</div>
<blockquote
cite="mid:CAMYG4GkUKgbtzvQRB2L8GXqSGD82dhfeYzLFEyxXjvoos14+3A@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Mar 27, 2015 at 11:07 AM,
Klaus Kaiser <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:kaiser@igpm.rwth-aachen.de" target="_blank">kaiser@igpm.rwth-aachen.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hallo Matthew,<br>
<br>
here is a short example I used for 8 cores:<br>
<br>
int d_nnz[3];<br>
int o_nnz[3];<br>
d_nnz[0] = 3; d_nnz[1] = 3; d_nnz[2] = 3;<br>
o_nnz[0] = 6; o_nnz[1] = 6; o_nnz[2] = 6;<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9,
72, 72, 0, d_nnz, 0, o_nnz,&A);<span class=""><br>
ierr =
MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr =
MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
</span> ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3,
9, 9, 72, 72, 0, d_nnz, 0, o_nnz,&Ah);<br>
ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9,
72, 72, 0, d_nnz, 0, o_nnz,&At);<br>
<br>
std::vector<double> insert(3*3*3*3, 1.0);<br>
for(int i=0;i<8;++i)<br>
{<br>
int rows[3] = {i,i+1,i+3};<br>
int cols[3] = {i,i+1,i+3};<br>
<br>
MatSetValuesBlocked(A, 3, rows, 3, cols,
&insert[0], ADD_VALUES);<br>
}<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I do not understand this example. All the procs (8) are
filling up only the first 32 rows. Is that</div>
<div>what you intended? Can you tell me what output you
expect?</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div> MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
MatDuplicate(A,MAT_COPY_VALUES,&Ah);<br>
MatDuplicate(A,MAT_COPY_VALUES,&At);<br>
MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY);<br>
<br>
MatAXPY(Ah,1.,At,SAME_NONZERO_PATTERN);<br>
MatAXPY(A,1.,Ah,SAME_NONZERO_PATTERN);<br>
<br>
MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
<br>
The result is, that only some values are computed
correctly.<br>
<br>
Best and Thank you a lot<span class="HOEnZb"><font
color="#888888"><br>
<br>
Klaus</font></span>
<div>
<div class="h5"><br>
<br>
<br>
<br>
On 03/27/2015 04:26 PM, Matthew Knepley wrote:<br>
</div>
</div>
</div>
<div>
<div class="h5">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Mar 27, 2015
at 10:16 AM, Klaus Kaiser <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:kaiser@igpm.rwth-aachen.de"
target="_blank">kaiser@igpm.rwth-aachen.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hallo Matthew,<br>
<br>
thanks for your fast response. With
"no beside the nonzero structure" I
meant, that I do not create a
different non-zero structure while I'm
adding values to my matrices.<br>
<br>
I also tried MAT_COPY_VALUES and the
result was the same.<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Please send the small example and we
will figure out what is going wrong.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div> I do not get a error message but
the resulting matrix is wrong. Here I
have a short example. The first 9 rows
and first 3 colomn of the matrices Ah,
At and Ah+At on the first processor
(of 8):<br>
<br>
Ah: <br>
0:
60.3553
-0.249975 2.77556e-17<br>
1:
0
60.3553 0<br>
2:
0
0 60.3553<br>
3:
17.6777
0.374962 0.124987<br>
4:
0
17.6777 0<br>
5:
0
0 17.6777<br>
6:
-7.32233
-0.124987 0.374962<br>
7:
0
-7.32233 0<br>
8:
0
0 -7.32233<br>
<br>
<br>
At:<br>
0:
0
0 0<br>
1:
2500
0 0<br>
2:
-4.54747e-13
0 0<br>
3:
0
0 0<br>
4:
1250
0 0<br>
5:
1250
0 0<br>
6:
0
0 0<br>
7:
1250
0 0<br>
8:
3750
0 0<br>
<br>
Ah+At<br>
0:
60.3553
-0.249975 2.77556e-17<br>
1:
2500
60.3553 0<br>
2:
-4.54747e-13
0 60.3553<br>
3:
17.6777
0.374962 0.124987<br>
4:
0
17.6777 0<br>
5:
0
0 17.6777<br>
6:
-7.32233
-0.124987 0.374962<br>
7:
0
-7.32233 0<br>
8:
0
0 -7.32233<br>
<br>
you can see the first 3 rows of the
resulting matrix looks exactly like
what I would expect, but the last 6
rows only consists of the values of
Ah. When you would also look on the
matrix A and A+Ah+At you would also
see, that values where both matrices
have an nonzero entry are not sum
correctly.<br>
<br>
Best<br>
<br>
Klaus<br>
<br>
On 03/27/2015 03:59 PM, Matthew
Knepley wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri,
Mar 27, 2015 at 9:48 AM, Klaus
Kaiser <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:kaiser@igpm.rwth-aachen.de"
target="_blank">kaiser@igpm.rwth-aachen.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">Hallo,<br>
<br>
I have a strange behavior in
my code concerning the
function MatAXPY. I create 3
different Matrices<br>
<br>
ierr =
MatCreateBAIJ(PETSC_COMM_WORLD,
block_size, local_size,
local_size, system_size,
system_size, 0, d_nnz, 0,
o_nnz,&A);<br>
ierr =
MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr =
MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
<br>
ierr =
MatCreateBAIJ(PETSC_COMM_WORLD,
block_size, local_size,
local_size, system_size,
system_size, 0, d_nnz, 0,
o_nnz,&At);<br>
ierr =
MatSetOption(At,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr =
MatSetOption(At,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(At,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
<br>
ierr =
MatCreateBAIJ(PETSC_COMM_WORLD,
block_size, local_size,
local_size, system_size,
system_size, 0, d_nnz, 0,
o_nnz,&Ah);<br>
ierr =
MatSetOption(Ah,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
ierr =
MatSetOption(Ah,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
ierr =
MatSetOption(Ah,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
</blockquote>
<div><br>
</div>
<div>These creations are
superfluous since you use
MatDuplicate() below.</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex"> and
want to sum these three
matrixes with different
factors. First I fill the
Matrix A with some values, and
duplicate the structure of A
to the other two matrices:<br>
<br>
MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&Ah);<br>
MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&At);<br>
MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY);<br>
MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY);<br>
</blockquote>
<div> <br>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex"> After
this I fill the matrices At
and Ah with some other values,
which are not beside the non
zero structure (I also tried
with just copying the Matrix
A). Now after another
MatAssembly</blockquote>
<div><br>
</div>
<div>I do not understand "no
beside the nonzero structure".
Do you mean that the nonzero
structure is the same?</div>
<div><br>
</div>
<div>Can you first test with
MAT_COPY_VALUES?</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex"> I
want to add these Matrices in
the form A+c*(Ah+d*At): </blockquote>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex"> <br>
MatAXPY(Ah,c,At,SAME_NONZERO_PATTERN);<br>
MatAXPY(A,d,Ah,SAME_NONZERO_PATTERN);<br>
<br>
When I run the method with mpi
and one core everything works
fine. Starting the same method
with more cores, the sum of
the matrices fails. It seems
like some values are added</blockquote>
<div><br>
</div>
<div>Please send the full output
of a failure when you use
MAT_COPY_VALUES on 2 procs.</div>
<div><br>
</div>
<div> Thanks,</div>
<div><br>
</div>
<div> Matt</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
correctly and many values are
missed. Using
DIFFERENT_NONZERO_STRUCTURE
leads to the right behavior in
the multi-core case, but is
very slow. I checked with a
viewer if all matrices have
the same nonzero structure and
this is the case.<br>
<br>
Does anyone know why this
fails, or do I have made any
wrong thoughts?<br>
<br>
I'm corrently working with a
petsc version (Petsc Release
Version 3.3.0, Patch 5, Sat
Dec 1 15:10:41 CST 2012), I
looked into the changelogs up
to the current version and did
not find any note about
MatAXPY or MatAYPX.<br>
<br>
<br>
Best and Thanks a lot for your
help<span><font
color="#888888"><br>
<br>
Klaus<br>
<br>
</font></span></blockquote>
</div>
<br>
<br clear="all">
<span><font color="#888888">
<div><br>
</div>
-- <br>
<div>What most experimenters
take for granted before they
begin their experiments is
infinitely more interesting
than any results to which
their experiments lead.<br>
-- Norbert Wiener</div>
</font></span></div>
</div>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div>What most experimenters take for granted
before they begin their experiments is
infinitely more interesting than any results
to which their experiments lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
<div class="gmail_signature">What most experimenters take for
granted before they begin their experiments is infinitely
more interesting than any results to which their experiments
lead.<br>
-- Norbert Wiener</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>