<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hong,<br>
<br>
It seems now I know why I used MAT_REUSE_MATRIX and preallocated
matrix.<br>
I removed MatCreateMPIAIJ() and use just MatTranspose with
MAT_INITIAL_MATRIX.<br>
As a consequence I get hundreds of errors like:<br>
<br>
[30]PETSC ERROR: MatSetValues_MPIAIJ() line 538 in
/home/mt/agrayver/lib/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c<br>
[30]PETSC ERROR: MatAssemblyEnd_MPIAIJ() line 653 in
/home/mt/agrayver/lib/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c<br>
[30]PETSC ERROR: MatAssemblyEnd() line 4978 in
/home/mt/agrayver/lib/petsc-dev/src/mat/interface/matrix.c<br>
[30]PETSC ERROR: MatTranspose_MPIAIJ() line 2061 in
/home/mt/agrayver/lib/petsc-dev/src/mat/impls/aij/mpi/mpiaij.c<br>
[30]PETSC ERROR: MatTranspose() line 4397 in
/home/mt/agrayver/lib/petsc-dev/src/mat/interface/matrix.c<br>
[31]PETSC ERROR: --------------------- Error Message
------------------------------------<br>
[31]PETSC ERROR: Argument out of range!<br>
[31]PETSC ERROR: New nonzero at (1659,53337) caused a malloc!<br>
<br>
Ans this is the lastest petsc-dev revision. <br>
I know there were some changes in petsc-dev concerning this issue. <br>
Can you give me a hint how to avoid this?<br>
<br>
As for why I need C=A^T*B. This product is used further as a system
matrix for LSQR solver. The largest dimension is on the order of
10^6<br>
Since I form A and B myself I can, of course, form A^T explicitly,
but I thought I would first implement everything as it is written
down on paper and then optimize once it works (I thought it's easier
way).<br>
<br>
On 07.02.2012 20:44, Hong Zhang wrote:
<blockquote
cite="mid:CAGCphBsVZfsAUJ1SD=fLs-nZJ=OZsKbpU_149vtfkvpg36Y45Q@mail.gmail.com"
type="cite">Alexander,
<div>I'm curious about why do you need parallel C=A^T*B?</div>
<div>How large your matrices are?</div>
<div><br>
</div>
<div>In petsc-dev, we have MatTransposeMatMult() for mpiaij and
mpiaij, but not mpiaij and mpidense.</div>
<div>We may add support of MatTransposeMatMult_MPIAIJ_MPIDense()
if there is such need.</div>
<div><br>
</div>
<div>Hong</div>
<div><br>
<br>
<div class="gmail_quote">On Tue, Feb 7, 2012 at 1:18 PM, <a
moz-do-not-send="true" href="mailto:agrayver@gfz-potsdam.de">agrayver@gfz-potsdam.de</a>
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:agrayver@gfz-potsdam.de">agrayver@gfz-potsdam.de</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">Hong,<br>
<br>
Thanks for explanation. I will try this tomorrow. Good to
have this stuff in the help now.<br>
<br>
And sorry for misleading you initially.<br>
<br>
Regards,<br>
Alexander
<div class="HOEnZb">
<div class="h5"><br>
<br>
<div>----- Reply message -----<br>
From: "Hong Zhang" <<a moz-do-not-send="true"
href="mailto:hzhang@mcs.anl.gov" target="_blank">hzhang@mcs.anl.gov</a>><br>
To: "For users of the development version of PETSc"
<<a moz-do-not-send="true"
href="mailto:petsc-dev@mcs.anl.gov" target="_blank">petsc-dev@mcs.anl.gov</a>><br>
Subject: [petsc-dev] MatMatMult gives different
results<br>
Date: Tue, Feb 7, 2012 19:09<br>
<br>
</div>
<br>
Alexander :
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin: 0pt 0pt
0pt 0.8ex; border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000"><br>
There is something I didn't get yet, I hope you
could clarify it. <br>
<br>
So, when I use flag MAT_INITIAL_MATRIX in test
program it works fine.<br>
</div>
</blockquote>
<div>Good to know :-)</div>
<div> </div>
<blockquote class="gmail_quote" style="margin: 0pt 0pt
0pt 0.8ex; border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000"> If I put this
flag in my original program I get dozens of
exceptions like:<br>
[42]PETSC ERROR: Argument out of range!<br>
[42]PETSC ERROR: New nonzero at (1336,153341)
caused a malloc!<br>
</div>
</blockquote>
<div>You cannot do </div>
<div> <span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">MatCreateMPIAIJ()</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">MatTranspose(A,MAT_INITIAL_MATRIX,&AT);</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;"><br>
</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">MatCreateMPIAIJ() creates AT and
preallocates approximate nonzeros, which does not
match exactly the nonzeros in </span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">MatTranspose(A,MAT_INITIAL_MATRIX,&AT);</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">MatTranspose(A,MAT_INITIAL_MATRIX,&AT)
creates matrix AT and sets correct nonzero pattern
and values in AT. </span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">MatTranspose() only takes in "</span><span
style="border-collapse: collapse; color: rgb(80,
0, 80); font-family: arial,sans-serif; font-size:
14px;">MAT_INITIAL_MATRIX" - for a new AT,</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">and "MAT_REUSE_MATRIX" when AT
is created with </span><span
style="border-collapse: collapse; color: rgb(80,
0, 80); font-family: arial,sans-serif; font-size:
14px;">MatTranspose(A,MAT_INITIAL_MATRIX,&AT)</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">and reuse for updating its
values (not nonzero patten).</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;"><br>
</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">I'm updating petsc help menu on </span><span
style="border-collapse: collapse; color: rgb(80,
0, 80); font-family: arial,sans-serif; font-size:
14px;">MatTranspose(). Thanks for the report.</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;"><br>
</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;">Hong</span></div>
<div><span style="border-collapse: collapse; color:
rgb(80, 0, 80); font-family: arial,sans-serif;
font-size: 14px;"><br>
</span></div>
<blockquote class="gmail_quote" style="margin: 0pt 0pt
0pt 0.8ex; border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000"> <br>
I changed this flag to MAT_REUSE_MATRIX and
exceptions disappeared, but result is incorrect
again (same as for MAT_IGNORE_MATRIX) <br>
I tried test program with MAT_REUSE_MATRIX and it
also gives different matrix product.<br>
<br>
Since there is no description of MatReuse
structure for MatTranspose it's a bit confusing
what to expect from it. <br>
<div>
<div> <br>
<blockquote type="cite">
<div class="gmail_quote">
<blockquote class="gmail_quote"
style="margin: 0pt 0pt 0pt 0.8ex;
border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">
<div>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div><br>
</div>
<div>Do you mean 'Cm =
A'*B;'? </div>
<div>'Cm = A.'*B;' gives
component-wise matrix
product, not matrix product.</div>
</div>
</div>
</blockquote>
<br>
</div>
.' operator means non-Hermitian
transpose. That is what I get with
MatTranspose (in contrast with
MatHermitianTranspose)<br>
component-wise matrix product would be
.*</div>
</blockquote>
<div>You are correct.</div>
<div><br>
</div>
<div>Hong </div>
<blockquote class="gmail_quote"
style="margin: 0pt 0pt 0pt 0.8ex;
border-left: 1px solid rgb(204, 204,
204); padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">
<div>
<div><br>
<br>
<blockquote type="cite">
<div>
<div class="gmail_quote">
<div><br>
</div>
<div>Hong</div>
<div><br>
</div>
<div> </div>
<blockquote
class="gmail_quote"
style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px
solid rgb(204, 204, 204);
padding-left: 1ex;">
<div bgcolor="#ffffff"
text="#000000"> C =
PetscBinaryRead('C.dat','complex',true);<br>
<br>
Matrix C is different
depending on number of
cores I use. <br>
My PETSc is:<br>
Using Petsc Development
HG revision:
876c894d95f4fa6561d0a91310ca914592527960
HG Date: Tue Jan 10
19:27:14 2012 +0100
<div>
<div><br>
<br>
On 06.02.2012 17:13,
Hong Zhang wrote:
<blockquote
type="cite">MatMatMult()
in petsc is not
well-tested for
complex - could be
buggy.
<div>Can you send
us the matrices
A and B in petsc
binary format
for
investigation?</div>
<div><br>
</div>
<div>Hong<br>
<br>
<div
class="gmail_quote">On
Mon, Feb 6,
2012 at 5:55
AM, Alexander
Grayver <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:agrayver@gfz-potsdam.de"
target="_blank">agrayver@gfz-potsdam.de</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:
0pt 0pt 0pt
0.8ex;
border-left:
1px solid
rgb(204, 204,
204);
padding-left:
1ex;">Dear
PETSc team,<br>
<br>
I try to use:<br>
call
MatMatMult(A,B,MAT_INITIAL_MATRIX,PETSC_DEFAULT_DOUBLE_PRECISION,C,ierr);CHKERRQ(ierr)<br>
<br>
Where both A
and B are
rectangular,
but A is
sparse and B
is dense. Both
are double
complex and
distributed.<br>
The product
PETSc gives me
contains some
errors in some
part of the
matrix.<br>
I output A, B
and C then
computed
product in
matlab.<br>
<br>
Attached you
see figure
plotted as:<br>
imagesc(log10(abs(C-Cm)))<br>
<br>
Where Cm --
product
computed in
matlab.<br>
<br>
The pattern
and amplitude
vary depending
on the number
of cores I
use. This
picture is
obtained for
48 cores (I've
tried 12, 64
cores as
well).<br>
<br>
Where should I
look for
possible
explanation?<span><font
color="#888888"><br>
<br>
-- <br>
Regards,<br>
Alexander<br>
</font></span></blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
</div>
</div>
<span><font
color="#888888">
<pre cols="72">--
Regards,
Alexander</pre>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
</div>
</div>
<span><font color="#888888">
<pre cols="72">--
Regards,
Alexander</pre>
</font></span></div>
</blockquote>
</div>
<br>
</blockquote>
<br>
<br>
</div>
</div>
<span><font color="#888888">
<pre cols="72">--
Regards,
Alexander</pre>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Regards,
Alexander</pre>
</body>
</html>