<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">Hallo Metthew,<br>
      <br>
      at the weekend I updated petsc to the newest release version 3.5.2
      and I still got this strange behavior.<br>
      <br>
      Best<br>
      <br>
      Klaus<br>
      <br>
      On 03/27/2015 05:07 PM, Klaus Kaiser wrote:<br>
    </div>
    <blockquote cite="mid:5515805B.3060406@igpm.rwth-aachen.de"
      type="cite">
      <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
      <div class="moz-cite-prefix">Hallo Matthew,<br>
        <br>
        here is a short example I used for 8 cores:<br>
        <br>
            int d_nnz[3];<br>
            int o_nnz[3];<br>
            d_nnz[0] = 3; d_nnz[1] = 3; d_nnz[2] = 3;<br>
            o_nnz[0] = 6; o_nnz[1] = 6; o_nnz[2] = 6;<br>
            ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0,
        d_nnz, 0, o_nnz,&A);<br>
            ierr =
        MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
            ierr = MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
            ierr =
        MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
            ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0,
        d_nnz, 0, o_nnz,&Ah);<br>
            ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0,
        d_nnz, 0, o_nnz,&At);<br>
        <br>
            std::vector<double> insert(3*3*3*3, 1.0);<br>
            for(int i=0;i<8;++i)<br>
            {<br>
                int rows[3] = {i,i+1,i+3};<br>
                int cols[3] = {i,i+1,i+3};<br>
                <br>
                MatSetValuesBlocked(A, 3, rows, 3, cols, &insert[0],
        ADD_VALUES);<br>
            }<br>
            <br>
            MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
            MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
            MatDuplicate(A,MAT_COPY_VALUES,&Ah);<br>
            MatDuplicate(A,MAT_COPY_VALUES,&At);<br>
            MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY);<br>
            MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY);<br>
            MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY);<br>
            MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY);<br>
        <br>
            MatAXPY(Ah,1.,At,SAME_NONZERO_PATTERN);<br>
            MatAXPY(A,1.,Ah,SAME_NONZERO_PATTERN);<br>
        <br>
            MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
            MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
        <br>
        The result is, that only some values are computed correctly.<br>
        <br>
        Best and Thank you a lot<br>
        <br>
        Klaus<br>
        <br>
        <br>
        <br>
        On 03/27/2015 04:26 PM, Matthew Knepley wrote:<br>
      </div>
      <blockquote
cite="mid:CAMYG4G=OR3=gMHxAf2NMQCkkh7PpPDRBaoFZ+AeamiKaE2xapw@mail.gmail.com"
        type="cite">
        <div dir="ltr">
          <div class="gmail_extra">
            <div class="gmail_quote">On Fri, Mar 27, 2015 at 10:16 AM,
              Klaus Kaiser <span dir="ltr"><<a
                  moz-do-not-send="true"
                  href="mailto:kaiser@igpm.rwth-aachen.de"
                  target="_blank">kaiser@igpm.rwth-aachen.de</a>></span>
              wrote:<br>
              <blockquote class="gmail_quote" style="margin:0 0 0
                .8ex;border-left:1px #ccc solid;padding-left:1ex">
                <div bgcolor="#FFFFFF" text="#000000">
                  <div>Hallo Matthew,<br>
                    <br>
                    thanks for your fast response. With "no beside the
                    nonzero structure" I meant, that I do not create a
                    different non-zero structure while I'm adding values
                    to my matrices.<br>
                    <br>
                    I also tried MAT_COPY_VALUES and the result was the
                    same.<br>
                  </div>
                </div>
              </blockquote>
              <div><br>
              </div>
              <div>Please send the small example and we will figure out
                what is going wrong.</div>
              <div><br>
              </div>
              <div>  Thanks,</div>
              <div><br>
              </div>
              <div>     Matt</div>
              <div> </div>
              <blockquote class="gmail_quote" style="margin:0 0 0
                .8ex;border-left:1px #ccc solid;padding-left:1ex">
                <div bgcolor="#FFFFFF" text="#000000">
                  <div> I do not get a error message but the resulting
                    matrix is wrong. Here I have a short example. The
                    first 9 rows and first 3 colomn of the matrices Ah,
                    At and Ah+At on the first processor (of 8):<br>
                    <br>
                    Ah: <br>
                    0:                   60.3553               
                    -0.249975              2.77556e-17<br>
                    1:                         0                 
                    60.3553                        0<br>
                    2:                         0                       
                    0                  60.3553<br>
                    3:                   17.6777                
                    0.374962                 0.124987<br>
                    4:                         0                 
                    17.6777                        0<br>
                    5:                         0                       
                    0                  17.6777<br>
                    6:                  -7.32233               
                    -0.124987                 0.374962<br>
                    7:                         0                
                    -7.32233                        0<br>
                    8:                         0                       
                    0                 -7.32233<br>
                    <br>
                    <br>
                    At:<br>
                    0:                         0                       
                    0                        0<br>
                    1:                      2500                       
                    0                        0<br>
                    2:              -4.54747e-13                       
                    0                        0<br>
                    3:                         0                       
                    0                        0<br>
                    4:                      1250                       
                    0                        0<br>
                    5:                      1250                       
                    0                        0<br>
                    6:                         0                       
                    0                        0<br>
                    7:                      1250                       
                    0                        0<br>
                    8:                      3750                       
                    0                        0<br>
                    <br>
                    Ah+At<br>
                    0:                   60.3553               
                    -0.249975              2.77556e-17<br>
                    1:                      2500                 
                    60.3553                        0<br>
                    2:              -4.54747e-13                       
                    0                  60.3553<br>
                    3:                   17.6777                
                    0.374962                 0.124987<br>
                    4:                         0                 
                    17.6777                        0<br>
                    5:                         0                       
                    0                  17.6777<br>
                    6:                  -7.32233               
                    -0.124987                 0.374962<br>
                    7:                         0                
                    -7.32233                        0<br>
                    8:                         0                       
                    0                 -7.32233<br>
                    <br>
                    you can see the first 3 rows of the resulting matrix
                    looks exactly like what I would expect, but the last
                    6 rows only consists of the values of Ah. When you
                    would also look on the matrix A and A+Ah+At you
                    would also see, that values where both matrices have
                    an nonzero entry are not sum correctly.<br>
                    <br>
                    Best<br>
                    <br>
                    Klaus<br>
                    <br>
                    On 03/27/2015 03:59 PM, Matthew Knepley wrote:<br>
                  </div>
                  <blockquote type="cite">
                    <div dir="ltr">
                      <div class="gmail_extra">
                        <div class="gmail_quote">On Fri, Mar 27, 2015 at
                          9:48 AM, Klaus Kaiser <span dir="ltr"><<a
                              moz-do-not-send="true"
                              href="mailto:kaiser@igpm.rwth-aachen.de"
                              target="_blank">kaiser@igpm.rwth-aachen.de</a>></span>
                          wrote:<br>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex">Hallo,<br>
                            <br>
                            I have a strange behavior in my code
                            concerning the function MatAXPY. I create 3
                            different Matrices<br>
                            <br>
                                ierr = MatCreateBAIJ(PETSC_COMM_WORLD,
                            block_size, local_size, local_size,
                            system_size, system_size, 0, d_nnz, 0,
                            o_nnz,&A);<br>
                                ierr =
                            MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
                                ierr =
                            MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
                                ierr =
                            MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
                            <br>
                                ierr = MatCreateBAIJ(PETSC_COMM_WORLD,
                            block_size, local_size, local_size,
                            system_size, system_size, 0, d_nnz, 0,
                            o_nnz,&At);<br>
                                ierr =
                            MatSetOption(At,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
                                ierr =
                            MatSetOption(At,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
                                ierr =
                            MatSetOption(At,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
                            <br>
                                ierr = MatCreateBAIJ(PETSC_COMM_WORLD,
                            block_size, local_size, local_size,
                            system_size, system_size, 0, d_nnz, 0,
                            o_nnz,&Ah);<br>
                                ierr =
                            MatSetOption(Ah,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);<br>
                                ierr =
                            MatSetOption(Ah,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);<br>
                                ierr =
                            MatSetOption(Ah,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);<br>
                          </blockquote>
                          <div><br>
                          </div>
                          <div>These creations are superfluous since you
                            use MatDuplicate() below.</div>
                          <div> </div>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex"> and want to
                            sum these three matrixes with different
                            factors. First I fill the Matrix A with some
                            values, and duplicate the structure of A to
                            the other two matrices:<br>
                            <br>
                                MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);<br>
                                MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);<br>
                               
                            MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&Ah);<br>
                               
                            MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&At);<br>
                                MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY);<br>
                                MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY);<br>
                                MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY);<br>
                                MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY);<br>
                          </blockquote>
                          <div> <br>
                          </div>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex"> After this I
                            fill the matrices At and Ah with some other
                            values, which are not beside the non zero
                            structure (I also tried with just copying
                            the Matrix A). Now after another MatAssembly</blockquote>
                          <div><br>
                          </div>
                          <div>I do not understand "no beside the
                            nonzero structure". Do you mean that the
                            nonzero structure is the same?</div>
                          <div><br>
                          </div>
                          <div>Can you first test with MAT_COPY_VALUES?</div>
                          <div> </div>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex"> I want to add
                            these Matrices in the form A+c*(Ah+d*At): </blockquote>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex"> <br>
                                MatAXPY(Ah,c,At,SAME_NONZERO_PATTERN);<br>
                                MatAXPY(A,d,Ah,SAME_NONZERO_PATTERN);<br>
                            <br>
                            When I run the method with mpi and one core
                            everything works fine. Starting the same
                            method with more cores, the sum of the
                            matrices fails. It seems like some values
                            are added</blockquote>
                          <div><br>
                          </div>
                          <div>Please send the full output of a failure
                            when you use MAT_COPY_VALUES on 2 procs.</div>
                          <div><br>
                          </div>
                          <div>  Thanks,</div>
                          <div><br>
                          </div>
                          <div>     Matt</div>
                          <div> </div>
                          <blockquote class="gmail_quote"
                            style="margin:0 0 0 .8ex;border-left:1px
                            #ccc solid;padding-left:1ex"> correctly and
                            many values are missed. Using
                            DIFFERENT_NONZERO_STRUCTURE leads to the
                            right behavior in the multi-core case, but
                            is very slow. I checked with a viewer if all
                            matrices have the same nonzero structure and
                            this is the case.<br>
                            <br>
                            Does anyone know why this fails, or do I
                            have made any wrong thoughts?<br>
                            <br>
                            I'm corrently working with a petsc version
                            (Petsc Release Version 3.3.0, Patch 5, Sat
                            Dec  1 15:10:41 CST 2012), I looked into the
                            changelogs up to the current version and did
                            not find any note about MatAXPY or MatAYPX.<br>
                            <br>
                            <br>
                            Best and Thanks a lot for your help<span><font
                                color="#888888"><br>
                                <br>
                                Klaus<br>
                                <br>
                              </font></span></blockquote>
                        </div>
                        <br>
                        <br clear="all">
                        <span class="HOEnZb"><font color="#888888">
                            <div><br>
                            </div>
                            -- <br>
                            <div>What most experimenters take for
                              granted before they begin their
                              experiments is infinitely more interesting
                              than any results to which their
                              experiments lead.<br>
                              -- Norbert Wiener</div>
                          </font></span></div>
                    </div>
                  </blockquote>
                  <br>
                </div>
              </blockquote>
            </div>
            <br>
            <br clear="all">
            <div><br>
            </div>
            -- <br>
            <div class="gmail_signature">What most experimenters take
              for granted before they begin their experiments is
              infinitely more interesting than any results to which
              their experiments lead.<br>
              -- Norbert Wiener</div>
          </div>
        </div>
      </blockquote>
      <br>
    </blockquote>
    <br>
  </body>
</html>