<div dir="ltr">Dear Matthew Knepley<div><br></div><div>However, I've already installed ScaLAPACK.</div><div><div>cd $PETSC_DIR</div><div>./configure --download-mpich --with-debugging=0 COPTFLAGS='-O3 -march=native -mtune=native' CXXOPTFLAGS='-O3 -march=native -mtune=native' FOPTFLAGS='-O3 -march=native -mtune=native' --download-mumps --<b>download-scalapack</b> --download-parmetis --download-metis --download-parmetis --download-hpddm --download-slepc</div></div><div><br></div><div>Is there some way to use ScaLAPCK?</div><div>Or, Can I run the part related to MatMatSolve with a single core? </div><div><br></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">2023년 5월 5일 (금) 오후 6:21, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>>님이 작성:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Fri, May 5, 2023 at 3:49 AM 권승리 / 학생 / 항공우주공학과 <<a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear Barry Smith<div><br></div><div>Thanks to you, I knew the difference between MATAIJ and MATDENSE.</div><div><br></div><div>However, I still have some problems.</div><div><br></div><div>There is no problem when I run with a single core. But, MatGetFactor error occurs when using multi-core.</div><div><br></div><div>Could you give me some advice?</div><div><br></div><div>The error message is</div><div><br></div><div>[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[0]PETSC ERROR: See <a href="https://petsc.org/release/overview/linear_solve_table/" target="_blank">https://petsc.org/release/overview/linear_solve_table/</a> for possible LU and Cholesky solvers<br>[0]PETSC ERROR: MatSolverType petsc does not support matrix type mpidense<br></div></div></blockquote><div><br></div><div>PETSc uses 3rd party packages for parallel dense factorization. You would need to reconfigure with either ScaLAPACK</div><div>or Elemental.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>[0]PETSC ERROR: See <a href="https://petsc.org/release/faq/" target="_blank">https://petsc.org/release/faq/</a> for trouble shooting.<br>[0]PETSC ERROR: Petsc Release Version 3.18.5, unknown <br>[0]PETSC ERROR: ./app on a arch-linux-c-opt named ubuntu by ksl Fri May 5 00:35:23 2023<br>[0]PETSC ERROR: Configure options --download-mpich --with-debugging=0 COPTFLAGS="-O3 -march=native -mtune=native" CXXOPTFLAGS="-O3 -march=native -mtune=native" FOPTFLAGS="-O3 -march=native -mtune=native" --download-mumps --download-scalapack --download-parmetis --download-metis --download-parmetis --download-hpddm --download-slepc<br>[0]PETSC ERROR: #1 MatGetFactor() at /home/ksl/petsc/src/mat/interface/matrix.c:4757<br>[0]PETSC ERROR: #2 main() at /home/ksl/Downloads/coding_test/coding/a1.c:66<br>[0]PETSC ERROR: No PETSc Option Table entries<br>[0]PETSC ERROR: ----------------End of Error Message -------send entire error message to petsc-maint@mcs.anl.gov----------<br>application called MPI_Abort(MPI_COMM_SELF, 92) - process 0<br></div><div><br></div><div>My code is below:</div><div><br></div><div>int main(int argc, char** args)<br>{<br> Mat A, E, A_temp, A_fac;<br> int n = 15;<br> PetscInitialize(&argc, &args, NULL, NULL);<br> PetscCallMPI(MPI_Comm_size(PETSC_COMM_WORLD, &size));<br><br> PetscCall(MatCreate(PETSC_COMM_WORLD, &A));<br> PetscCall(MatSetType(A,MATDENSE));<br> PetscCall(MatSetSizes(A, PETSC_DECIDE, PETSC_DECIDE, n, n));<br> PetscCall(MatSetFromOptions(A));<br> PetscCall(MatSetUp(A));<br> // Insert values<br> double val;<br> for (int i = 0; i < n; i++) {<br> for (int j = 0; j < n; j++) {<br> if (i == j){<br> val = 2.0;<br> }<br> else{<br> val = 1.0;<br> }<br> PetscCall(MatSetValue(A, i, j, val, INSERT_VALUES));<br> }<br> }<br> PetscCall(MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY));<br> PetscCall(MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY));<br><br> // Make Identity matrix<br> PetscCall(MatCreate(PETSC_COMM_WORLD, &E));<br> PetscCall(MatSetType(E,MATDENSE));<br> PetscCall(MatSetSizes(E, PETSC_DECIDE, PETSC_DECIDE, n, n));<br> PetscCall(MatSetFromOptions(E));<br> PetscCall(MatSetUp(E));<br> PetscCall(MatShift(E,1.0));<br> PetscCall(MatAssemblyBegin(E, MAT_FINAL_ASSEMBLY));<br> PetscCall(MatAssemblyEnd(E, MAT_FINAL_ASSEMBLY));<br> </div><div> PetscCall(MatDuplicate(A, MAT_DO_NOT_COPY_VALUES, &A_temp));<br> PetscCall(MatGetFactor(A, MATSOLVERPETSC, MAT_FACTOR_LU, &A_fac));<br><br> IS isr, isc; MatFactorInfo info;<br> MatGetOrdering(A, MATORDERINGNATURAL, &isr, &isc);<br> PetscCall(MatLUFactorSymbolic(A_fac, A, isr, isc, &info));<br> PetscCall(MatLUFactorNumeric(A_fac, A, &info));<br> MatMatSolve(A_fac, E, A_temp);<br> <br> PetscCall(MatView(A_temp, PETSC_VIEWER_STDOUT_WORLD));<br> MatDestroy(&A);<br> MatDestroy(&A_temp);<br> MatDestroy(&A_fac);<br> MatDestroy(&E);<br> PetscCall(PetscFinalize());<br>}<br></div><div><br></div><div><div>Best regards</div><div>Seung Lee Kwon</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">2023년 5월 4일 (목) 오후 10:19, Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>>님이 작성:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><br></div> The code in ex125.c contains<div><br></div><div><div> PetscCall(MatCreate(PETSC_COMM_WORLD, &C));</div><div> PetscCall(MatSetOptionsPrefix(C, "rhs_"));</div><div> PetscCall(MatSetSizes(C, m, PETSC_DECIDE, PETSC_DECIDE, nrhs));</div><div> PetscCall(MatSetType(C, MATDENSE));</div><div> PetscCall(MatSetFromOptions(C));</div><div> PetscCall(MatSetUp(C));</div><div><br></div><div>This dense parallel matrix is suitable for passing to MatMatSolve() as the right-hand side matrix. Note it is created with PETSC_COMM_WORLD and its type is set to be MATDENSE.</div><div><br></div><div> You may need to make a sample code by stripping out all the excess code in ex125.c to just create an MATAIJ and MATDENSE and solves with MatMatSolve() to determine why you code does not work.</div><div><br></div><div><br></div><div><br><blockquote type="cite"><div>On May 4, 2023, at 3:20 AM, 권승리 / 학생 / 항공우주공학과 <<a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a>> wrote:</div><br><div><div dir="ltr">Dear Barry Smith<div><br></div><div>Thank you for your reply.</div><div><br></div><div>I've already installed MUMPS.</div><div><br></div><div>And I checked the example you said (ex125.c), I don't understand why the RHS matrix becomes the SeqDense matrix.</div><div><br></div><div>Could you explain in more detail?</div><div><br></div><div>Best regards</div><div>Seung Lee Kwon</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">2023년 5월 4일 (목) 오후 12:08, Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>>님이 작성:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><br></div> You can configure with MUMPS ./configure --download-mumps --download-scalapack --download-ptscotch --download-metis --download-parmetis <div><br></div><div> And then use MatMatSolve() as in <span style="color:rgb(200,20,201);font-family:Menlo;font-size:14px">src/mat/tests/ex125.c</span> with parallel MatMatSolve() using MUMPS as the solver.</div><div><br></div><div> Barry</div><div><br><div><br><blockquote type="cite"><div>On May 3, 2023, at 10:29 PM, 권승리 / 학생 / 항공우주공학과 <<a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a>> wrote:</div><br><div><div dir="ltr">Dear developers<div><br></div><div>Thank you for your explanation.</div><div><br></div><div>But I should use the MatCreateSeqDense because I want to use the MatMatSolve that B matrix must be a SeqDense matrix.</div><div><br></div><div>Using MatMatSolve is an inevitable part of my code.</div><div><br></div><div>Could you give me a comment to avoid this error?</div><div><br></div><div>Best,</div><div><br></div><div>Seung Lee Kwon</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">2023년 5월 3일 (수) 오후 7:30, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>>님이 작성:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, May 3, 2023 at 6:05 AM 권승리 / 학생 / 항공우주공학과 <<a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear developers<div><br></div><div>I'm trying to use parallel computing and I ran the command 'mpirun -np 4 ./app'</div><div><br></div><div>In this case, there are two problems.</div><div><br></div><div><b>First,</b> I encountered error message</div><div>///</div><div>[0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<br>[1]PETSC ERROR: Invalid argument<br>[1]PETSC ERROR: Comm must be of size 1</div><div>///<br><div><div>The code on the error position is </div><div>MatCreateSeqDense(PETSC_COMM_SELF, nns, ns, NULL, &Kns));</div></div></div></div></blockquote><div><br></div><div>1) "Seq" means sequential, that is "not parallel".</div><div><br></div><div>2) This line should still be fine since PETSC_COMM_SELF is a serial communicator</div><div><br></div><div>3) You should be checking the error code for each call, maybe using the CHKERRQ() macro</div><div><br></div><div>4) Please always send the entire error message, not a snippet</div><div><br></div><div> THanks</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div>Could "MatCreateSeqDense" not be used in parallel computing?</div><div><br></div><div><b>Second</b>, the same error message is repeated as many times as the number of cores.</div><div>if I use command -np 4, then the error message is repeated 4 times.</div><div>Could you recommend some advice related to this?<br></div><div><br></div><div><div>Best,</div><div>Seung Lee Kwon</div></div><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div>Seung Lee Kwon, Ph.D.Candidate</div><div>Aerospace Structures and Materials Laboratory</div><div>Department of Mechanical and Aerospace Engineering</div><div>Seoul National University</div><div>Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826</div><div>E-mail : <a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a></div><div>Office : +82-2-880-7389</div><div>C. P : +82-10-4695-1062</div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div>Seung Lee Kwon, Ph.D.Candidate</div><div>Aerospace Structures and Materials Laboratory</div><div>Department of Mechanical and Aerospace Engineering</div><div>Seoul National University</div><div>Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826</div><div>E-mail : <a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a></div><div>Office : +82-2-880-7389</div><div>C. P : +82-10-4695-1062</div></div></div>
</div></blockquote></div><br></div></div></blockquote></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div>Seung Lee Kwon, Ph.D.Candidate</div><div>Aerospace Structures and Materials Laboratory</div><div>Department of Mechanical and Aerospace Engineering</div><div>Seoul National University</div><div>Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826</div><div>E-mail : <a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a></div><div>Office : +82-2-880-7389</div><div>C. P : +82-10-4695-1062</div></div></div>
</div></blockquote></div><br></div></div></blockquote></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div>Seung Lee Kwon, Ph.D.Candidate</div><div>Aerospace Structures and Materials Laboratory</div><div>Department of Mechanical and Aerospace Engineering</div><div>Seoul National University</div><div>Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826</div><div>E-mail : <a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a></div><div>Office : +82-2-880-7389</div><div>C. P : +82-10-4695-1062</div></div></div>
</blockquote></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div></div><br clear="all"><div><br></div><span>-- </span><br><div dir="ltr"><div dir="ltr"><div>Seung Lee Kwon, Ph.D.Candidate</div><div>Aerospace Structures and Materials Laboratory</div><div>Department of Mechanical and Aerospace Engineering</div><div>Seoul National University</div><div>Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826</div><div>E-mail : <a href="mailto:ksl7912@snu.ac.kr" target="_blank">ksl7912@snu.ac.kr</a></div><div>Office : +82-2-880-7389</div><div>C. P : +82-10-4695-1062</div></div></div>