On Thu, Jun 18, 2009 at 10:48 AM, Christian Klettner <span dir="ltr"><<a href="mailto:christian.klettner@ucl.ac.uk">christian.klettner@ucl.ac.uk</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi Matt,<br>
Thank you for the reply. I have now gotten MUMPS to work with the<br>
following implementation:<br>
<br>
MatFactorInfoInitialize(&info);<br>
MatGetFactor(A,MAT_SOLVER_MUMPS,MAT_FACTOR_LU,&B);<br>
MatLUFactorSymbolic(B,A, rows_ck_is, cols_ck_is,&info);<br>
MatLUFactorNumeric(B,A,&info);<br>
<br>
for(){ ///temporal loop<br>
/*coding*/<br>
<div class="im">ierr=MatSolve(A,vecr,vecu);CHKERRQ(ierr);<br>
}<br>
<br>
</div>I explicitly use MUMPS to factor the matrix. However, from my<br>
implementation I do not see what package is being used to solve Ax=b.<br>
Could I be using PETSc to solve the system?? Is there any way of knowing<br>
if MUMPS is actually being used to solve the system?<br>
I am not passing in any arguments to use MUMPS explicitly from the command<br>
line, have<br>
not put in any MUMPS header files and I do not use any other MUMPS<br>
functions (e.g. MatCreate_AIJMUMPS()).</blockquote><div><br>Yes, it is determined from the type. However, I think this is completely the wrong way<br>to program. Why hardcode yourself into a corner? It is hard to change packages, to<br>
wrap this in a Krylov solver, compare with other PCs. This is crazy. I recommend<br><br> KSPCreate(comm, &ksp);<br> KSPSetOperators(ksp, A, A, ...);<br> KSPSetFromOptions(ksp);<br> KSPSolve(ksp, b, x);<br><br>Now with options you can control everything<br>
<br> -ksp_type preonly -pc_type lu -mat <b>-pc_factor_mat_solver_package mumps</b><br> <br>will reproduce what you have, but you can test it against<br><br> -ksp_type gmres -pc_type asm -sub_pc_type ilu<br><br>To see what is happening you can always use<br>
<br> -ksp_view<br><br> Matt<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
Thank you for any advice,<br>
Best regards,<br>
<font color="#888888">Christian Klettner<br>
</font><div><div></div><div class="h5"><br>
<br>
> On Tue, Jun 16, 2009 at 7:40 AM, Christian Klettner <<br>
> <a href="mailto:christian.klettner@ucl.ac.uk">christian.klettner@ucl.ac.uk</a>> wrote:<br>
><br>
>> Hi Matt,<br>
>> I have tried to use MUMPS and came across the following problems. My<br>
>> application solves Ax=b about 10000-100000 times with A remaining<br>
>> constant.<br>
>> When I tried to use it through KSP I was not finding good performance.<br>
>> Could this be because it was refactoring etc. at each time step?<br>
>> With this in mind I have tried to implement the following:<br>
><br>
><br>
> 1) I have no idea what you would mean by good performance. Always, always,<br>
> always<br>
> send the -log_summary.<br>
><br>
> 2) A matrix is never refactored during the KSP iteration<br>
><br>
><br>
>> A is a parallel matrix created with MatCreateMPIAIJ().<br>
>> rows is a list of the global row numbers on the process.cols is a list<br>
>> of<br>
>> the global columns that are on the process.<br>
>> I run the program with ./mpiexec -n 4 ./ex4<br>
>> -pc_factor_mat_solver_package<br>
>> mumps<br>
>><br>
>> (1) MatFactorInfo *info;<br>
>> (2) ierr=MatFactorInfoInitialize(info);CHKERRQ(ierr);<br>
><br>
><br>
> You have passed a meaningless pointer here.<br>
><br>
> (1) MatFactorInfo info;<br>
> (2) ierr=MatFactorInfoInitialize(&info);CHKERRQ(ierr);<br>
><br>
> Matt<br>
><br>
><br>
>> (3)<br>
>> ierr=MatGetFactor(A,MAT_SOLVER_MUMPS,MAT_FACTOR_LU,&F);CHKERRQ(ierr);<br>
>> (4) ierr=MatLUFactorSymbolic(F, A, rows, cols,info);CHKERRQ(ierr);<br>
>> (5) MatLUFactorNumeric(F, A,info);CHKERRQ(ierr);<br>
>><br>
>><br>
>> for(){ ///TEMPORAL LOOP<br>
>><br>
>> /*CODING*/<br>
>><br>
>> ierr=MatSolve(A,vecr,vecu);CHKERRQ(ierr);<br>
>> }<br>
>><br>
>> I get the following error messages:<br>
>><br>
>> Line (2) above gives the following error message:<br>
>><br>
>> [0]PETSC ERROR: --------------------- Error Message<br>
>> ------------------------------------<br>
>> [2]PETSC ERROR: [0]PETSC ERROR: Null argument, when expecting valid<br>
>> pointer!<br>
>> [0]PETSC ERROR: Trying to zero at a null pointer!<br>
>> [0]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [0]PETSC ERROR: Petsc Release Version 3.0.0, Patch 4, Fri Mar 6<br>
>> 14:46:08<br>
>> CST 2009<br>
>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>
>> [0]PETSC ERROR: See docs/index.html for manual pages.<br>
>> [0]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [0]PETSC ERROR: ./ex4 on a linux-gnu named christian-desktop by<br>
>> christian<br>
>> Tue Jun 16 13:33:33 2009<br>
>> [0]PETSC ERROR: Libraries linked from<br>
>> /home/christian/Desktop/petsc-3.0.0-p4/linux-gnu-c-debug/lib<br>
>> [0]PETSC ERROR: Configure run at Mon Jun 15 17:05:31 2009<br>
>> [0]PETSC ERROR: Configure options --with-cc="gcc -fPIC"<br>
>> --download-mpich=1<br>
>> --download-f-blas-lapack --download-scalapack --download-blacs<br>
>> --download-mumps --download-parmetis --download-hypre<br>
>> --download-triangle<br>
>> --with-shared=0<br>
>> [0]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [0]PETSC ERROR: PetscMemzero() line 189 in src/sys/utils/memc.c<br>
>> [0]PETSC ERROR: MatFactorInfoInitialize() line 7123 in<br>
>> src/mat/interface/matrix.c<br>
>> [0]PETSC ERROR: main() line 1484 in src/dm/ao/examples/tutorials/ex4.c<br>
>> application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0[cli_0]:<br>
>> aborting job:<br>
>> application called MPI_Abort(MPI_COMM_WORLD, 85) - process 0<br>
>> --------------------- Error Message ------------------------------------<br>
>> [2]PETSC ERROR: Null argument, when expecting valid pointer!<br>
>> [2]PETSC ERROR: Trying to zero at a null pointer!<br>
>> [2]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [2]PETSC ERROR: Petsc Release Version 3.0.0, Patch 4, Fri Mar 6<br>
>> 14:46:08<br>
>> CST 2009<br>
>> [2]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
>> [2]PETSC ERROR: See docs/faq.html[0]0:Return code = 85<br>
>> [0]1:Return code = 0, signaled with Interrupt<br>
>> [0]2:Return code = 0, signaled with Interrupt<br>
>> [0]3:Return code = 0, signaled with Interrupt<br>
>><br>
>> /////////////////////////////////////////////////////////////////<br>
>><br>
>> Line (3) gives the following error messages:<br>
>><br>
>> [0]PETSC ERROR: --------------------- Error Message<br>
>> ------------------------------------<br>
>> [0]PETSC ERROR: Invalid argument!<br>
>> [0]PETSC ERROR: Wrong type of object: Parameter # 2!<br>
>> [0]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [0]PETSC ERROR: [2]PETSC ERROR: --------------------- Error Message<br>
>> ------------------------------------<br>
>> [2]PETSC ERROR: Invalid argument!<br>
>> [2]PETSC ERROR: Wrong type of object: Parameter # 2!<br>
>> [2]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [2]PETSC ERROR: Petsc Release Version 3.0.0, Patch 4, Fri Mar 6<br>
>> 14:46:08<br>
>> CST 2009<br>
>> [2]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
>> [2]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>
>> [2]PETSC ERROR: See docs/index.html for manual pages.<br>
>> [2]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [2]PETSC ERROR: ./ex4 on a linux-gnu named christian-desktop by<br>
>> christian<br>
>> Tue Jun 16 13:37:36 2009<br>
>> [2]PETSC ERROR: Libraries linked from<br>
>> /home/christian/Desktop/petsc-3.0.0-p4/linux-gnu-c-debug/lib<br>
>> [2]PETSC ERROR: Configure run at Mon Jun 15 17:05:31 2009<br>
>> [2]PETSC ERROR: Configure options --with-cc="gcc -fPIC"<br>
>> --download-mpich=1<br>
>> --download-f-blas-lapack --download-scalapack --download-blacs<br>
>> --downlPetsc Release Version 3.0.0, Patch 4, Fri Mar 6 14:46:08 CST<br>
>> 2009<br>
>> [0]PETSC ERROR: See docs/changes/index.html for recent updates.<br>
>> [0]PETSC ERROR: See docs/faq.html for hints about trouble shooting.<br>
>> [0]PETSC ERROR: See docs/index.html for manual pages.<br>
>> [0]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [0]PETSC ERROR: ./ex4 on a linux-gnu named christian-desktop by<br>
>> christian<br>
>> Tue Jun 16 13:37:36 2009<br>
>> [0]PETSC ERROR: Libraries linked from<br>
>> /home/christian/Desktop/petsc-3.0.0-p4/linux-gnu-c-debug/lib<br>
>> [0]PETSC ERROR: Configure run at Mon Jun 15 17:05:31 2009<br>
>> [0]PETSC ERROR: Configure options --with-cc="gcc -fPIC"<br>
>> --download-mpich=1<br>
>> --download-f-blas-lapack --download-scalapack --download-blacs<br>
>> --download-mumps --download-parmetis --download-hypre<br>
>> --download-triangle<br>
>> --with-shared=0<br>
>> [0]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [0]PETSC ERROR: MatLUFactorSymbolic() line 2311 in<br>
>> src/mat/interface/matrix.c<br>
>> [0]PETSC ERROR: main() line 148oad-mumps --download-parmetis<br>
>> --download-hypre --download-triangle --with-shared=0<br>
>> [2]PETSC ERROR:<br>
>> ------------------------------------------------------------------------<br>
>> [2]PETSC ERROR: MatLUFactorSymbolic() line 2311 in<br>
>> src/mat/interface/matrix.c<br>
>> [2]PETSC ERROR: main() line 1488 in src/dm/ao/examples/tutorials/ex4.c<br>
>> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 2[cli_2]:<br>
>> aborting job:<br>
>> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 2<br>
>> 8 in src/dm/ao/examples/tutorials/ex4.c<br>
>> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0[cli_0]:<br>
>> aborting job:<br>
>> application called MPI_Abort(MPI_COMM_WORLD, 62) - process 0<br>
>> [0]0:Return code = 62<br>
>> [0]1:Return code = 0, signaled with Interrupt<br>
>> [0]2:Return code = 62<br>
>> [0]3:Return code = 0, signaled with Interrupt<br>
>><br>
>> Is the matrix (ie parameter 2) in teh wrong state because it's not a<br>
>> MUMPS<br>
>> matrix?<br>
>><br>
>> Any help would be greatly appreciated,<br>
>> Best regards,<br>
>> Christian Klettner<br>
>><br>
>><br>
>><br>
>> > The problem is small enough that you might be able to use MUMPS.<br>
>> ><br>
>> > Matt<br>
>> ><br>
>> > On Fri, Jun 12, 2009 at 9:31 AM, Lisandro Dalcin <<a href="mailto:dalcinl@gmail.com">dalcinl@gmail.com</a>><br>
>> > wrote:<br>
>> ><br>
>> >> On Fri, Jun 12, 2009 at 11:13 AM, Christian<br>
>> >> Klettner<<a href="mailto:christian.klettner@ucl.ac.uk">christian.klettner@ucl.ac.uk</a>> wrote:<br>
>> >> > Sorry that I sent this twice. No subject in the first one.<br>
>> >> ><br>
>> >> > Dear PETSc Team,<br>
>> >> > I am writing a CFD finite element code in C. From the<br>
>> discretization<br>
>> >> of<br>
>> >> > the governing equations I have to solve a Poisson type equation<br>
>> which<br>
>> >> is<br>
>> >> > really killing my performance. Which solver/preconditioner from<br>
>> PETSc<br>
>> >> or<br>
>> >> > any external packages would you recommend? The size of my problem<br>
>> is<br>
>> >> from<br>
>> >> > ~30000-100000 DOF per core. What kind of performance would I be<br>
>> able<br>
>> >> to<br>
>> >> > expect with this solver/preconditioner?<br>
>> >><br>
>> >> I would suggest KSPCG. As preconditioner I would use ML or<br>
>> >> HYPRE/BoomerAMG (both are external packages)<br>
>> >><br>
>> >> > I am using a 2*quad core 2.3 GHz Opteron. I have decomposed the<br>
>> domain<br>
>> >> > with Parmetis. The mesh is unstructured.<br>
>> >> > Also, I am writing a code which studies free surface phenomena so<br>
>> the<br>
>> >> mesh<br>
>> >> > is continually changing. Does this matter when choosing a<br>
>> >> > solver/preconditioner? My left hand side matrix (A in Ax=b) does<br>
>> not<br>
>> >> > change in time.<br>
>> >><br>
>> >> ML has a faster setup that BoomerAMG, but the convergence is a bit<br>
>> >> slower. If your A matrix do not change, then likely BoomerAMG will be<br>
>> >> better for you. In any case, you can try both: just build PETSc with<br>
>> >> both packages, then you can change the preconditioner by just passing<br>
>> >> a command line option.<br>
>> >><br>
>> >> ><br>
>> >> > Best regards and thank you in advance,<br>
>> >> > Christian Klettner<br>
>> >> ><br>
>> >><br>
>> >> Disclaimer: the convergence of multigrid preconditioners depends a<br>
>> lot<br>
>> >> on your actual problem. What I've suggested is just my limited<br>
>> >> experience in a few problems I've run solving electric potentials.<br>
>> >><br>
>> >><br>
>> >> --<br>
>> >> Lisandro Dalcín<br>
>> >> ---------------<br>
>> >> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)<br>
>> >> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)<br>
>> >> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)<br>
>> >> PTLC - Güemes 3450, (3000) Santa Fe, Argentina<br>
>> >> Tel/Fax: +54-(0)342-451.1594<br>
>> >><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> > What most experimenters take for granted before they begin their<br>
>> > experiments<br>
>> > is infinitely more interesting than any results to which their<br>
>> experiments<br>
>> > lead.<br>
>> > -- Norbert Wiener<br>
>> ><br>
>><br>
>><br>
>><br>
><br>
><br>
> --<br>
> What most experimenters take for granted before they begin their<br>
> experiments<br>
> is infinitely more interesting than any results to which their experiments<br>
> lead.<br>
> -- Norbert Wiener<br>
><br>
<br>
<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>