<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>Hi all,<br><br> I am trying to minimize the computing time to solve a large sparse matrix. The matrix dimension is with m=321 n=321 and p=321. I am trying to reduce the computing time from two directions: 1 finding a Pre-conditioner to reduce the number of iterations which reduces the time numerically, 2 requesting more cores.<br><br>----For the first method, I tried several methods:<br> 1 default KSP and PC,<br> 2 -ksp_type fgmres -ksp_gmres_restart 30 -pc_type ksp -ksp_pc_type jacobi, <br> 3 -ksp_type lgmres -ksp_gmres_restart 40 -ksp_lgmres_augment 10,<br> 4 -ksp_type lgmres -ksp_gmres_restart 50 -ksp_lgmres_augment 10,<br> 5 -ksp_type lgmres -ksp_gmres_restart 40 -ksp_lgmres_augment 10 -pc_type asm (PCASM)<br><br>The iterations and timing is like the following with 128 cores requested:<br>case# iter timing (s)<br>1 1436 816 <br>2 3 12658<br>3 1069 669.64<br>4 872 768.12<br>5 927 513.14<br><br>It can be seen that change -ksp_gmres_restart and -ksp_lgmres_augment can help to reduce the iterations but not the timing (comparing case 3 and 4). Second, the PCASM helps a lot. Although the second option is able to reduce iterations, the timing increases very much. Is it because more operations are needed in the PC?<br><br>My questions here are: 1. Which direction should I take to select -ksp_gmres_restart and -ksp_lgmres_augment? For example, if larger restart with large augment is better or larger restart with smaller augment is better? <br><br>----For the second method, I tried with -ksp_type lgmres -ksp_gmres_restart 40 -ksp_lgmres_augment 10 -pc_type asm with different number of cores. I found the speedup ratio increases slowly when more than 32 to 64 cores are requested. I searched the milling list archives and found that I am very likely running into the memory bandwidth bottleneck. <a href="http://www.mail-archive.com/petsc-users@mcs.anl.gov/msg19152.html:" target="_blank">http://www.mail-archive.com/petsc-users@mcs.anl.gov/msg19152.html:</a><br><br># of cores iter timing<br> 1 923 19541.83<br> 4 929 5897.06<br> 8 932 4854.72<br> 16 924 1494.33<br> 32 924 1480.88<br> 64 928 686.89<br>128 927 627.33<br>256 926 552.93<br><br>My question here is: Is there any other PC can help on both reducing iterations and increasing scalability? Thanks. <br><br><br><br> </div></body>
</html>