<html><head><title>Re: [petsc-users] Is OpenMP still available for PETSc?</title>
</head>
<body>
<span style=" font-family:'courier new'; font-size: 9pt;">Hi,<br>
<br>
Yes, this is exactly what I meant.<br>
Please find attached output for 2 input datasets and for 2 various slurm configs each:<br>
<br>
A/ Matrix size=8000000x8000000<br>
<br>
1/ slurm-14432809.out, 930 ksp steps, ~90 secs<br>
<br>
<br>
#SBATCH --nodes=2<br>
#SBATCH --ntasks=32<br>
#SBATCH --ntasks-per-node=16<br>
#SBATCH --cpus-per-task=4<br>
<br>
2/ slurm-14432810.out , 100.000 ksp steps, ~9700 secs<br>
<br>
#SBATCH --nodes=2<br>
#SBATCH --ntasks=32<br>
#SBATCH --ntasks-per-node=16<br>
#SBATCH --cpus-per-task=2<br>
<br>
<br>
<br>
B/ Matrix size=1000x1000<br>
<br>
1/ slurm-23716.out, 511 ksp steps, ~ 28 secs<br>
#SBATCH --nodes=1<br>
#SBATCH --ntasks=64<br>
#SBATCH --ntasks-per-node=64<br>
#SBATCH --cpus-per-task=4<br>
<br>
<br>
2/ slurm-23718.out, 94 ksp steps, ~ 4 secs<br>
<br>
#SBATCH --nodes=1<br>
#SBATCH --ntasks=4<br>
#SBATCH --ntasks-per-node=4<br>
#SBATCH --cpus-per-task=4<br>
<br>
<br>
I would really appreciate any help...:)<br>
<br>
Best,<br>
Damian<br>
<br>
<br>
<br>
W liście datowanym 3 lipca 2017 (16:29:15) napisano:<br>
<br>
</span><table>
<tr>
<td width=2 bgcolor= #0000ff><br>
</td>
<td><span style=" font-family:'Tahoma'; font-size: 10pt;">On Mon, Jul 3, 2017 at 9:23 AM, Damian Kaliszan <</span><a style=" font-family:'tahoma'; font-size: 10pt;" href="mailto:damian@man.poznan.pl">damian@man.poznan.pl</a><span style=" font-family:'Tahoma'; font-size: 10pt;">> wrote:<br>
Hi,<br>
<br>
<br>
>> 1) You can call Bcast on PETSC_COMM_WORLD<br>
<br>
To be honest I can't find Bcast method in petsc4py.PETSc.Comm (I'm<br>
using petsc4py)<br>
<br>
>> 2) If you are using WORLD, the number of iterates will be the same on each process since iteration is collective.<br>
<br>
Yes, this is how it should be. But what I noticed is that for<br>
different --cpus-per-task numbers in slurm script I get different<br>
number of solver iterations which is in turn related to timings. The<br>
imparity is huge. For example for some configurations where --cpus-per-task=1 I receive 900<br>
iterations and for --cpus-per-task=2 I receive valid number of 100.000 which is set as max<br>
iter number set when setting solver tolerances.<br>
<br>
I am trying to understand what you are saying. You mean that you make 2 different runs and get a different<br>
number of iterates with a KSP? In order to answer questions about convergence, we need to see the output<br>
of<br>
<br>
-ksp_view -ksp_monitor_true_residual -ksp_converged_reason<br>
<br>
for all cases.<br>
<br>
Thanks,<br>
<br>
Matt<br>
<br>
Best,<br>
Damian<br>
<br>
<br>
<br>
<br>
-- <br>
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>
<br>
</span><a style=" font-family:'tahoma'; font-size: 10pt;" href="http://www.caam.rice.edu/~mk51/">http://www.caam.rice.edu/~mk51/</a></td>
</tr>
</table>
<br><br>
<br>
<br>
<br>
<span style=" font-family:'Tahoma'; font-size: 10pt;">-------------------------------------------------------<br>
Damian Kaliszan<br>
<br>
Poznan Supercomputing and Networking Center<br>
HPC and Data Centres Technologies<br>
ul. Jana Pawła II 10<br>
61-139 Poznan<br>
POLAND<br>
<br>
phone (+48 61) 858 5109<br>
e-mail </span><a style=" font-family:'courier new'; font-size: 9pt;" href="mailto:damian@man.poznan.pl">damian@man.poznan.pl</a><br>
<span style=" font-family:'courier new'; font-size: 9pt;">www - </span><a style=" font-family:'courier new'; font-size: 9pt;" href="http://www.man.poznan.pl/">http://www.man.poznan.pl/</a><br>
<span style=" font-family:'courier new'; font-size: 9pt;">------------------------------------------------------- </body></html>