[petsc-users] How to get bandwidth peak out of PETSc log ?

[HOUSSEN Franck]

Hello,

I am new to PETSc.

I have written a (MPI) PETSc code to solve an AX=B system : how to know the bandwidth peak for a given run ? The code does not scale as I would expect (doubling the number of MPI processes does not half the elapsed time) : I would like to understand if this behavior is related to a bad MPI parallelization (that I may be able to improve), or, to the fact that the bandwidth limit as been reached (and in this case, my understanding is that I can not do anything to improve neither the performance, nor the scaling). I would like to know what's going on and why !

Concerning the computer, I have tried to estimate the bandwidth peak with the "stream benchmark" : www.streambench.org/index.html. I get this :
~>./stream_c.exe
...
Function    Best Rate MB/s  Avg time     Min time     Max time
Copy:           11473.8     0.014110     0.013945     0.015064
Scale:          11421.2     0.014070     0.014009     0.014096
Add:            12974.4     0.018537     0.018498     0.018590
Triad:          12964.6     0.018683     0.018512     0.019277
As a conclusion, my understanding is that the bandwidth peak of my computer is about 12 200 MB / s (= average between 11 400 and 12 900) which is about 12200 / 1024 = 11.9 GB/s.

Concerning PETSc, I tried to find (without success) a figure to compare to 11.9 GB/s. First, I tried to run "make streams" but it doesn't compile (I run petsc3.4.1. on Ubuntu 12.04). Then, I looked into the PETSc log (using -log_summary and -ksp_view) but I was not able to find out an estimate of the bandwidth out of it (I get information about time and flops but not about the amount of data transfered between MPI processes in MB). How can I get bandwidth peak out of PETSc log ? Is there a specific option for that or is this not possible ?

I have attached the PETSc log (MPI run over 2 processes).

Thanks,

FH
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130620/32035d9a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PETSc.log
Type: text/x-log
Size: 20458 bytes
Desc: PETSc.log
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20130620/32035d9a/attachment-0001.bin>