Load Balancing and KSPSolve

[Tim Stitt]

Satish,

Logs attached...hope they help.

Thanks,

Tim.

Satish Balay wrote:
> Can you send the -log_summary for your runs [say p=1, p=8]
>
> Satish
>
> On Tue, 20 Nov 2007, Tim Stitt wrote:
>
>   
>> Hi all (again),
>>
>> I finally got some data back from the KSP PETSc code that I put together to
>> solve this sparse inverse matrix problem I was looking into. Ideally I am
>> aiming for a O(N) (time complexity) approach to getting the first 'k' columns
>> of the inverse of a sparse matrix.
>>
>> To recap the method: I have my solver which uses KSPSolve in a loop that
>> iterates over the first k columns of an identity matrix B and computes the
>> corresponding x vector.
>>
>> I am just a bit curious about some of the timings I am obtaining...which I
>> hope someone can explain. Here are the timings I obtained for a global sparse
>> matrix (4704 x 4704) and solving for the first 1176 columns in the identity
>> using P processes (processors) on our cluster.
>>
>> (Timings are given in seconds for each process performing work in the loop and
>> were obtained by encapsulating the loop with the cpu_time() Fortran intrinsic.
>> The MUMPS package was requested for factorisation/solving, although similar
>> timings were obtained for both the native solver and SUPERLU)
>>
>> P=1  [30.92]
>> P=2  [15.47, 15.54]
>> P=4  [4.68, 5.49, 4.67, 5.07]
>> P=8  [2.36, 4,23, 2.81, 2.54, 3.42, 2.22, 1.41, 3.15]
>> P=16 [1.04, 0.45, 1.08, 0.27, 0.87, 0.93, 1.1, 1.06, 0.29, 0.34, 0.73, 0.25,
>> 0.43, 1.09, 1.08, 1.1]
>>
>> Firstly, I notice very good scalability up to 16 processes...is this expected
>> (by those people who use these solvers regularly)?
>>
>> Also I notice that the timings per process vary as we scale up. Is this a
>> load-balancing problem related to more non-zero values being on a given
>> processor than others? Once again is this expected?
>>
>> Please excuse my ignorance of matters relating to these solvers and their
>> operation...as it really isn't my field of expertise.
>>
>> Regards,
>>
>> Tim.
>>
>>
>>     
>
>   


-- 
Dr. Timothy Stitt <timothy_dot_stitt_at_ichec.ie>
HPC Application Consultant - ICHEC (www.ichec.ie)

Dublin Institute for Advanced Studies
5 Merrion Square - Dublin 2 - Ireland

+353-1-6621333 (tel) / +353-1-6621477 (fax)

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: log.1
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20071120/6325b7c9/attachment.diff>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: log.8
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20071120/6325b7c9/attachment-0001.diff>