[petsc-users] Using direct solvers in parallel

[Dave May]

Ah okay. Thanks for the timings.

Have you monitored the CPU usage when you using umfpack?
On my machine, it's definitely not running on a single process,
so I wouldn't consider it a sequential solver.




On 15 May 2012 09:54, Thomas Witkowski <thomas.witkowski at tu-dresden.de> wrote:
> Am 15.05.2012 09:36, schrieb Dave May:
>
>> I have seem similar behaviour comparing umfpack and superlu_dist,
>> however the difference wasn't enormous, possibly umfpack was a factor
>> of 1.2-1.4 times faster on 1 - 4 cores.
>> What sort of time differences are you observing? Can you post the
>> numbers somewhere?
>
> I attached my data to this mail. For the largest matrix, umfpack failed
> after allocating 4 GB of memory. I have not tried to figure out what's the
> problem there. As you can see, for these matrices the distributed solvers
> are slower by a factor of 2 or 3 compared to umfpack. For all solvers, I
> have used the standard parameters, so I have not played around with the
> permutation strategies and such things. This may be also the reason why
> superlu is much slower than superlu_dist with just one core as it makes use
> of different col and row permutation strategies.
>
>> However, umpack will not work on a distributed memory machine.
>> My personal preference is to use superlu_dist in parallel. In my
>> experience using it as a coarse grid solver for multigrid, I find it
>> much more reliable than mumps. However, when mumps works, its is
>> typically slightly faster than superlu_dist. Again, not by a large
>> amount - never more than a factor of 2 faster.
>
> In my codes I also make use of the distributed direct solvers for the coarse
> grid problems. I just wanted to make some tests how far away these solvers
> are from the sequential counterparts.
>
> Thomas
>
>>
>> The failure rate using mumps is definitely higher (in my experience)
>> when running on large numbers of cores compared to superlu_dist. I've
>> never got to the bottom as to why it fails.
>>
>> Cheers,
>>   Dave
>>
>>
>> On 15 May 2012 09:25, Thomas Witkowski<thomas.witkowski at tu-dresden.de>
>>  wrote:
>>>
>>> I made some comparisons of using umfpack, superlu, superlu_dist and mumps
>>> to
>>> solve systems with sparse matrices arising from finite element method.
>>> The
>>> size of the matrices range from around 50000 to more than 3 million
>>> unknowns. I used 1, 2, 4, 8 and 16 nodes to make the benchmark. Now, I
>>> wonder that in all cases the sequential umfpack was the fastest one. So
>>> even
>>> with 16 cores, superlu_dist and mumps are slower. Can anybody of you
>>> confirm
>>> this observation? Are there any other parallel direct solvers around
>>> which
>>> are more efficient?
>>>
>>> Thomas
>
>