[petsc-users] MatSetValues timing

[Dave May]

On 8 February 2016 at 12:31, Jacek Miloszewski <jacek.miloszewski at gmail.com>
wrote:

> Dear PETSc users,
>
> I use PETSc to assembly a square matrix (in the attached example it is n =
> 4356) which has around 12% of non-zero entries. I timed my code using
> various number of process (data in table). Now I have 2 questions:
>
> 1. Why with doubling number of processes the speed-up is 4x? I would
> expect 2x at the most.
>
> 2. Is there a way to speed-up matrix construction in general? I attach
> piece of my fortran code at the bottom. At compilation time I have the
> following knowledge about the matrix: the total number of non-zero matrix
> elements, all diagonal elements are non-zero.
>

Performance improvements of the assembly require the preallocation of the
matrix be specified.
Check out these pages for further information

http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatSeqAIJSetPreallocation.html

http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatMPIAIJSetPreallocation.html

also note the FAQ item

"Assembling large sparse matrices takes a long time. What can I do make
this process faster? or MatSetValues() is so slow, what can I do to speed
it up?"

http://www.mcs.anl.gov/petsc/documentation/faq.html


Thanks,
  Dave


> Timing data:
>
> number of proc time [s] speedup ratio
> 1 2044.941
> 2 504.692 4.051859352
> 4 149.678 3.371851575
> 8 64.102 2.334997348
> 16 17.296 3.706174838
> 32 4.43 3.904288939
> 64 1.096 4.041970803
>
> Code:
>
> call MatCreate(PETSC_COMM_WORLD, a, ierr)
> call MatSetSizes(a, PETSC_DECIDE, PETSC_DECIDE, n, n, ierr)
> call MatSetFromOptions(a, ierr)
> call MatSetUp(a, ierr)
>
> call MatGetOwnershipRange(a, Istart, Iend, ierr)
>
> call system_clock(t1)
> t_dim = h_dim*e_dim
> do row = Istart, Iend - 1 ! row
>    do col = 0, t_dim - 1 ! col
>       call h_ij(row + 1, col + 1, n_h, n_e, b_h, b_e, h_dim, e_dim, e_sp,
> v, basis, ht, hs, info_1)
>       if (hs) then
>          hh = ht
>          call MatSetValues(a, 1, row, 1, col, hh, INSERT_VALUES, ierr)
>       end if
>    end do
> end do
> call system_clock(t2, ct)
> if (rank == 0) then
>    write(*, '(a, f0.3)') 'Matrix assembly time: ', real((t2 - t1),
> r8)/real(ct, r8)
> end if
>
> --
> Best Wishes
> Jacek Miloszewski
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160208/86570678/attachment.html>