[petsc-users] MatSetValues timing

[Matthew Knepley]

On Mon, Feb 8, 2016 at 5:37 AM, Dave May <dave.mayhem23 at gmail.com> wrote:

>
>
> On 8 February 2016 at 12:31, Jacek Miloszewski <
> jacek.miloszewski at gmail.com> wrote:
>
>> Dear PETSc users,
>>
>> I use PETSc to assembly a square matrix (in the attached example it is n
>> = 4356) which has around 12% of non-zero entries. I timed my code using
>> various number of process (data in table). Now I have 2 questions:
>>
>> 1. Why with doubling number of processes the speed-up is 4x? I would
>> expect 2x at the most.
>>
>
> Your timing doesn't appear to include the time required to scatter
> off-processor values.
> You should move the timer to be after the following calls
>
> MatAssemblyBegin(a,MAT_FINAL_ASSEMBLY);
> MatAssemblyEnd(a,MAT_FINAL_ASSEMBLY);
>

Also, I would guess that you are not preallocating the matrix:

  http://www.mcs.anl.gov/petsc/documentation/faq.html#efficient-assembly

    Matt


>
>
>>
>> 2. Is there a way to speed-up matrix construction in general? I attach
>> piece of my fortran code at the bottom. At compilation time I have the
>> following knowledge about the matrix: the total number of non-zero matrix
>> elements, all diagonal elements are non-zero.
>>
>> Timing data:
>>
>> number of proc time [s] speedup ratio
>> 1 2044.941
>> 2 504.692 4.051859352
>> 4 149.678 3.371851575
>> 8 64.102 2.334997348
>> 16 17.296 3.706174838
>> 32 4.43 3.904288939
>> 64 1.096 4.041970803
>>
>> Code:
>>
>> call MatCreate(PETSC_COMM_WORLD, a, ierr)
>> call MatSetSizes(a, PETSC_DECIDE, PETSC_DECIDE, n, n, ierr)
>> call MatSetFromOptions(a, ierr)
>> call MatSetUp(a, ierr)
>>
>> call MatGetOwnershipRange(a, Istart, Iend, ierr)
>>
>> call system_clock(t1)
>> t_dim = h_dim*e_dim
>> do row = Istart, Iend - 1 ! row
>>    do col = 0, t_dim - 1 ! col
>>       call h_ij(row + 1, col + 1, n_h, n_e, b_h, b_e, h_dim, e_dim, e_sp,
>> v, basis, ht, hs, info_1)
>>       if (hs) then
>>          hh = ht
>>          call MatSetValues(a, 1, row, 1, col, hh, INSERT_VALUES, ierr)
>>       end if
>>    end do
>> end do
>> call system_clock(t2, ct)
>> if (rank == 0) then
>>    write(*, '(a, f0.3)') 'Matrix assembly time: ', real((t2 - t1),
>> r8)/real(ct, r8)
>> end if
>>
>> --
>> Best Wishes
>> Jacek Miloszewski
>>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20160208/f944b05b/attachment-0001.html>