[petsc-users] Various Questions Regarding PETSC

[Mohammed Mostafa]

Regarding the first point
>
> 1) Are you timing only the insertion of values, or computation and
> insertion?

I am timing both, the computation and insertion of values but as I said I
timed three scenarios
1-computation only and no insertion
Computation_no_insertion                            100 1.0 1.6747e-01 1.2
0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  2  0  0  0  0  22  0  0  0  0     0
2-computation and insertion into petsc matrix

> FillPetscMat_with_MatSetValues              100 1.0 3.8820e+00 1.1
> 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 23  0  0  0  0 * 96 * 0  0  0  0
> 0
>
3-computation and insertion into eigen matrix

> FilEigenMat                                               100 1.0
> 2.8727e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 18  0  0  0  0  88  0
>  0  0  0     0

I timed 100 times to get a reasonably accurate timings

as for the second point

>  2) Can you tell me how many values are inserted?

For a total of nearly 186062 rows per process (with  6 processes in total,
the matrix global size is 1116376)
In most rows ( about 99.35%)  4 non-zeros per rows and in the remaining
0.35% 2 or 3 non-zeros per row
the number of off-diagonal onnz in total is 648 nnz
So I insert nearly 4 values 186062 times ~= 744248 times per mpi process


Thanks,
Kamra

On Wed, Jul 17, 2019 at 11:59 PM Matthew Knepley <knepley at gmail.com> wrote:

> On Wed, Jul 17, 2019 at 8:51 AM Mohammed Mostafa <
> mo7ammedmostafa at gmail.com> wrote:
>
>> Sorry for the confusion
>> First I fully acknowledge that setting Matrix non-zeros or copying in
>> general is not cheap and memory access pattern can play an important role.
>> So to establish a baseline to compare with, I tried setting the same
>> matrix but in an Eigen Sparse Matrix  and the timings are as follows
>> FillPetscMat_with_MatSetValues              100 1.0 3.8820e+00 1.1
>> 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 23  0  0  0  0 * 96 * 0  0  0  0
>>   0
>> FilEigenMat                                               100 1.0
>> 2.8727e+00 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00 18  0  0  0  0  88  0
>>  0  0  0     0
>>
>
> Great. This helps. Two things would help me narrow down what is happening.
>
>   1) Are you timing only the insertion of values, or computation and
> insertion?
>
>   2) Can you tell me how many values are inserted?
>
>   Thanks,
>
>     Matt
>
>
>> I used the same code but simply filled a different Matrix something like
>>
>> for ( i =0; i < nRows;i++)
>> {
>> //
>> .......
>> // Some code to get  j_index, coefvalues
>> // Method1
>> MatSetValues(A, 1, &cell_global_index, nnz_per_row, j_index, coefvalues,
>> INSERT_VALUES);
>>
>> //Method2
>> for ( int k = 0;k < nnz_per_row; k++)
>>      EigenMat.coeffRef(i, j_index[k] ) = coefvalues[k];
>>
>>  }
>> Please note that only one of the two methods is being used at a time.
>> Also, I separately time the code section used to <  j_index,   coefvalues>
>> but simpling disabling both Method1 and Method2.
>> I found the cost to be trivial in comparison to when either one of the
>> methods is used.
>> I used Eigen out of convenience since I used for some vector and tensor
>> arithmetics somewhere else in the code and it may not be the best choice.
>> Since in PetscMatrix we technically fill two matrices: diagonal and
>> off-diagonal so I expected some difference but is that normal or am I
>> missing something. ?
>> Maybe some setting or MatOption I should be using so far this what I have
>> been using
>>
>> MatCreateMPIAIJWithArrays(PETSC_COMM_WORLD, local_size, local_size,
>> PETSC_DETERMINE,
>> PETSC_DETERMINE, ptr, j , v, A);
>>  MatSetOption(A,MAT_NO_OFF_PROC_ENTRIES,PETSC_TRUE);
>> MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE);
>> MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE);
>> MatSetOption(A,MAT_NEW_NONZERO_LOCATION_ERR,PETSC_TRUE);
>> MatSetOption(A,MAT_NEW_NONZERO_LOCATIONS,PETSC_FALSE);
>> MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE);
>> MatSetUp(A);
>> MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY);
>> MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY);
>>
>> Thanks,
>> Kamra
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
> https://www.cse.buffalo.edu/~knepley/
> <http://www.cse.buffalo.edu/~knepley/>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20190718/24e56dd4/attachment.html>