[petsc-users] Questions about setting values for GPU based matrices

[Matthew Knepley]

On Mon, Oct 31, 2011 at 12:31 PM, Fredrik Heffer Valdmanis <
fredva at ifi.uio.no> wrote:

> 2011/10/30 Matthew Knepley <knepley at gmail.com>
>
>> On Sun, Oct 30, 2011 at 12:22 PM, Fredrik Heffer Valdmanis <
>> fredva at ifi.uio.no> wrote:
>>
>>> 2011/10/28 Matthew Knepley <knepley at gmail.com>
>>>
>>>> On Fri, Oct 28, 2011 at 10:24 AM, Fredrik Heffer Valdmanis <
>>>> fredva at ifi.uio.no> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am working on integrating the new GPU based vectors and matrices
>>>>> into FEniCS. Now, I'm looking at the possibility for getting some speedup
>>>>> during finite element assembly, specifically when inserting the local
>>>>> element matrix into the global element matrix. In that regard, I have a few
>>>>> questions I hope you can help me out with:
>>>>>
>>>>> - When calling MatSetValues with a MATSEQAIJCUSP matrix as parameter,
>>>>> what exactly is it that happens? As far as I can see, MatSetValues is not
>>>>> implemented for GPU based matrices, neither is the mat->ops->setvalues set
>>>>> to point at any function for this Mat type.
>>>>>
>>>>
>>>> Yes, MatSetValues always operates on the CPU side. It would not make
>>>> sense to do individual operations on the GPU.
>>>>
>>>> I have written batched of assembly for element matrices that are all
>>>> the same size:
>>>>
>>>>
>>>> http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/docs/manualpages/Mat/MatSetValuesBatch.html
>>>>
>>>
>>> Thanks. I assume that the best way to use the batch function is to batch
>>> up all element matrices and insert all with one function call? Or is it
>>> recommended to split it up into several smaller batches?
>>>
>>
>> Right now, several batches does not work.For insertion to be efficient,
>> you should keep the matrices in COO
>> format, or convert them back. We do not do either right now. The idea is
>> to see if it ever matters for applications.
>>
>>
>> OK, thanks.
>
> Any estimate on when additive mode will be added to MatSetValuesBatch? As
> it is now, this batch function is of limited use to us, as it forces us to
> maintain an extra internal data structure to handle accumulation of numbers
> that are inserted at the same indices in the matrix.
>

I cannot understand what you need this for. All you need is the complete
list of element matrices.

Any particular reason you chose not to support additive mode in this first
> implementation? Are there any considerations I should be aware of?
>

I said why above. It would require data structure changes and code support,
and we would need some level of user request for that.

   Matt


> Thanks,
>
> Fredrik
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20111031/78e65acd/attachment.htm>