[petsc-users] Accessing submatrices without additional memory usage

[Jed Brown]

Michał Dereziński <michal.derezinski at gmail.com> writes:

>> Wiadomość napisana przez Jed Brown <jed at jedbrown.org> w dniu 24.05.2017, o godz. 12:06:
>> 
>> Okay, do you have more parameters than observations?  
>
> No (not necessarily). The biggest matrix is 50M observations and 12M parameters.
>
>> And each segment
>> of the matrix will be fully distributed?
>
> Yes.
>
>>  Do you have a parallel file
>> system?
>
> Yes.
>
>>  Is your matrix sparse or dense?
>
> Yes.

By that you mean sparse?

You'll need some sort of segmented storage (could be separate files or a
file format that allows seeking).  (If the matrix is generated by some
other process, you'd benefit from skipping the file system entirely, but
I understand that may not be possible.)

I would use MatNest, creating a new one after each segment is loaded.
There isn't currently a MatLoadBegin/End interface, but that could be
created if it would be useful.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20170524/62f0f65d/attachment.pgp>