[petsc-users] Petsc Binary Write - Memory

[Matthew Knepley]

On Wed, Jun 10, 2015 at 9:04 AM, venkatesh g <venkateshgk.j at gmail.com>
wrote:

> Hi,
>
> Yes I will try Elemental but there is lots of complex routines and
> functions with long codes in Matlab which goes in the creation of the A and
> B matrices, so re-writing in C will take a long time.
>
> Even if Matlab takes 128 GB out of 256 GB, and the matrix A to be written
> is around 25 GB, then PetscBinaryWrite.m is using more than 100 GB just to
> run.. This is the problem.
>
> so is there any other way ?
>

Don't write it as a sparse matrix. Write it as a dense matrix. This is just
a 4 integer header

MAT_FILE_CLASSID, M, N, nz


and then the data.


https://bitbucket.org/petsc/petsc/src/7fe65498eb3a5a672e8885998851e0d5742e4c1e/src/mat/impls/dense/mpi/mpidense.c?at=master#cl-1571

   Matt


> cheers,
> Venkatesh
>
> On Wed, Jun 10, 2015 at 7:07 PM, Matthew Knepley <knepley at gmail.com>
> wrote:
>
>> On Wed, Jun 10, 2015 at 8:32 AM, venkatesh g <venkateshgk.j at gmail.com>
>> wrote:
>>
>>> The size of the matrix is 84500 x 84500
>>>
>>> the no. of non-zero elements is 2.7338e+09
>>>
>>
>> This matrix is not sparse, it has 40% fill. You should treat it as dense.
>> For dense matrices of this size,
>> you should consider using Elemental. We have an interface to Elemental in
>> PETSc.
>>
>> I recommend writing the code to create these entries on the fly since it
>> will probably be faster than loading
>> them from disk.
>>
>>   Thanks,
>>
>>     Matt
>>
>>
>>> cheers,
>>> Venkatesh
>>>
>>> On Wed, Jun 10, 2015 at 6:35 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>>>
>>>> Whats the size of the matrix? How many non-zeros?
>>>>
>>>> Satish
>>>>
>>>> On Wed, 10 Jun 2015, venkatesh g wrote:
>>>>
>>>> > Hi
>>>> >
>>>> > I am trying to write very large sparse matrices A and B for solving
>>>> > generalized Eigenvalue problem
>>>> >
>>>> > so that I can use SLEPC ex7.c code.
>>>> >
>>>> > I want to read matrices from file according to that code. And I
>>>> generate
>>>> > these matrices from Matlab using PetscBinaryWrite.m
>>>> >
>>>> > However, it exceeds my 256 GB RAM in one of the machines. So I am
>>>> unable to
>>>> > generate these binary matrices.
>>>> >
>>>> > Kindly let me know how to write them efficiently.
>>>> >
>>>> > cheers,
>>>> >
>>>> > Venkatesh
>>>> >
>>>>
>>>>
>>>
>>
>>
>> --
>> What most experimenters take for granted before they begin their
>> experiments is infinitely more interesting than any results to which their
>> experiments lead.
>> -- Norbert Wiener
>>
>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20150610/df46ee0c/attachment.html>