[petsc-users] loading huge matrix with MatLoad in parallel implementation

[Barry Smith]

On Jul 29, 2010, at 3:30 PM, Yujie wrote:

> Thanks, Barry. I use X86_64 CPU, do I need to use "--with-64-bit-indices"?

   Yes, and when you save the binary matrices you need to make sure the integers you save into the binary file are 64 bit. (If you use MatView() to save the matrices this just means that the PETSc uses to save the matrices has to have been built with --with-64-bit-indices

   Barry

> 
> Regards,
> Yujie
> 
> On Thu, Jul 29, 2010 at 3:27 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
>> 
>> On Jul 29, 2010, at 3:06 PM, Yujie wrote:
>> 
>>> Dear PETSc developers,
>>> 
>>> When the program based on PETSc runs in parallel mode, how much size
>>> of the matrix can MatLoad load?
>> 
>>   Currently the limit is that each individual process cannot hold more then 2 billion nonzeros when using MatLoad(). So if you have more processes you can hold as big a matrix as you want. Within a few days PETSc-dev will eliminate this limit (which comes from how we use MPI).
>> 
>>  But you need to ./configure PETSc with the option --with-64-bit-indices and need an operating system that supports 64 bit points (which you likely have).
>> 
>>> Does the program need to save the whole matrix in one CPU,
>> 
>>  No, absolutely NOT.
>> 
>> 
>>  Barry
>> 
>> 
>>> or is each
>>> CPU in charge of the part of the matrix?
>>> 
>>> For example, I have one matrix of ~30G. There are two CPUs on each
>>> nodes with totally16G memory. I have 8 nodes. I use 16CPUs for
>>> running. Can I load this matrix using MatLoad? Thanks a lot.
>>> 
>>> Regards,
>>> Yujie
>> 
>>