[petsc-users] Out-of-memory, reading ASCII files using single processor

[Barry Smith]

  What code are you using to do the conversion? Is it reading the ASCII file twice and doing the preallocation of the PETSc matrix correctly?  So long as a bit more than one copy of the matrix in compressed sparse row format can fit in your physical memory you should be able to do the conversion. How much physical memory do you have and how many non zeros in your matrix?

  Barry




On Aug 18, 2014, at 9:46 AM, jifeng zhao <jifengzhao2015 at u.northwestern.edu> wrote:

> Hi Jed,
> 
> Actually the out-of-memory happened when converting ASCII files to petsc format using a serial PETSc program.
> 
> It seems a single processor doesn't have enough memory to preallocate enough space to store the matrices. Is there any way around it...?
> 
> Best regards,
> Jifeng Zhao 
> 
> 
> On Sun, Aug 17, 2014 at 11:40 PM, Jed Brown <jed at jedbrown.org> wrote:
> jifeng zhao <jifengzhao2015 at u.northwestern.edu> writes:
> 
> > Hello all,
> >
> > I am trying to read large matrices from ASCII files and save it into petsc
> > binary format using uni-processor.
> >
> > But I am encountered a out-of-memory error, when reading too-large ASCII
> > file.
> 
> Don't use ASCII.  It is a monumentally bad format for parallel
> computing.  Convert your ASCII files to PETSc binary files (or HDF5, but
> PETSc binary is sufficient and simpler), then read in parallel.  You can
> do this easily in MATLAB or Python, or with a serial PETSc program.
> 
> 
> 
> -- 
> Jifeng Zhao
> PhD candidate at Northwestern University, US
> Theoretical and Applied Mechanics Program