[petsc-users] MatView for huge matrix output

[Jed Brown]

On Tue, Jun 29, 2010 at 19:35, Yujie <recrusader at gmail.com> wrote:

> Dear PETSc developers,
>
> I want to output an about 36K*36K dense matrix using MatView in binary
> format. I use RedHat Enterprise 5 64bits system. However, when the
> file size of output matrix reaches about 2.7G, the codes pause and
> don't response for a long time (almost 3 hours). Could you help me
> figure out what happened? Thanks a lot.
>

Were you writing this in serial or parallel?  MPICH2 and Open MPI don't
properly handle large message sizes fixes require ABI-incompatible changes
that they don't want to push out in a minor release.  I believe the latest
versions of both will actually do the send, but MPI_Get_count does not
return the correct value, and it probably would not be surprising if some
MPI-IO functionality did not work correctly with large messages.  Tickets
that I'm familiar with:

https://trac.mcs.anl.gov/projects/mpich2/ticket/1005
https://svn.open-mpi.org/trac/ompi/ticket/2241

I think they should both be fine for MPI-IO as long as each processor sends
less than 2 GiB (even though the final output may be much bigger).

If this happens again, you could attach a debugger to the running process
(gdb -pid XXX) and get a backtrace.  Note that you can build "optimized"
with debugging symbols at a very small runtime penalty.

Jed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20100629/1c5338cc/attachment-0001.htm>