Problem with saving the binary matrix via Matview
Barry Smith
bsmith at mcs.anl.gov
Fri Sep 4 15:23:33 CDT 2009
Is this a Cray? You may need to set the environmental variables MPI
UNEX BUFFER SIZE and/or MPICH PTL MATCH OFF and/or MPICH PTL OTHER
EVENTS and/or MPICH MSGS PER PROC and/or MPICH PTL SEND CREDITS you
will likely need to hunt through Cray documentation to find the
meaning of all this stuff. The problems will be worse if you don't
have the latest Cray software on the system.
Note we generally frown up saving huge matrices to disk except for
debugging testing purposes. But as Matt notes this is not a
particularly huge matrix. With any reasonable configuration it should
take very little time to write the file.
Good luck,
Barry
On Sep 4, 2009, at 2:25 PM, Nguyen, Hung V ERDC-ITL-MS wrote:
>
> Hello,
>
> I have a problem with saving a matrix with the binary viewer
> PetscBinaryViewerOpen() for the matrix with nrow=853564,
> nnzeros=47191472.
> The application ran with 32 pes for more than 3 hours without
> writing any
> into file.
> However, the same executable ran for small size of application
> (nrows-12856,nnzeros=675744) and were able to write successfully
> binary
> files.
>
> Note: using Totalview I found that it hang in the line of
> Matview(matrix->petsc, fd);
>
> Thank you,
>
> -hung
>
> --- code:
> petsc_analyst_mat(matrix->petsc);
> sprintf(file[LOAD_MATRIX],"Matrix.at%f",t_prev);
> PetscViewerBinaryOpen(PETSC_COMM_WORLD,
> file[LOAD_MATRIX],FILE_MODE_WRITE,&fd);
> MatView(matrix->petsc, fd);
> PetscViewerDestroy(fd);
>
> ---- Matrix info using salsa/AnaMod module:
>
> Computed <simple:trace> as <9.362113e+03>
> Computed <simple:trace-abs> as <9.362623e+03>
> Computed <simple:norm1> as <2.800000e+01>
> Computed <simple:normInf> as <2.800000e+01>
> Computed <simple:normF> as <2.404511e+02>
> Computed <simple:diagonal-dominance> as <-3.064463e-02>
> Could not compute <simple:symmetry-snorm>
> Could not compute <simple:symmetry-anorm>
> Could not compute <simple:symmetry-fsnorm>
> Could not compute <simple:symmetry-fanorm>
> Could not compute <structure:n-struct-unsymm>
> Computed <structure:nrows> as <853564>
> Could not compute <structure:symmetry>
> Computed <structure:nnzeros> as <47191472>
> Computed <structure:max-nnzeros-per-row> as <112>
> Computed <structure:min-nnzeros-per-row> as <16>
More information about the petsc-users
mailing list