Problem with saving the binary matrix via Matview

[Barry Smith]

  Is this a Cray? You may need to set the environmental variables MPI  
UNEX BUFFER SIZE and/or MPICH PTL MATCH OFF and/or MPICH PTL OTHER  
EVENTS and/or MPICH MSGS PER PROC and/or MPICH PTL SEND CREDITS you  
will likely need to hunt through Cray documentation to find the  
meaning of all this stuff. The problems will be worse if you don't  
have the latest Cray software on the system.

   Note we generally frown up saving huge matrices to disk except for  
debugging testing purposes. But as Matt notes this is not a  
particularly huge matrix. With any reasonable configuration it should  
take very little time to write the file.

    Good luck,

    Barry




On Sep 4, 2009, at 2:25 PM, Nguyen, Hung V ERDC-ITL-MS wrote:

>
> Hello,
>
> I have a problem with saving a matrix with the binary viewer
> PetscBinaryViewerOpen() for the matrix with nrow=853564,  
> nnzeros=47191472.
> The application ran with 32 pes for more than 3 hours without  
> writing any
> into file.
> However, the same executable ran for small size of application
> (nrows-12856,nnzeros=675744) and were able to write successfully  
> binary
> files.
>
> Note: using Totalview I found that it hang in the line of
> Matview(matrix->petsc, fd);
>
> Thank you,
>
> -hung
>
> --- code:
>      petsc_analyst_mat(matrix->petsc);
>      sprintf(file[LOAD_MATRIX],"Matrix.at%f",t_prev);
>      PetscViewerBinaryOpen(PETSC_COMM_WORLD,
> file[LOAD_MATRIX],FILE_MODE_WRITE,&fd);
>      MatView(matrix->petsc, fd);
>      PetscViewerDestroy(fd);
>
> ---- Matrix info using salsa/AnaMod module:
>
> Computed <simple:trace> as <9.362113e+03>
> Computed <simple:trace-abs> as <9.362623e+03>
> Computed <simple:norm1> as <2.800000e+01>
> Computed <simple:normInf> as <2.800000e+01>
> Computed <simple:normF> as <2.404511e+02>
> Computed <simple:diagonal-dominance> as <-3.064463e-02>
> Could not compute <simple:symmetry-snorm>
> Could not compute <simple:symmetry-anorm>
> Could not compute <simple:symmetry-fsnorm>
> Could not compute <simple:symmetry-fanorm>
> Could not compute <structure:n-struct-unsymm>
> Computed <structure:nrows> as <853564>
> Could not compute <structure:symmetry>
> Computed <structure:nnzeros> as <47191472>
> Computed <structure:max-nnzeros-per-row> as <112>
> Computed <structure:min-nnzeros-per-row> as <16>