[petsc-users] Non zero entries

[Barry Smith]

  1) you can call both MatMPIAIJSetPreallocation() and MatSeqAIJSetPreallocation() BOTH, the one that is not needed gets ignored so you don’t need that ugly if in your code.

  2) unfortunately PETSc removes any excessive space that was not needed during the first MatAssemblyBegin/End() thus when you add the additional entries later it needs to reallocate all new space. The way to make it always fast is to make sure that the first set of calls to MatSetValues and MatAssemblyBegin/End include all possible nonzero locations (just put zeros in any place that is not immediately needed) then PETSc will keep that space around and the later calls will be fast.

  Barry



On Aug 11, 2014, at 2:50 PM, Chung-Kan Huang <ckhuangf at gmail.com> wrote:

> Hi,
>  
> I have some confusion related to NONZERO in the matrix.
> In my application I created Matrix using following procedure.
> 
>   MatCreate(* comm_, & J_);
>   PetscInt * d_nnz = NULL;
>   PetscInt * o_nnz = NULL;
>   PetscMalloc(local_size * sizeof(PetscInt), & d_nnz);
>   PetscMalloc(local_size * sizeof(PetscInt), & o_nnz);
>   for (int i = 0; i < local_size; i++) {
>     d_nnz[i] = d_nnz_v[i];
>     o_nnz[i] = o_nnz_v[i];
>   }
>   MatSetType(J_, MATAIJ);
>   MatSetSizes(J_,
>        local_size,
>        local_size,
>        PETSC_DECIDE,
>        PETSC_DECIDE);
>   
>   
>   if (comm_->Get_size() > 1) {
>     // MPI
>     MatMPIAIJSetPreallocation(J_,
>          max_d_nz,
>          d_nnz,
>          max_o_nz,
>          o_nnz);
>   } else {
>     // Seq
>     MatSeqAIJSetPreallocation(J_,
>          PETSC_DEFAULT, 
>          d_nnz);
>   }
>   PetscFree(d_nnz);
>   PetscFree(o_nnz);
>  
> The sructure of the matrix changes from time to time during the application (as the connectivity list is varying) however  o_nnz d_nnz are made based on the densest possible scenario.
> 
> In a recently test I noticed that matrix consturcion time become extremely long when the sturucture changes (form less dense to denser).  I dug a little and found out it become very slow in
>   MatSetValues(J_,
>         NumRows,
>         idxm,
>         NumCols,
>         idxn,
>         data,
>         ADD_VALUES);
> and it becomes normal again after the first chanaged matrix has been assembled.
>  
> In my case I didn't do MatSetValues for the connection doesn't exisit in on time connectivity list although I count them in d_nnz and o_nnz but I started to suspect that PETSC was allocating memory for the new additional entries base on previous matrix structure. 
>  
> Any suggestions?
>  
> Cheers
>