[Nek5000-users] Building mass and stiffness matrices

[nek5000-users at lists.mcs.anl.gov]

Hi Paul,
I am aware of the size of the stiffness and convective matrices. For the moment 48 Gb wouldn't be an issue, and I think parallel i/o can be avoided, since I need to compute and save such matrices only once, then they would be loaded and manipulated in parallel by the PETSc routines. 
Anyway I know that probably is a hard task, and I'm exploring alternative paths.
I am running on 120 cores.
Thanks again.
Giuseppe




Il giorno 13/feb/2014, alle ore 14:35, <nek5000-users at lists.mcs.anl.gov>
 <nek5000-users at lists.mcs.anl.gov> ha scritto:

> 
> Hi Giuseppe,
> 
> Just so you're aware, 3000 elements with lx1=10 in 3D
> would entail 3 billion nonzeros for the stiffness and
> convective matrices, each, so 24 GB files for each matrix
> (actually, 48 GB, because you need the CSR pointers).
> 
> Presumably you would need to do something clever about
> parallel i/o.
> 
> How many processors are you using?
> 
> Paul
> 
> 
> 
> 
> On Thu, 13 Feb 2014, nek5000-users at lists.mcs.anl.gov wrote:
> 
>> Hi Paul,
>> thank you for your answer, now I understand why I couldn't find the matrices in the code.
>> For the mass matrix I think it is clear how to proceed, but how could I do the same for the stiffness matrix and the convective matrix?
>> My problem consists in about 3000 elements, with lx1=10, but I plan to move to 6000 or 9000 elements.
>> I will now work on a subroutine for this task, then I will come back if I find other issues.
>> Giuseppe
>> 
>> 
>> 
>> 
>> Il giorno 13/feb/2014, alle ore 13:34, <nek5000-users at lists.mcs.anl.gov>
>> <nek5000-users at lists.mcs.anl.gov> ha scritto:
>> 
>>> 
>>> Hi Giuseppe,
>>> 
>>> Nek uses a completely matrix-free approach.  The standard way to generate
>>> a matrix in this case is to start passing in unit column vectors, e_j
>>> (the jth column of the Identity matrix) and saving the result a_j, the
>>> jth column of A.   For sparse matrices there is a fair amount of bookeeping required so that you store only the nonzeros.  (I normally
>>> use compressed sparse row format to do so.)
>>> 
>>> The mass matrix is diagonal and therefore readily retrieved.
>>> 
>>> How large is your problem (i.e., how many elements and what is the
>>> order, lxl) ?
>>> 
>>> Note that the number of nonzeros scales like E*(lx1^6).
>>> 
>>> Paul
>>> 
>>> On Thu, 13 Feb 2014, nek5000-users at lists.mcs.anl.gov wrote:
>>> 
>>>> Dear users and developers,
>>>> I am using NEK5000 for a 3d unsteady simulation with mixed Dirichlet and
>>>> periodic bc. For the post-processing (a POD-based dynamics) I need the global
>>>> mass and stiffness matrices as built by NEK5000 on my mesh. I would also need
>>>> the matrix having as entries (phi_i, grad(phi_j)) where phi are the basis
>>>> polynomials.  Is it possible to have NEK build these matrices and then save
>>>> them on file? The ultimate goal is to import them on a PETSc program to
>>>> perform some algebraic manipulations. I already found a way to import the
>>>> simulations' results.  Surfing the code, I have found some 1-d routines, but
>>>> I don't know how to extend them to my needs.  Thank you in advance for any
>>>> help or hint.
>>>> Best regards,
>>>> Giuseppe
>>>> 
>>>> _______________________________________________
>>>> Nek5000-users mailing list
>>>> Nek5000-users at lists.mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>>> 
>>> _______________________________________________
>>> Nek5000-users mailing list
>>> Nek5000-users at lists.mcs.anl.gov
>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>> 
>> _______________________________________________
>> Nek5000-users mailing list
>> Nek5000-users at lists.mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>> 
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users