[Nek5000-users] Help With Basic NEK Configuration

[nek5000-users at lists.mcs.anl.gov]

Harish,

We can help you with the larger runs.

The memory constraint is only one constraint, and it does
not limit how _small_ a problem can be.

There is a different constraint on problem size/processor
count, which is that you must have at least one element
per processor.   So if the problem has E=512 elements,
then P_max = 512.

It sounds like you've done a pretty good job of exploring
the parameter space for the memory footprint etc.

Memory requirements on a processor for a given compilation 
is, to leading order, given by the product:

      S*lx1*ly1*lz1*lelt

where S is between 200 and 400 words, each word being 8 bytes.

There are other terms that can be important in some specific
cases, particularly for large element (and hence, processor)
counts.   But these are pretty well under control.   Specifically,
there are two (4 byte) integer arrays that scale as lelg.

Your total problem size (E==nelt in the .rea file) is constrained
by:

     E =< min(lelg,P*lelt)

where P is the number of processors at run time.   Similarly, P
is constrained by:

     P =< lp


There should be no memory impact for having a large value of lp.

Paul



On Thu, 4 Aug 2011, nek5000-users at lists.mcs.anl.gov wrote:

> Hi,
>
> I am a new to NEK and I am trying to run a few examples for my OS scaling 
> experiments. I went through the manual which is at: 
> http://www.mcs.anl.gov/~fischer/new.pdf. In the chapter that speaks about 
> a basic problem setup, there is a line in the 'Memory Requirements' 
> section that says:
>
> Total Memory Required for simulation = E(N +1)^3 points × 400(wds/pt) × 
> 8bytes/wd
>
> I have been playing around with the SIZE file a little bit and adjusting, 
> lx1, lelt, lxd, lp and lelg parameters. However I am unable to find where 
> I can adjust 'E' from the equation. I understand that 'lelt' is maximum 
> elements per processor and 'lelg' is maximum elements per simulation? Is 
> 'lelg' same as 'E'? is 'lelt = lelg / nprocs' ? I assumed that 'E = lelg' 
> and changed 'lelg' for 'timing' example to run it on a 4096 node 
> configuration (1 rack of BG/P:Intrepid in VN mode). I keep getting the 
> error:
>
> Too many processors ( 4096 ) for to few elements ( 512 )
>
> ... and the value 512 does not seem to change although I have changed 
> 'lelg' and run 'makenek timing'. Is the value of 'E' picked up from the 
> .re2 file? Would I have to generate it using 'genbox' after I change some 
> other configuration?
>
> I understand that NEK has been scaled upto 40 racks of Intrepid at 
> Argonne. I am also trying to run experiments of upto 32 racks if possible. 
> But I want to get started with smaller runs. Is there a repository of 
> configurations or tarred files where I could get larger, successfully run 
> problem instances?
>
> Thanks.
>
> Regards,
> Harish Naik
> Predoctoral Appointee
> Mathematics & Computer Science Division
> Argonne National Laboratory
> P: (630) 252-2029
> W: http://www.mcs.anl.gov/~hnaik
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>