[Nek5000-users] relocation errors on small example?

Tue Feb 4 19:50:21 CST 2014

Hi Tom,

Josh is correct --- one other thing to keep in mind is that the memory layout, controlled by the product

  lx1*ly1*lz1*lelt

is the _per_processor layout.  So, if you need more memory for your problem, simply adding more
processors (or even processes, in some cases) will suffice to meet the memory demands.
Suppose for example that you wanted to run a 1 million point problem (which is of modest
size, btw).   You could do this with

    lx1=11,ly1=11,lz1=11, lelt=35 and use 32 processors.

Net result is an upper bound of the number of elements being 32 x 35, each element of order N=10.
(The lx1=11 comes from the fact that you need N+1 points to prescribe a polynomial of degree N.)

There is more explanation of the memory issues in the online manual --- I cut and paste some of
it below.

Paul

Per-processor memory requirements for Nek5000 scale roughly as 400 8-byte words per allocated gridpoint. The number of allo- cated gridpoints per processor is nmax=lx1*ly1*lz1*lelt. (For 3D, lz1=ly1=lx1; for 2D, lz1=1, ly1=lx1.) If required for a particular simulation, more memory may be made available by using addi- tional processors. For example, suppose one needed to run a simu- lation with 6000 elements of order N = 9. To leading order, the total memory requirements would be ≈ E(N +1)3points×400(wds/pt)× 8bytes/wd = 6000 × 103 × 400 × 8 = 19.2 GB. Assuming there is 400 MB of memory per core available to the user (after account- ing for OS requirements), then one could run this simulation with P ≥ 19, 200MB/(400MB/proc) = 48 processors. To do so, it would be necessary to set lelt ≥ 6000/48 = 125.

We note two other parameters of interest in the parallel context: lp, the maximum number of processors that can be used.

lelg, an upper bound on the number of elements in the simulation.

There is a slight memory penalty associated with these variables, so one generally does not want to have them excessively large. It is common, however, to have lp be as large as anticipated for a given case so that the executable can be run without recompiling on any admissable number of processors (Pmem ≤ P ≤ E, where Pmem is the value computed above).

________________________________
From: nek5000-users-bounces at lists.mcs.anl.gov [nek5000-users-bounces at lists.mcs.anl.gov] on behalf of nek5000-users at lists.mcs.anl.gov [nek5000-users at lists.mcs.anl.gov]
Sent: Tuesday, February 04, 2014 5:51 PM
To: nek5000-users
Subject: Re: [Nek5000-users] relocation errors on small example?

Hi Tom,

The variable lx1 in the SIZE file essentially specifies the number of grid points used for _each_ element in the "x" direction (ly1 for "y", lz1 for "z".  Note here that by "x", "y", "z", we are really talking about the coordinates defined on the master element, not in physical space).  Thus, the total number of grid points you are reserving for each variable (u,v,w, t, etc) is lx1*ly1*lz1*lelt.  So for your specification, you are actually attempting to allocate space for a very very large problem.  Even if you could allocate that amount of space (as you were able to by removing some variables while compiling), I believe NEK by default is only setup to handle lx1 ~ 30 or below.

I'm assuming you are wanting to have 192 points in each direction for your problem?  You can either change the number of elements (contained in the .rea file) or vary lx1 in the SIZE file so that lx1*(number of element in each direction) ~= 192.

Hopefully this helps!

Josh

On Tue, Feb 4, 2014 at 5:14 PM, <nek5000-users at lists.mcs.anl.gov<mailto:nek5000-users at lists.mcs.anl.gov>> wrote:
I am trying to set up an example problem with Nek5000.  Unfortunately,
I'm hitting the notorious:

  relocation truncated to fit: R_X86_64_PC32 against symbol

error with it.  I gather from past posts that this is an issue with
grid/problem sizes. However, I'm just using eddy_uv at the moment, with
minor modifications to bump the grid size up.  Diff appended.

By removing some (seemingly) unused common blocks for this problem I
managed to get it to link with a 192x192x1 grid, but it aborts quickly,
complaining that the problem size is too large.  I feel like I must be
doing something, because 192x192 of 32bit floats is tiny: 144K.

Could anyone point me towards what I've messed up, here?

Thanks,

-tom

_______________________________________________
Nek5000-users mailing list
Nek5000-users at lists.mcs.anl.gov<mailto:Nek5000-users at lists.mcs.anl.gov>
https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

--
Josh Camp

"All that is necessary for the triumph of evil is that good men do nothing" -- Edmund Burke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/nek5000-users/attachments/20140205/221e74c0/attachment-0001.html>