[Nek5000-users] Proper setup for AMG solver

[nek5000-users at lists.mcs.anl.gov]

How much time are you spending in your scalar fields?


Do you have projection turned on for all these fields?


grep tep logfile


will tell you how much time per step


grep gmr logfile will tell you how much time in the pressure on each step


What's left over is mostly passive scalar, unless you are using characteristics.


I would not recommend characteristics when running 10 scalar fields.


Paul


________________________________
From: Nek5000-users <nek5000-users-bounces at lists.mcs.anl.gov> on behalf of nek5000-users at lists.mcs.anl.gov <nek5000-users at lists.mcs.anl.gov>
Sent: Saturday, September 1, 2018 9:50:38 AM
To: nek5000-users at lists.mcs.anl.gov
Subject: [Nek5000-users] Proper setup for AMG solver


Dear Nek users & experts,

I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013.
Additionally, I solve for 10 thermal fields being treated as passive scalars.

The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700.
As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury).

As the simulation is still very heavy, I have been looking into ways for speeding it up.
I found some good suggestions here:
http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed

and here (older version?)
http://nek5000.github.io/NekDoc/large_scale.html


However, I have some questions regarding these suggestions.
1) Dealiasing:
Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1?

2) Tolerances:
I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes?

3) Time discretisation: BDF2 and OIFS with Courant=2-5
If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger?

4) AMG instead of XXT:
I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this
4.1) Set SEMG_AMG in the par file.
4.2) Run the simulation once to dump amg files.
4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings)
4.4) Run the simulation.
Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them.


I am grateful for any advice regarding these aspects.

Best Regards,
Steffen



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/nek5000-users/attachments/20180901/62611526/attachment-0001.html>