[Nek5000-users] Significant computational time increase from 2D to 3D runs

[nek5000-users at lists.mcs.anl.gov]

ok, I assumed lx1=9... sorry about that.
Philipp

On 2018-03-21 22:52, nek5000-users at lists.mcs.anl.gov wrote:
> 
> Dear Kat,
> 
> 
> When you say 9-points per element, are you running with lx1=3?
> 
> 
> Let me state that this is really suboptimal for Nek, which was designed 
> for lx1 = 8-20, with lx1=5 about as low as I would go for any reasonable 
> calculation.
> 
> 
> That being said, your analysis is somewhat correct, but there are a few 
> other things to be concerned with in going from 2D to 3D.
> 
> 
> You start with 2D on a 4x100 mesh, 9 points per element.
> 
> 
> Going to 3D, you have 4x4x100, 3x9 points per element, so an increase of 
> 12x.
> 
> 
> In addition, many of the operations go from 2x2 to 3x3, which is a ratio 
> of 2.25.  (e.g., the Laplacian now involves more work as you have 
> derivatives with respect to r,s, and t in the local coordinates and 
> their transposes, etc.).  In addition to each operation being more 
> expensive per gridpiont, you now have 3 velocity components instead of 
> 2, so there is another multiplier there.   Moreover, there are more 
> surface points, 6 x 3^2, instead of 4 x 3, so 54 instead of 12, for each 
> element.
> 
> 
> My suggestion would be to first rethink your 2D and 3D problems in terms 
> of 8x8 = 64 or 8x8x8=512 points per element,  at the target resolution. 
>    Then, start the calculation with lx1=5 or 6 to converge to a 
> reasonable solution and restart with at the target value.
> 
> 
> hth,
> 
> 
> Paul
> 
> 
> ------------------------------------------------------------------------
> *From:* Nek5000-users <nek5000-users-bounces at lists.mcs.anl.gov> on 
> behalf of nek5000-users at lists.mcs.anl.gov <nek5000-users at lists.mcs.anl.gov>
> *Sent:* Wednesday, March 21, 2018 11:45:35 AM
> *To:* nek5000-users at lists.mcs.anl.gov
> *Subject:* [Nek5000-users] Significant computational time increase from 
> 2D to 3D runs
> Hello,
> 
> I am running a DNS cross-flow, two density fluid, turbulent duct 
> simulation and I see a significant increase in computation time going 
> from 2D to 3D that cannot be attributed to a simple increase in grid 
> points. I believe I am at the small end of simulation size for Nek5000, 
> as I am only running on average 4-500k grid points and running on 1-32 
> cores. Keeping all other things the same (i.e., parameters, physical 
> set-up, etc), I’ve run a 2D simulation of 100x4 elements and a 3D 
> simulation with 100x4x4 elements, both with 9 grid points per element 
> and on 4 cores, and found a increase in computational time of more than 
> 100 fold. I’ve also run 2D simulations of 200x16 elements on 4 cores 
> (again everything else the same) and found it still ran ~100 times 
> faster than the 3D simulations that had half the amount of grid points. 
> This doesn’t make sense to me and implies that this issue would not be 
> simply resolved by following the “Performance and Memory Considerations” 
> advice in the Nek5000 Docs as it is not just about an increase in grid 
> points. It seems to be linked to going from 2D to 3D, rather than due to 
> an increase in grid points.
> 
> I feel that I am missing something fundamental here. Is there something 
> one should do when switching from 2D to 3D in Nek5000 that I am not 
> aware of? I have searched through old email threads but could not find 
> anything that mentions something like this.
> 
> Thanks!
> - Kat
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> 
> 
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>