From nek5000-users at lists.mcs.anl.gov Sat Sep 1 05:24:12 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 1 Sep 2018 12:24:12 +0200 Subject: [Nek5000-users] Parameter settings on filterWeight and filterCutoffRatio In-Reply-To: References: Message-ID: The meaning of the two parameters depends on the filterType. Typically a filterCutOffRatio of 0.9 works well (for say N=7). There is a weak dependence to the filterWeight but that?s something you need to explore for your case. May be Philipp can tell you more about it including some references. Stefan On 1 Sep 2018, at 06:16, "nek5000-users at lists.mcs.anl.gov " > wrote: Hi neks, ? ?Recently I am working on an LES simulation in an annular pipe to see how transverse velocity goes with Re increasing. But I am quite wondering how I should set filterWeight and filterCutoffRatio in my example while in the turbChannel demo they are selected to 10 an 0.9 respectively as proposed value. They should be sensitive parameters in LES simulation, don't they? Is there any references concerned about them to help me see them more clearly? If it is convenient for you could you please reply me a url to the reference? Thank you so much for your assistence. best regard -- _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sat Sep 1 09:50:38 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 1 Sep 2018 14:50:38 +0000 Subject: [Nek5000-users] Proper setup for AMG solver Message-ID: Dear Nek users & experts, I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. Additionally, I solve for 10 thermal fields being treated as passive scalars. The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). As the simulation is still very heavy, I have been looking into ways for speeding it up. I found some good suggestions here: http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed and here (older version?) http://nek5000.github.io/NekDoc/large_scale.html However, I have some questions regarding these suggestions. 1) Dealiasing: Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? 2) Tolerances: I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes? 3) Time discretisation: BDF2 and OIFS with Courant=2-5 If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? 4) AMG instead of XXT: I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this 4.1) Set SEMG_AMG in the par file. 4.2) Run the simulation once to dump amg files. 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) 4.4) Run the simulation. Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. I am grateful for any advice regarding these aspects. Best Regards, Steffen -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sat Sep 1 10:10:54 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 1 Sep 2018 17:10:54 +0200 Subject: [Nek5000-users] Proper setup for AMG solver Message-ID: Try to use lx1=8/lxd=10 with a (potentially) finer mesh BDF2 + OIFS with a targetCFL=3.5 set dt = 0 (this will adjust dt to targetCFL) pressure tol = 1e-5 (residual projection turned on) 1e-6 for velocity and scalars (residual projection turned off) Note, a mesh with more than 350k elements requires AMG. The default parameters are fine. What version of Nek5000 are you using? Cheers, Stefan -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Saturday 1st September 2018 16:50 > To: nek5000-users at lists.mcs.anl.gov > Subject: [Nek5000-users] Proper setup for AMG solver > > Dear Nek users & experts, > > I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. > > Additionally, I solve for 10 thermal fields being treated as passive scalars. > > The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. > > As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). > > As the simulation is still very heavy, I have been looking into ways for speeding it up. > I found some good suggestions here: > http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed > > and here (older version?) > http://nek5000.github.io/NekDoc/large_scale.html > > > However, I have some questions regarding these suggestions. > 1) Dealiasing: > Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? > > 2) Tolerances: > I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear > stress budgets or budget of turbulent heat fluxes? > > 3) Time discretisation: BDF2 and OIFS with Courant=2-5 > If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? > > 4) AMG instead of XXT: > I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this > 4.1) Set SEMG_AMG in the par file. > 4.2) Run the simulation once to dump amg files. > 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) > 4.4) Run the simulation. > Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. > > > I am grateful for any advice regarding these aspects. > > Best Regards, > Steffen > > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Sat Sep 1 10:15:29 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 1 Sep 2018 15:15:29 +0000 Subject: [Nek5000-users] Proper setup for AMG solver In-Reply-To: References: Message-ID: How much time are you spending in your scalar fields? Do you have projection turned on for all these fields? grep tep logfile will tell you how much time per step grep gmr logfile will tell you how much time in the pressure on each step What's left over is mostly passive scalar, unless you are using characteristics. I would not recommend characteristics when running 10 scalar fields. Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Saturday, September 1, 2018 9:50:38 AM To: nek5000-users at lists.mcs.anl.gov Subject: [Nek5000-users] Proper setup for AMG solver Dear Nek users & experts, I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. Additionally, I solve for 10 thermal fields being treated as passive scalars. The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). As the simulation is still very heavy, I have been looking into ways for speeding it up. I found some good suggestions here: http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed and here (older version?) http://nek5000.github.io/NekDoc/large_scale.html However, I have some questions regarding these suggestions. 1) Dealiasing: Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? 2) Tolerances: I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes? 3) Time discretisation: BDF2 and OIFS with Courant=2-5 If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? 4) AMG instead of XXT: I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this 4.1) Set SEMG_AMG in the par file. 4.2) Run the simulation once to dump amg files. 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) 4.4) Run the simulation. Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. I am grateful for any advice regarding these aspects. Best Regards, Steffen -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sat Sep 1 18:28:59 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 1 Sep 2018 23:28:59 +0000 Subject: [Nek5000-users] re2torea Message-ID: Hi , I am trying to convert re2 to rea file, but the following error appears: name.re2\ byte_read() :: fopen failure2! Does anybody know what the problem is? Thanks, Ali -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sun Sep 2 05:05:28 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sun, 2 Sep 2018 10:05:28 +0000 Subject: [Nek5000-users] Proper setup for AMG solver In-Reply-To: References: Message-ID: Hello Stefan & Paul, thanks for your suggestions. I use Nek5000 v17 (mver 17.0.4 in makenek). I have taken a closer look at the logfiles as suggested by Paul. It seems like I spend the most time for the scalar fields second most for the velocity fields and pressure solve is the smallest share. The numbers below are in seconds for a typical timestep without calculating statistics or writing out files at Re_b = 5300 (Re_t = u_t D / nu = 360, 768 cores) and Re_b = 37700 (Re_t = 2000, 6144 cores). I use projection for all 10 scalar fields. 360: Scalars done 0.064 Fluid done 0.039 U-PRES gmres 0.021 Step 0.127 2000: Scalars done 0.94 Fluid done 0.51 U-PRES gmres 0.21 Step 1.72 Paul, could you elaborate on why you would not use characteristics when running 10 scalar fields? In my short tests at Re_b=5300, it appears to be more time consuming per step but due to the increase in DT it is worth it to go to characteristics. The time per timestep for targetCFL=2.0 increases by a factor of 3 but DT is increased by a factor of 6. Besides, are such increased timesteps still small enough to capture the temporal evolution of the flow? When I am back on my workstation, I will create a different mesh with lx1=8, lxd=10 and run with the settings Stefan suggested. I expect to get a significant speedup when using the lower tolerances also for velocity and scalars and changing to characteristics. I know about the limit of 350k elements for XXT as I have once commented out the part of the code where this is tested. Since for my setups AMG was always slower than XXT, I am thinking about sticking to XXT. Is there any other reason than your experience with AMG and XXT at large number of elements to enforce AMG? Best Regards, Steffen Message: 5 Date: Sat, 1 Sep 2018 17:10:54 +0200 From: nek5000-users at lists.mcs.anl.gov To: nek5000-users at lists.mcs.anl.gov Subject: Re: [Nek5000-users] Proper setup for AMG solver Message-ID: Content-Type: text/plain; charset=utf-8 Try to use lx1=8/lxd=10 with a (potentially) finer mesh BDF2 + OIFS with a targetCFL=3.5 set dt = 0 (this will adjust dt to targetCFL) pressure tol = 1e-5 (residual projection turned on) 1e-6 for velocity and scalars (residual projection turned off) Note, a mesh with more than 350k elements requires AMG. The default parameters are fine. What version of Nek5000 are you using? Cheers, Stefan -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Saturday 1st September 2018 16:50 > To: nek5000-users at lists.mcs.anl.gov > Subject: [Nek5000-users] Proper setup for AMG solver > > Dear Nek users & experts, > > I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. > > Additionally, I solve for 10 thermal fields being treated as passive scalars. > > The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. > > As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). > > As the simulation is still very heavy, I have been looking into ways for speeding it up. > I found some good suggestions here: > http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed > > and here (older version?) > http://nek5000.github.io/NekDoc/large_scale.html > > > However, I have some questions regarding these suggestions. > 1) Dealiasing: > Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? > > 2) Tolerances: > I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear > stress budgets or budget of turbulent heat fluxes? > > 3) Time discretisation: BDF2 and OIFS with Courant=2-5 > If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? > > 4) AMG instead of XXT: > I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this > 4.1) Set SEMG_AMG in the par file. > 4.2) Run the simulation once to dump amg files. > 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) > 4.4) Run the simulation. > Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. > > > I am grateful for any advice regarding these aspects. > > Best Regards, > Steffen > > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users ------------------------------ Message: 6 Date: Sat, 1 Sep 2018 15:15:29 +0000 From: nek5000-users at lists.mcs.anl.gov To: "nek5000-users at lists.mcs.anl.gov" Subject: Re: [Nek5000-users] Proper setup for AMG solver Message-ID: Content-Type: text/plain; charset="us-ascii" How much time are you spending in your scalar fields? Do you have projection turned on for all these fields? grep tep logfile will tell you how much time per step grep gmr logfile will tell you how much time in the pressure on each step What's left over is mostly passive scalar, unless you are using characteristics. I would not recommend characteristics when running 10 scalar fields. Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Saturday, September 1, 2018 9:50:38 AM To: nek5000-users at lists.mcs.anl.gov Subject: [Nek5000-users] Proper setup for AMG solver Dear Nek users & experts, I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. Additionally, I solve for 10 thermal fields being treated as passive scalars. The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). As the simulation is still very heavy, I have been looking into ways for speeding it up. I found some good suggestions here: http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed and here (older version?) http://nek5000.github.io/NekDoc/large_scale.html However, I have some questions regarding these suggestions. 1) Dealiasing: Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? 2) Tolerances: I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes? 3) Time discretisation: BDF2 and OIFS with Courant=2-5 If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? 4) AMG instead of XXT: I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this 4.1) Set SEMG_AMG in the par file. 4.2) Run the simulation once to dump amg files. 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) 4.4) Run the simulation. Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. I am grateful for any advice regarding these aspects. Best Regards, Steffen -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Subject: Digest Footer _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users ------------------------------ End of Nek5000-users Digest, Vol 115, Issue 1 ********************************************* From nek5000-users at lists.mcs.anl.gov Sun Sep 2 05:38:14 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sun, 2 Sep 2018 12:38:14 +0200 Subject: [Nek5000-users] Proper setup for AMG solver Message-ID: Do you really need to solve for 10 scalars fields? I would turn off the residual projection for scalars. However it's hard to come up with a general recommondation. That's something you may want to experiment with. Turned it on/off and see what happens. How much faster was the pressure solve with XXT vs AMG? I have never seen a case with more than 10-15%. Based on my experience the crossing point (where AMG beats XXT) is often somewhere between 300-500k elements. The work scales O(N^5/3) vs O(N). Also, the xxt setup timings can be significant for larger element counts. For OIFS I would either use a targetCFL = 1.9 or 3.9 (1 or 2 RK4 substep). Are you using a variable dt? If your time step size is small enough to be accurate depends heavily on what you are interested in. I have done similiar pipe and channel flows simulations using different time step sizes and the results (first and second order statistics) where more or less the same. Again, that's something you need to play with to be sure it's true for your case. Please contact me off-list. I am sure there are ways to speed-up your case ;) Cheers, Stefan -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Sunday 2nd September 2018 12:06 > To: nek5000-users at lists.mcs.anl.gov > Subject: Re: [Nek5000-users] Proper setup for AMG solver > > Hello Stefan & Paul, > > thanks for your suggestions. I use Nek5000 v17 (mver 17.0.4 in makenek). > > I have taken a closer look at the logfiles as suggested by Paul. It seems like I spend the most time for the scalar fields second most for the velocity fields and pressure solve is the smallest share. The numbers below are in seconds for a typical timestep without calculating statistics or writing out files at Re_b = 5300 (Re_t = u_t D / nu = 360, 768 cores) and Re_b = 37700 (Re_t = 2000, 6144 cores). > I use projection for all 10 scalar fields. > > 360: > Scalars done 0.064 > Fluid done 0.039 > U-PRES gmres 0.021 > Step 0.127 > > 2000: > Scalars done 0.94 > Fluid done 0.51 > U-PRES gmres 0.21 > Step 1.72 > > > Paul, could you elaborate on why you would not use characteristics when running 10 scalar fields? > In my short tests at Re_b=5300, it appears to be more time consuming per step but due to the increase in DT it is worth it to go to characteristics. The time per timestep for targetCFL=2.0 increases by a factor of 3 but DT is increased by a factor of 6. > Besides, are such increased timesteps still small enough to capture the temporal evolution of the flow? > > > When I am back on my workstation, I will create a different mesh with lx1=8, lxd=10 and run with the settings Stefan suggested. I expect to get a significant speedup when using the lower tolerances also for velocity and scalars and changing to characteristics. > > I know about the limit of 350k elements for XXT as I have once commented out the part of the code where this is tested. > Since for my setups AMG was always slower than XXT, I am thinking about sticking to XXT. > Is there any other reason than your experience with AMG and XXT at large number of elements to enforce AMG? > > > Best Regards, > Steffen > > > > Message: 5 > Date: Sat, 1 Sep 2018 17:10:54 +0200 > From: nek5000-users at lists.mcs.anl.gov > To: nek5000-users at lists.mcs.anl.gov > Subject: Re: [Nek5000-users] Proper setup for AMG solver > Message-ID: > > Content-Type: text/plain; charset=utf-8 > > Try to use > > lx1=8/lxd=10 with a (potentially) finer mesh > BDF2 + OIFS with a targetCFL=3.5 > set dt = 0 (this will adjust dt to targetCFL) > pressure tol = 1e-5 (residual projection turned on) 1e-6 for velocity and scalars (residual projection turned off) > > Note, a mesh with more than 350k elements requires AMG. The default parameters are fine. > > What version of Nek5000 are you using? > > Cheers, > Stefan > > > -----Original message----- > > From:nek5000-users at lists.mcs.anl.gov > > Sent: Saturday 1st September 2018 16:50 > > To: nek5000-users at lists.mcs.anl.gov > > Subject: [Nek5000-users] Proper setup for AMG solver > > > > Dear Nek users & experts, > > > > I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. > > > > Additionally, I solve for 10 thermal fields being treated as passive scalars. > > > > The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. > > > > As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). > > > > As the simulation is still very heavy, I have been looking into ways for speeding it up. > > I found some good suggestions here: > > http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed > > > > and here (older version?) > > http://nek5000.github.io/NekDoc/large_scale.html > > > > > > However, I have some questions regarding these suggestions. > > 1) Dealiasing: > > Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? > > > > 2) Tolerances: > > I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear > > stress budgets or budget of turbulent heat fluxes? > > > > 3) Time discretisation: BDF2 and OIFS with Courant=2-5 > > If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? > > > > 4) AMG instead of XXT: > > I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this > > 4.1) Set SEMG_AMG in the par file. > > 4.2) Run the simulation once to dump amg files. > > 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) > > 4.4) Run the simulation. > > Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. > > > > > > I am grateful for any advice regarding these aspects. > > > > Best Regards, > > Steffen > > > > > > > > _______________________________________________ > > Nek5000-users mailing list > > Nek5000-users at lists.mcs.anl.gov > > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > > ------------------------------ > > Message: 6 > Date: Sat, 1 Sep 2018 15:15:29 +0000 > From: nek5000-users at lists.mcs.anl.gov > To: "nek5000-users at lists.mcs.anl.gov" > > Subject: Re: [Nek5000-users] Proper setup for AMG solver > Message-ID: > > Content-Type: text/plain; charset="us-ascii" > > > How much time are you spending in your scalar fields? > > > Do you have projection turned on for all these fields? > > > grep tep logfile > > > will tell you how much time per step > > > grep gmr logfile will tell you how much time in the pressure on each step > > > What's left over is mostly passive scalar, unless you are using characteristics. > > > I would not recommend characteristics when running 10 scalar fields. > > > Paul > > > ________________________________ > From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov > Sent: Saturday, September 1, 2018 9:50:38 AM > To: nek5000-users at lists.mcs.anl.gov > Subject: [Nek5000-users] Proper setup for AMG solver > > > Dear Nek users & experts, > > I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. > Additionally, I solve for 10 thermal fields being treated as passive scalars. > > The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. > As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). > > As the simulation is still very heavy, I have been looking into ways for speeding it up. > I found some good suggestions here: > http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed > > and here (older version?) > http://nek5000.github.io/NekDoc/large_scale.html > > > However, I have some questions regarding these suggestions. > 1) Dealiasing: > Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? > > 2) Tolerances: > I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes? > > 3) Time discretisation: BDF2 and OIFS with Courant=2-5 > If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? > > 4) AMG instead of XXT: > I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this > 4.1) Set SEMG_AMG in the par file. > 4.2) Run the simulation once to dump amg files. > 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) > 4.4) Run the simulation. > Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. > > > I am grateful for any advice regarding these aspects. > > Best Regards, > Steffen > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > > ------------------------------ > > End of Nek5000-users Digest, Vol 115, Issue 1 > ********************************************* > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Wed Sep 5 15:15:01 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 5 Sep 2018 20:15:01 +0000 Subject: [Nek5000-users] Cylindrical Channel in Rectangular Prism using .re2 Message-ID: Hello Nek developers and users, I am currently developing a case in which a 3D rectangular domain containing a cylindrical channel located at its center. This channel will have a constant temperature BC. Does anyone have any experience with this or a similar setup, i.e. placing a cylinder in the mesh and applying boundary conditions to it? Please direct me to any literature referencing this process or help with some suggestions. All the examples I've found make use of .rea and I am using .re2. Any help is greatly appreciated. I've considered altering the mesh, like in the hillp example, however this leaves the question of BC's. Best Regards, Mario -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sun Sep 9 06:28:00 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sun, 9 Sep 2018 12:28:00 +0100 Subject: [Nek5000-users] findpts Message-ID: Hi nek, I wrote before in terms of findpts subroutine and tried some code. Unfortunately, I failed to get it right. May I attach my code here and please help me to sort it. I want to map the given set of arbitrary points xyz(ldim,n) below to the local coordinates in the mesh (velocity grid). Currently, I stop at how to use findpts results to return ix, iy, iz and iel (element number). Can anyone help me finish it? subroutine interp_p(ielll,ixx,iyy,izz,xyz,n) !find elememt and processor for points xyz include 'SIZE' include 'TOTAL' real xyz(ldim,n), r(n), s(n), s(n) logical ifjac,ifpts integer rcodee(n), ielll(n), nidd(n) integer nfail, inn real dist(lpart) real rst(lpart*ldim) parameter(nmax=lpart,nfldmax=ldim) common /rv_intp/ pts(ldim*nmax) common /iv_intp/ ihandle integer icalld,e save icalld data icalld /0/ nfail = 0 nxyz = nx1*ny1*nz1 ntot = nxyz*nelt if (n.gt.nmax) call exitti ('ABORT: interp_p() n > nmax!$',n) if (nelgt.ne.nelgv) call exitti $ ('ABORT: interp_p() nelgt.ne.nelgv not yet supported!$',nelgv) do i=1,n ! ? not moving -> save? pts(i) = xyz(1,i) pts(i + n) = xyz(2,i) if (if3d) pts(i + n*2) = xyz(3,i) enddo if (icalld.eq.0) then ! interpolation setup !? intpts_done(ih_intp_v)? icalld = 1 tolin = 1.e-8 call intpts_setup(tolin,ihandle) endif nflds = ndim ! number of fields to interpolate ! interpolate ifjac = .true. ! output transpose (of Jacobian) ifpts = .true. ! find points if(nio.eq.0) write(6,*) 'call findpts' call fintpts(ihandle,rcodee,1,ielll,1,rst,ndim,dist,1,pts(1),1,pts(n+1) $ ,1,pts(2*n+1),1,n) do inn = 1, n ! check return code if(rcodee(inn) .eq. 1) then nfail = nfail + 1 if(nfail .le. 5) write(6,'(a,1p4e15.7)') $ ' WARNING: point on boundary or outside the mesh xy[z]d^2: ', $ (pts(inn+k*n),k=0,ndim-1),dist(inn) endif elseif(rcodee(inn) .eq. 2) then nfail = nfail + 1 if(nfail .le. 5) write(6,'(a,1p3e15.7)') $ ' WARNING: point not within mesh xy[z]: !', $ (pts(inn+k*n),k=0,ndim-1) endif enddo Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sun Sep 9 06:41:09 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sun, 9 Sep 2018 13:41:09 +0200 Subject: [Nek5000-users] findpts In-Reply-To: References: Message-ID: Check out the hemi example. This will show you how to use the interpolation tool. > On 9 Sep 2018, at 13:27, "nek5000-users at lists.mcs.anl.gov" wrote: > > Hi nek, > > I wrote before in terms of findpts subroutine and tried some code. Unfortunately, I failed to get it right. May I attach my code here and please help me to sort it. > > I want to map the given set of arbitrary points xyz(ldim,n) below to the local coordinates in the mesh (velocity grid). Currently, I stop at how to use findpts results to return ix, iy, iz and iel (element number). Can anyone help me finish it? > > subroutine interp_p(ielll,ixx,iyy,izz,xyz,n) !find elememt and processor for points xyz > > include 'SIZE' > include 'TOTAL' > > real xyz(ldim,n), r(n), s(n), s(n) > logical ifjac,ifpts > > integer rcodee(n), ielll(n), nidd(n) > integer nfail, inn > real dist(lpart) > real rst(lpart*ldim) > > parameter(nmax=lpart,nfldmax=ldim) > common /rv_intp/ pts(ldim*nmax) > common /iv_intp/ ihandle > > integer icalld,e > save icalld > data icalld /0/ > > nfail = 0 > nxyz = nx1*ny1*nz1 > ntot = nxyz*nelt > > if (n.gt.nmax) call exitti ('ABORT: interp_p() n > nmax!$',n) > > if (nelgt.ne.nelgv) call exitti > $ ('ABORT: interp_p() nelgt.ne.nelgv not yet supported!$',nelgv) > > do i=1,n ! ? not moving -> save? > pts(i) = xyz(1,i) > pts(i + n) = xyz(2,i) > if (if3d) pts(i + n*2) = xyz(3,i) > enddo > > if (icalld.eq.0) then ! interpolation setup !? intpts_done(ih_intp_v)? > icalld = 1 > tolin = 1.e-8 > call intpts_setup(tolin,ihandle) > endif > > nflds = ndim ! number of fields to interpolate > > ! interpolate > ifjac = .true. ! output transpose (of Jacobian) > ifpts = .true. ! find points > if(nio.eq.0) write(6,*) 'call findpts' > call fintpts(ihandle,rcodee,1,ielll,1,rst,ndim,dist,1,pts(1),1,pts(n+1) > $ ,1,pts(2*n+1),1,n) > do inn = 1, n ! check return code > if(rcodee(inn) .eq. 1) then > nfail = nfail + 1 > if(nfail .le. 5) write(6,'(a,1p4e15.7)') > $ ' WARNING: point on boundary or outside the mesh xy[z]d^2: ', > $ (pts(inn+k*n),k=0,ndim-1),dist(inn) > endif > elseif(rcodee(inn) .eq. 2) then > nfail = nfail + 1 > if(nfail .le. 5) write(6,'(a,1p3e15.7)') > $ ' WARNING: point not within mesh xy[z]: !', > $ (pts(inn+k*n),k=0,ndim-1) > endif > enddo > > > Kind regards, > > Jian > > _______________________________________________ > > Nek5000-users mailing list > > Nek5000-users at lists.mcs.anl.gov > > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Mon Sep 10 03:11:50 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 10:11:50 +0200 Subject: [Nek5000-users] Parameter settings on filterWeight and filterCutoffRatio In-Reply-To: References: Message-ID: Hi, Indeed, there is a weak dependence on the actual value, which is case (and resolution) dependent. Note that the weight has the unit of an inverse time scale. The filter is to some extent self-regulating as the actual forcing is proportional to the high-pass filtered velocity itself, so it can be viewed as a P controller. There is some discussion on the dependence for a similar LES methodology in my thesis some years ago: http://e-collection.library.ethz.ch/eserv/eth:27816/eth-27816-02.pdf and in some related papers. Philipp On 2018-09-01 12:24, nek5000-users at lists.mcs.anl.gov wrote: > The meaning of the two parameters depends on the filterType. Typically a > filterCutOffRatio of 0.9 works well (for say N=7). There is a weak > dependence to the filterWeight but that?s something you need to explore > for your case. May be Philipp can tell you more about it including some > references. > > Stefan > > On 1 Sep 2018, at 06:16, "nek5000-users at lists.mcs.anl.gov > " > > wrote: > >> Hi neks, >> ? ?Recently I am working on an LES simulation in an annular pipe to >> see how transverse velocity goes with Re increasing. >> But I am quite wondering how I should set filterWeight and >> filterCutoffRatio in my example while in the turbChannel demo they are >> selected to 10 an 0.9 respectively as proposed value. >> They should be sensitive parameters in LES simulation, don't they? Is >> there any references concerned about them to help me see them more >> clearly? >> If it is convenient for you could you please reply me a url to the >> reference? Thank you so much for your assistence. >> > > >> best regard >> -- >> _______________________________________________ >> >> Nek5000-users mailing list >> >> Nek5000-users at lists.mcs.anl.gov >> >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Mon Sep 10 05:50:21 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 12:50:21 +0200 Subject: [Nek5000-users] Proper setup for AMG solver In-Reply-To: References: Message-ID: Hi Steffen, It is interesting that you find that the characteristic scheme is indeed faster; similar tests with a a pipe (a few years ago) have shown the opposite, which is why we ran our 2013 pipe using conventional time steppers. But I guess we should re-do these tests! I think your question regarding the temporal evolution is interesting, and I guess in the end this has to be judged using physical time scales (which scale in plus units for the velocity here). Not being too familiar with the characteristic scheme, what is really the smallest resolved time scale? Is it the actual time step, or the RK steps, or some intermediate scale? We have found that AMG is indeed surpassing XXT at some point, but I guess this is machine dependent (and number of processors in addition to number of elements). Do you have any issues with the setup of XXT taking quite some time? Best, Philipp On 2018-09-02 12:05, nek5000-users at lists.mcs.anl.gov wrote: > Hello Stefan & Paul, > > thanks for your suggestions. I use Nek5000 v17 (mver 17.0.4 in makenek). > > I have taken a closer look at the logfiles as suggested by Paul. It seems like I spend the most time for the scalar fields second most for the velocity fields and pressure solve is the smallest share. The numbers below are in seconds for a typical timestep without calculating statistics or writing out files at Re_b = 5300 (Re_t = u_t D / nu = 360, 768 cores) and Re_b = 37700 (Re_t = 2000, 6144 cores). > I use projection for all 10 scalar fields. > > 360: > Scalars done 0.064 > Fluid done 0.039 > U-PRES gmres 0.021 > Step 0.127 > > 2000: > Scalars done 0.94 > Fluid done 0.51 > U-PRES gmres 0.21 > Step 1.72 > > > Paul, could you elaborate on why you would not use characteristics when running 10 scalar fields? > In my short tests at Re_b=5300, it appears to be more time consuming per step but due to the increase in DT it is worth it to go to characteristics. The time per timestep for targetCFL=2.0 increases by a factor of 3 but DT is increased by a factor of 6. > Besides, are such increased timesteps still small enough to capture the temporal evolution of the flow? > > > When I am back on my workstation, I will create a different mesh with lx1=8, lxd=10 and run with the settings Stefan suggested. I expect to get a significant speedup when using the lower tolerances also for velocity and scalars and changing to characteristics. > > I know about the limit of 350k elements for XXT as I have once commented out the part of the code where this is tested. > Since for my setups AMG was always slower than XXT, I am thinking about sticking to XXT. > Is there any other reason than your experience with AMG and XXT at large number of elements to enforce AMG? > > > Best Regards, > Steffen > > > > Message: 5 > Date: Sat, 1 Sep 2018 17:10:54 +0200 > From: nek5000-users at lists.mcs.anl.gov > To: nek5000-users at lists.mcs.anl.gov > Subject: Re: [Nek5000-users] Proper setup for AMG solver > Message-ID: > > Content-Type: text/plain; charset=utf-8 > > Try to use > > lx1=8/lxd=10 with a (potentially) finer mesh > BDF2 + OIFS with a targetCFL=3.5 > set dt = 0 (this will adjust dt to targetCFL) > pressure tol = 1e-5 (residual projection turned on) 1e-6 for velocity and scalars (residual projection turned off) > > Note, a mesh with more than 350k elements requires AMG. The default parameters are fine. > > What version of Nek5000 are you using? > > Cheers, > Stefan > > > -----Original message----- >> From:nek5000-users at lists.mcs.anl.gov >> Sent: Saturday 1st September 2018 16:50 >> To: nek5000-users at lists.mcs.anl.gov >> Subject: [Nek5000-users] Proper setup for AMG solver >> >> Dear Nek users & experts, >> >> I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. >> >> Additionally, I solve for 10 thermal fields being treated as passive scalars. >> >> The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. >> >> As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). >> >> As the simulation is still very heavy, I have been looking into ways for speeding it up. >> I found some good suggestions here: >> http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed >> >> and here (older version?) >> http://nek5000.github.io/NekDoc/large_scale.html >> >> >> However, I have some questions regarding these suggestions. >> 1) Dealiasing: >> Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? >> >> 2) Tolerances: >> I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear >> stress budgets or budget of turbulent heat fluxes? >> >> 3) Time discretisation: BDF2 and OIFS with Courant=2-5 >> If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? >> >> 4) AMG instead of XXT: >> I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this >> 4.1) Set SEMG_AMG in the par file. >> 4.2) Run the simulation once to dump amg files. >> 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) >> 4.4) Run the simulation. >> Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. >> >> >> I am grateful for any advice regarding these aspects. >> >> Best Regards, >> Steffen >> >> >> >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > > ------------------------------ > > Message: 6 > Date: Sat, 1 Sep 2018 15:15:29 +0000 > From: nek5000-users at lists.mcs.anl.gov > To: "nek5000-users at lists.mcs.anl.gov" > > Subject: Re: [Nek5000-users] Proper setup for AMG solver > Message-ID: > > Content-Type: text/plain; charset="us-ascii" > > > How much time are you spending in your scalar fields? > > > Do you have projection turned on for all these fields? > > > grep tep logfile > > > will tell you how much time per step > > > grep gmr logfile will tell you how much time in the pressure on each step > > > What's left over is mostly passive scalar, unless you are using characteristics. > > > I would not recommend characteristics when running 10 scalar fields. > > > Paul > > > ________________________________ > From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov > Sent: Saturday, September 1, 2018 9:50:38 AM > To: nek5000-users at lists.mcs.anl.gov > Subject: [Nek5000-users] Proper setup for AMG solver > > > Dear Nek users & experts, > > I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. > Additionally, I solve for 10 thermal fields being treated as passive scalars. > > The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. > As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). > > As the simulation is still very heavy, I have been looking into ways for speeding it up. > I found some good suggestions here: > http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed > > and here (older version?) > http://nek5000.github.io/NekDoc/large_scale.html > > > However, I have some questions regarding these suggestions. > 1) Dealiasing: > Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? > > 2) Tolerances: > I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes? > > 3) Time discretisation: BDF2 and OIFS with Courant=2-5 > If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? > > 4) AMG instead of XXT: > I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this > 4.1) Set SEMG_AMG in the par file. > 4.2) Run the simulation once to dump amg files. > 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) > 4.4) Run the simulation. > Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. > > > I am grateful for any advice regarding these aspects. > > Best Regards, > Steffen > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > > ------------------------------ > > End of Nek5000-users Digest, Vol 115, Issue 1 > ********************************************* > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Mon Sep 10 06:12:49 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 13:12:49 +0200 Subject: [Nek5000-users] Proper setup for AMG solver In-Reply-To: References: Message-ID: The slow AMG problem he reported was related to an old (buggy) version of amg_hypre. Note, in my experience AMG is faster at least for non-power-of-two ranks (quite common today) independent of the problem size. Regarding OIFS it?s important to use BDF2 with a possible low lxd otherwise it?s quite expensive. It depends heavily on how expensive the pressure solve is. Cheers Stefan > On 10 Sep 2018, at 12:51, "nek5000-users at lists.mcs.anl.gov" wrote: > > Hi Steffen, > It is interesting that you find that the characteristic scheme is indeed > faster; similar tests with a a pipe (a few years ago) have shown the > opposite, which is why we ran our 2013 pipe using conventional time > steppers. But I guess we should re-do these tests! > > I think your question regarding the temporal evolution is interesting, > and I guess in the end this has to be judged using physical time scales > (which scale in plus units for the velocity here). Not being too > familiar with the characteristic scheme, what is really the smallest > resolved time scale? Is it the actual time step, or the RK steps, or > some intermediate scale? > > We have found that AMG is indeed surpassing XXT at some point, but I > guess this is machine dependent (and number of processors in addition to > number of elements). Do you have any issues with the setup of XXT taking > quite some time? > > Best, > Philipp > > >> On 2018-09-02 12:05, nek5000-users at lists.mcs.anl.gov wrote: >> Hello Stefan & Paul, >> >> thanks for your suggestions. I use Nek5000 v17 (mver 17.0.4 in makenek). >> >> I have taken a closer look at the logfiles as suggested by Paul. It seems like I spend the most time for the scalar fields second most for the velocity fields and pressure solve is the smallest share. The numbers below are in seconds for a typical timestep without calculating statistics or writing out files at Re_b = 5300 (Re_t = u_t D / nu = 360, 768 cores) and Re_b = 37700 (Re_t = 2000, 6144 cores). >> I use projection for all 10 scalar fields. >> >> 360: >> Scalars done 0.064 >> Fluid done 0.039 >> U-PRES gmres 0.021 >> Step 0.127 >> >> 2000: >> Scalars done 0.94 >> Fluid done 0.51 >> U-PRES gmres 0.21 >> Step 1.72 >> >> >> Paul, could you elaborate on why you would not use characteristics when running 10 scalar fields? >> In my short tests at Re_b=5300, it appears to be more time consuming per step but due to the increase in DT it is worth it to go to characteristics. The time per timestep for targetCFL=2.0 increases by a factor of 3 but DT is increased by a factor of 6. >> Besides, are such increased timesteps still small enough to capture the temporal evolution of the flow? >> >> >> When I am back on my workstation, I will create a different mesh with lx1=8, lxd=10 and run with the settings Stefan suggested. I expect to get a significant speedup when using the lower tolerances also for velocity and scalars and changing to characteristics. >> >> I know about the limit of 350k elements for XXT as I have once commented out the part of the code where this is tested. >> Since for my setups AMG was always slower than XXT, I am thinking about sticking to XXT. >> Is there any other reason than your experience with AMG and XXT at large number of elements to enforce AMG? >> >> >> Best Regards, >> Steffen >> >> >> >> Message: 5 >> Date: Sat, 1 Sep 2018 17:10:54 +0200 >> From: nek5000-users at lists.mcs.anl.gov >> To: nek5000-users at lists.mcs.anl.gov >> Subject: Re: [Nek5000-users] Proper setup for AMG solver >> Message-ID: >> >> Content-Type: text/plain; charset=utf-8 >> >> Try to use >> >> lx1=8/lxd=10 with a (potentially) finer mesh >> BDF2 + OIFS with a targetCFL=3.5 >> set dt = 0 (this will adjust dt to targetCFL) >> pressure tol = 1e-5 (residual projection turned on) 1e-6 for velocity and scalars (residual projection turned off) >> >> Note, a mesh with more than 350k elements requires AMG. The default parameters are fine. >> >> What version of Nek5000 are you using? >> >> Cheers, >> Stefan >> >> >> -----Original message----- >>> From:nek5000-users at lists.mcs.anl.gov >>> Sent: Saturday 1st September 2018 16:50 >>> To: nek5000-users at lists.mcs.anl.gov >>> Subject: [Nek5000-users] Proper setup for AMG solver >>> >>> Dear Nek users & experts, >>> >>> I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. >>> >>> Additionally, I solve for 10 thermal fields being treated as passive scalars. >>> >>> The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. >>> >>> As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). >>> >>> As the simulation is still very heavy, I have been looking into ways for speeding it up. >>> I found some good suggestions here: >>> http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed >>> >>> and here (older version?) >>> http://nek5000.github.io/NekDoc/large_scale.html >>> >>> >>> However, I have some questions regarding these suggestions. >>> 1) Dealiasing: >>> Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? >>> >>> 2) Tolerances: >>> I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear >>> stress budgets or budget of turbulent heat fluxes? >>> >>> 3) Time discretisation: BDF2 and OIFS with Courant=2-5 >>> If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? >>> >>> 4) AMG instead of XXT: >>> I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this >>> 4.1) Set SEMG_AMG in the par file. >>> 4.2) Run the simulation once to dump amg files. >>> 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) >>> 4.4) Run the simulation. >>> Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. >>> >>> >>> I am grateful for any advice regarding these aspects. >>> >>> Best Regards, >>> Steffen >>> >>> >>> >>> _______________________________________________ >>> Nek5000-users mailing list >>> Nek5000-users at lists.mcs.anl.gov >>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> >> >> ------------------------------ >> >> Message: 6 >> Date: Sat, 1 Sep 2018 15:15:29 +0000 >> From: nek5000-users at lists.mcs.anl.gov >> To: "nek5000-users at lists.mcs.anl.gov" >> >> Subject: Re: [Nek5000-users] Proper setup for AMG solver >> Message-ID: >> >> Content-Type: text/plain; charset="us-ascii" >> >> >> How much time are you spending in your scalar fields? >> >> >> Do you have projection turned on for all these fields? >> >> >> grep tep logfile >> >> >> will tell you how much time per step >> >> >> grep gmr logfile will tell you how much time in the pressure on each step >> >> >> What's left over is mostly passive scalar, unless you are using characteristics. >> >> >> I would not recommend characteristics when running 10 scalar fields. >> >> >> Paul >> >> >> ________________________________ >> From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov >> Sent: Saturday, September 1, 2018 9:50:38 AM >> To: nek5000-users at lists.mcs.anl.gov >> Subject: [Nek5000-users] Proper setup for AMG solver >> >> >> Dear Nek users & experts, >> >> I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. >> Additionally, I solve for 10 thermal fields being treated as passive scalars. >> >> The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. >> As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). >> >> As the simulation is still very heavy, I have been looking into ways for speeding it up. >> I found some good suggestions here: >> http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed >> >> and here (older version?) >> http://nek5000.github.io/NekDoc/large_scale.html >> >> >> However, I have some questions regarding these suggestions. >> 1) Dealiasing: >> Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? >> >> 2) Tolerances: >> I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes? >> >> 3) Time discretisation: BDF2 and OIFS with Courant=2-5 >> If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? >> >> 4) AMG instead of XXT: >> I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this >> 4.1) Set SEMG_AMG in the par file. >> 4.2) Run the simulation once to dump amg files. >> 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) >> 4.4) Run the simulation. >> Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. >> >> >> I am grateful for any advice regarding these aspects. >> >> Best Regards, >> Steffen >> >> >> >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: >> >> ------------------------------ >> >> Subject: Digest Footer >> >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> >> >> ------------------------------ >> >> End of Nek5000-users Digest, Vol 115, Issue 1 >> ********************************************* >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Mon Sep 10 07:54:04 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 12:54:04 +0000 Subject: [Nek5000-users] Proper setup for AMG solver In-Reply-To: References: , Message-ID: Hi Steffen, Sorry for the delayed response. My feeling re. characteristics + 10 PS is the following. A) characteristics is designed to trade off more work on advection for less work on the implicit part. The principal implicit cost in NS is the pressure solve. Typically, we see that we spend 50% of our time in advection when using the characteristics scheme (but this can vary, depending on whether you go for 2nd-order (recommended, as Stefan suggested) or 3rd-order in time (not recommend for characteristics), and on the amount of over-integration for the advection - typically lxd=3*lx1/2, but you can get away with less and might want to do so for characteristics. In the case of multiple PS, you increase your advection work proportionally, but the pressure work stays the same. Thus, an approach that puts more emphasis on the expensive part will ultimately not be a winning strategy. Just to give you some estimates: Standard advection (IFCHAR=F): 1 advection evaluation per time step for each component of velocity and for each passive scalar: Advection work unit: W ~ 18(MN)^2 ops, where M=3/2N for standard dealiasing. (This is a crude estimate.) So, IFCHAR=F --> (3+PS)*W advection work per step, for "PS" passive scalars, including temperature. With IFCHAR=T, assume you are running at a CFL of 2.0 (with p26==1, which implies one RK4 step to achieve CFL=2). For 2nd-order, there are two RK4 steps, each requiring 4 sub steps, so the advection work is 8 x (3+PS)*W. If you run 3rd-order, it is 12x(3+PS)*W because 3rd-order characteristics requires 3 RK4 steps. More details can be found in some old notes posted here: http://www.mcs.anl.gov/~fischer/oifs.pdf and in a forthcoming paper that's currently under review. B) If the time savings is not significant, I generally prefer to set IFCHAR=F and then run 3rd order. The std. BDFk/EXTk (IFCHAR=F) case requires only (3+PS)W advection work per step, independent of temporal order. hth Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Monday, September 10, 2018 6:12:49 AM To: nek5000-users at lists.mcs.anl.gov Subject: Re: [Nek5000-users] Proper setup for AMG solver The slow AMG problem he reported was related to an old (buggy) version of amg_hypre. Note, in my experience AMG is faster at least for non-power-of-two ranks (quite common today) independent of the problem size. Regarding OIFS it?s important to use BDF2 with a possible low lxd otherwise it?s quite expensive. It depends heavily on how expensive the pressure solve is. Cheers Stefan > On 10 Sep 2018, at 12:51, "nek5000-users at lists.mcs.anl.gov" wrote: > > Hi Steffen, > It is interesting that you find that the characteristic scheme is indeed > faster; similar tests with a a pipe (a few years ago) have shown the > opposite, which is why we ran our 2013 pipe using conventional time > steppers. But I guess we should re-do these tests! > > I think your question regarding the temporal evolution is interesting, > and I guess in the end this has to be judged using physical time scales > (which scale in plus units for the velocity here). Not being too > familiar with the characteristic scheme, what is really the smallest > resolved time scale? Is it the actual time step, or the RK steps, or > some intermediate scale? > > We have found that AMG is indeed surpassing XXT at some point, but I > guess this is machine dependent (and number of processors in addition to > number of elements). Do you have any issues with the setup of XXT taking > quite some time? > > Best, > Philipp > > >> On 2018-09-02 12:05, nek5000-users at lists.mcs.anl.gov wrote: >> Hello Stefan & Paul, >> >> thanks for your suggestions. I use Nek5000 v17 (mver 17.0.4 in makenek). >> >> I have taken a closer look at the logfiles as suggested by Paul. It seems like I spend the most time for the scalar fields second most for the velocity fields and pressure solve is the smallest share. The numbers below are in seconds for a typical timestep without calculating statistics or writing out files at Re_b = 5300 (Re_t = u_t D / nu = 360, 768 cores) and Re_b = 37700 (Re_t = 2000, 6144 cores). >> I use projection for all 10 scalar fields. >> >> 360: >> Scalars done 0.064 >> Fluid done 0.039 >> U-PRES gmres 0.021 >> Step 0.127 >> >> 2000: >> Scalars done 0.94 >> Fluid done 0.51 >> U-PRES gmres 0.21 >> Step 1.72 >> >> >> Paul, could you elaborate on why you would not use characteristics when running 10 scalar fields? >> In my short tests at Re_b=5300, it appears to be more time consuming per step but due to the increase in DT it is worth it to go to characteristics. The time per timestep for targetCFL=2.0 increases by a factor of 3 but DT is increased by a factor of 6. >> Besides, are such increased timesteps still small enough to capture the temporal evolution of the flow? >> >> >> When I am back on my workstation, I will create a different mesh with lx1=8, lxd=10 and run with the settings Stefan suggested. I expect to get a significant speedup when using the lower tolerances also for velocity and scalars and changing to characteristics. >> >> I know about the limit of 350k elements for XXT as I have once commented out the part of the code where this is tested. >> Since for my setups AMG was always slower than XXT, I am thinking about sticking to XXT. >> Is there any other reason than your experience with AMG and XXT at large number of elements to enforce AMG? >> >> >> Best Regards, >> Steffen >> >> >> >> Message: 5 >> Date: Sat, 1 Sep 2018 17:10:54 +0200 >> From: nek5000-users at lists.mcs.anl.gov >> To: nek5000-users at lists.mcs.anl.gov >> Subject: Re: [Nek5000-users] Proper setup for AMG solver >> Message-ID: >> >> Content-Type: text/plain; charset=utf-8 >> >> Try to use >> >> lx1=8/lxd=10 with a (potentially) finer mesh >> BDF2 + OIFS with a targetCFL=3.5 >> set dt = 0 (this will adjust dt to targetCFL) >> pressure tol = 1e-5 (residual projection turned on) 1e-6 for velocity and scalars (residual projection turned off) >> >> Note, a mesh with more than 350k elements requires AMG. The default parameters are fine. >> >> What version of Nek5000 are you using? >> >> Cheers, >> Stefan >> >> >> -----Original message----- >>> From:nek5000-users at lists.mcs.anl.gov >>> Sent: Saturday 1st September 2018 16:50 >>> To: nek5000-users at lists.mcs.anl.gov >>> Subject: [Nek5000-users] Proper setup for AMG solver >>> >>> Dear Nek users & experts, >>> >>> I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. >>> >>> Additionally, I solve for 10 thermal fields being treated as passive scalars. >>> >>> The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. >>> >>> As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). >>> >>> As the simulation is still very heavy, I have been looking into ways for speeding it up. >>> I found some good suggestions here: >>> http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed >>> >>> and here (older version?) >>> http://nek5000.github.io/NekDoc/large_scale.html >>> >>> >>> However, I have some questions regarding these suggestions. >>> 1) Dealiasing: >>> Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? >>> >>> 2) Tolerances: >>> I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear >>> stress budgets or budget of turbulent heat fluxes? >>> >>> 3) Time discretisation: BDF2 and OIFS with Courant=2-5 >>> If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? >>> >>> 4) AMG instead of XXT: >>> I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this >>> 4.1) Set SEMG_AMG in the par file. >>> 4.2) Run the simulation once to dump amg files. >>> 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) >>> 4.4) Run the simulation. >>> Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. >>> >>> >>> I am grateful for any advice regarding these aspects. >>> >>> Best Regards, >>> Steffen >>> >>> >>> >>> _______________________________________________ >>> Nek5000-users mailing list >>> Nek5000-users at lists.mcs.anl.gov >>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> >> >> ------------------------------ >> >> Message: 6 >> Date: Sat, 1 Sep 2018 15:15:29 +0000 >> From: nek5000-users at lists.mcs.anl.gov >> To: "nek5000-users at lists.mcs.anl.gov" >> >> Subject: Re: [Nek5000-users] Proper setup for AMG solver >> Message-ID: >> >> Content-Type: text/plain; charset="us-ascii" >> >> >> How much time are you spending in your scalar fields? >> >> >> Do you have projection turned on for all these fields? >> >> >> grep tep logfile >> >> >> will tell you how much time per step >> >> >> grep gmr logfile will tell you how much time in the pressure on each step >> >> >> What's left over is mostly passive scalar, unless you are using characteristics. >> >> >> I would not recommend characteristics when running 10 scalar fields. >> >> >> Paul >> >> >> ________________________________ >> From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov >> Sent: Saturday, September 1, 2018 9:50:38 AM >> To: nek5000-users at lists.mcs.anl.gov >> Subject: [Nek5000-users] Proper setup for AMG solver >> >> >> Dear Nek users & experts, >> >> I am currently running a turbulent pipe flow very similar to the simulations from El Khoury et al 2013. >> Additionally, I solve for 10 thermal fields being treated as passive scalars. >> >> The Reynolds number is the same as the highest in El Khoury (2013), Re_b = 37700. >> As I am using the relaxation term filtering (RT-Filter), I have a slightly lower resolution of about 250,000 elements at N=11 (5 times less than El Khoury). >> >> As the simulation is still very heavy, I have been looking into ways for speeding it up. >> I found some good suggestions here: >> http://nek5000.github.io/NekDoc/faq.html?highlight=amg#computational-speed >> >> and here (older version?) >> http://nek5000.github.io/NekDoc/large_scale.html >> >> >> However, I have some questions regarding these suggestions. >> 1) Dealiasing: >> Usually I use lxd = 3/2*lx1. Can I lower that or even use lxd=lx1? >> >> 2) Tolerances: >> I have tested to reduce the tolerance for pressure from 1e-8 to 5e-5 for a run at Re_b=5300 without any significant speedup. Would you consider 5e-5 for pressure accurate enough for evaluating statistics like turbulent kinetic energy budgets, Reynolds shear stress budgets or budget of turbulent heat fluxes? >> >> 3) Time discretisation: BDF2 and OIFS with Courant=2-5 >> If I go from BDF3/EXT3 at C=0.5 to BDF2/OIFS at C=5.0, will I not miss high frequency fluctuations in time, since DT is much larger? >> >> 4) AMG instead of XXT: >> I have tested AMG instead of XXT for both Re_b=5300 and Re_b=37700 without any speedup. Time/timestep is even higher with AMG. My workflow looks like this >> 4.1) Set SEMG_AMG in the par file. >> 4.2) Run the simulation once to dump amg files. >> 4.3) Run amg_hypre (Here I do not know which options to choose, thus I have only uses default settings) >> 4.4) Run the simulation. >> Maybe I should choose different options for amg_hypre, or should I rather use the amg_matlab2 tools? For the matlab tools I have not found an explanation on how to use them. >> >> >> I am grateful for any advice regarding these aspects. >> >> Best Regards, >> Steffen >> >> >> >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: >> >> ------------------------------ >> >> Subject: Digest Footer >> >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> >> >> ------------------------------ >> >> End of Nek5000-users Digest, Vol 115, Issue 1 >> ********************************************* >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Sep 10 09:19:00 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 16:19:00 +0200 Subject: [Nek5000-users] Proper setup for AMG solver In-Reply-To: References: Message-ID: Dear Nek experts, I wanted to add some remarks on my performance tests conducted last week mostly for the turbulent pipe flow at Re_b=5300 - The bad performance of the AMG solver was indeed due to an old version of amg_hypre, as Stefan mentioned already. For Re_b=5300 XXT and AMG showed the same performance. - Additionally, I could increase the performance by about 20% using the following settings: -- lxd=10 (instead of 12, for lx1=8) -- lx2=lx1-0 (PN-PN) -- lower tolerances for p=1e-5 (instead of 1e-8) and 1e-6 for velocity and passive scalars -- turning of projection except for the fields of low Prandtl number - Going from BDF3/EXT3 with a variable DT and targetCFL=0.5 to BDF2/OIFS with variable DT and targetCFL=0.5 resulted in a longer time / timestep (3x) but a also a larger average DT (6x). Assuming the collected statistics require a similar averaging time in wash-outs, BDF2/OIFS is advantageous (even for 10 passive scalars). Before this discussion, I was running at constant DT and collected statistics every 10th step. Now with a larger DT using characteristics, should I collect statistics for each step? - Regarding the filterWeight and filterCutoffRatio, I found out (with Stefan's help) that the divergence error in L2 norm (when using PN-PN) is affected by filterWeight. For the setup at Re_b=5300 a filterWeight of 27 seemed to be OK considering a divergence error in L2 norm of L2=3e-2, whereas a weight of 54 resulted in L2=1e-1. How these settings will affect the statistics, I am testing now. Thank you all for your help and suggestions. Cheers, Steffen From nek5000-users at lists.mcs.anl.gov Mon Sep 10 10:04:21 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 15:04:21 +0000 Subject: [Nek5000-users] Proper setup for AMG solver In-Reply-To: References: , Message-ID: Hi Steffen, Thanks for the note. I would recommend fixed dt at a value near your target CFL. There are several reasons - 1. I doubt you gain that much with variable dt 2. I personally don't like Nek's auto-dt selection scheme 3. You have to reorthogonalize the projection basis every time dt changes (expensive) Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Monday, September 10, 2018 9:19:00 AM To: nek5000-users at lists.mcs.anl.gov Subject: Re: [Nek5000-users] Proper setup for AMG solver Dear Nek experts, I wanted to add some remarks on my performance tests conducted last week mostly for the turbulent pipe flow at Re_b=5300 - The bad performance of the AMG solver was indeed due to an old version of amg_hypre, as Stefan mentioned already. For Re_b=5300 XXT and AMG showed the same performance. - Additionally, I could increase the performance by about 20% using the following settings: -- lxd=10 (instead of 12, for lx1=8) -- lx2=lx1-0 (PN-PN) -- lower tolerances for p=1e-5 (instead of 1e-8) and 1e-6 for velocity and passive scalars -- turning of projection except for the fields of low Prandtl number - Going from BDF3/EXT3 with a variable DT and targetCFL=0.5 to BDF2/OIFS with variable DT and targetCFL=0.5 resulted in a longer time / timestep (3x) but a also a larger average DT (6x). Assuming the collected statistics require a similar averaging time in wash-outs, BDF2/OIFS is advantageous (even for 10 passive scalars). Before this discussion, I was running at constant DT and collected statistics every 10th step. Now with a larger DT using characteristics, should I collect statistics for each step? - Regarding the filterWeight and filterCutoffRatio, I found out (with Stefan's help) that the divergence error in L2 norm (when using PN-PN) is affected by filterWeight. For the setup at Re_b=5300 a filterWeight of 27 seemed to be OK considering a divergence error in L2 norm of L2=3e-2, whereas a weight of 54 resulted in L2=1e-1. How these settings will affect the statistics, I am testing now. Thank you all for your help and suggestions. Cheers, Steffen _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Sep 10 10:27:15 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 16:27:15 +0100 Subject: [Nek5000-users] findpts Message-ID: Dear Nek, When I look at the hemi.usr, the only subroutine it calls for interpolating velocities for list of points as shown below. call interp_nfld(fpts,fwrk,ndim,pts(1),pts(1+n),pts(2*n+1), $ n,iwk,rwk,INTP_NMAX,.true.,intp_h) Could you please explain more? What I want are processor number and ix iy and iz for each point. Thanks so much for your help. Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Sep 10 11:56:12 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 10 Sep 2018 16:56:12 +0000 Subject: [Nek5000-users] findpts In-Reply-To: References: Message-ID: Dear Jian, I recommend using this interface from the hemi.usr file. hth, Paul c----------------------------------------------------------------------- subroutine interp_v(uvw,xyz,n) c c evaluate velocity for list of points xyz c include 'SIZE' include 'TOTAL' real uvw(ldim,n),xyz(ldim,n) ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Monday, September 10, 2018 10:27:15 AM To: nek5000-users at lists.mcs.anl.gov Subject: Re: [Nek5000-users] findpts Dear Nek, When I look at the hemi.usr, the only subroutine it calls for interpolating velocities for list of points as shown below. call interp_nfld(fpts,fwrk,ndim,pts(1),pts(1+n),pts(2*n+1), $ n,iwk,rwk,INTP_NMAX,.true.,intp_h) Could you please explain more? What I want are processor number and ix iy and iz for each point. Thanks so much for your help. Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 12 08:05:16 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 12 Sep 2018 15:05:16 +0200 Subject: [Nek5000-users] Traction BC Message-ID: Dear Neks, I have modified the turbChannel example (available in v17.0) to a WRLES flat plate test case and validated it against DNS data. One step further, I am now imposing traction boundary condition on the wall, i.e tauw (TRX,TRZ) and computing the wall shear stress thanks to torque_calc routine. The computed wall shear stress corresponds to the one I am imposing. However the fluid accelerates at the wall and never converges to a value. 1. The wall shear stress in correct but the velocity increases over time. How can this be possible ? 2. Are you aware of the formalism surrounding traction boundary condition in the discrete equations ? Sincerely, Armand. -- *Armand Shams* Master Thesis Intern D?partement a?rodynamique, a?ro?lasticit?, acoustique MSAT T?l: +33 6 99 51 15 84 ONERA?-?The French Aerospace Lab?-?Centre de Meudon 8, rue des Vertugadins - 92190 MEUDON Nous suivre sur : www.onera.fr ?| Twitter ?| LinkedIn ?| Facebook Avertissement/disclaimer https://www.onera.fr/en/emails-terms -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ndkacandgeeinlpj.gif Type: image/gif Size: 1041 bytes Desc: not available URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 12 10:33:17 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 12 Sep 2018 17:33:17 +0200 (CEST) Subject: [Nek5000-users] Solution Diverge in the begeninng for NACA0012 Message-ID: Hi NEKs, It has been a long time since I came up with something. Here I have a problem and I just would like to know that whether have you experienced it before? I am running a simulation of self-sustained pitching of NACA0012 at Re 62000 and at the initial steps itself, the solution diverges due to a rapid increment of velocity at the tip of the trailing edge. I compute the torque from elastic axis (which is 18.6% from leading edge) and used equation of motion to compute the mesh velocity to pitch the NACA0012. My Moment of Inertia is 0.0014, Stiffness is 0.3 and Damping Coefficient is 0.002. In the begeinning of simulation the torq increases rapidly and gradually the mesh velocity increases and collpase. I divided the moment by 10 just to see at least my code works. And it worked. But as the moment is divided by 10, I get a less amplitude of pitching . So my question is have you experienced like this before? If yes could you please tell me why it because of? If not, What can be the reason for a large velocity at the tip of trailing edge? NOTE : I have taken air as the fluid so I think the added mass effect is negligible. But do you think is there any possibility of that? Any ideas are welcome. Thanks Sijo GEORGE -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Fri Sep 14 16:03:31 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 14 Sep 2018 22:03:31 +0100 Subject: [Nek5000-users] findpts Message-ID: Dear Paul, When I looked at the interp_v in hemi case, I didn't see it returns the element number, processor number, and ix, iy and iz (ranging from 1 to nx1, ny1, and nz1, respectively). So for that purpose, I worte two subroutines which are similar to interp_v and intpts to return the data above. But the results show those data are not reasonable. Would you please help me to look at the subroutines below? Thanks so much. I just took the case with one point as example. Results are call interp_pp(piel,pproc,pix,piy,piz,ppos0,lpart) ppx: -9.995177957692734E-002 pix: 0.523597350230634 === ppy: -0.232351957858704 piy: 0.605225220747395 === ppz: 6.08294103920131 piz: 4.940656458412465E-324 === piel: 3.695611030892524E-321 pproc: 1.432790372939615E-322 *ppx ppy ppy are local point's coordinates to be interpolated C----------------------------------------------------------------------- subroutine interp_p(fieldin,nfld,pts,n,filedout,ifot,ifpts,ih, $ elid,proc,rst) c based on intpts() c in: c fieldin ... input field(s) to interpolate (lelt*lxyz,nfld) c nfld ... number of fields in fieldin c pts ... packed list of interpolation points c pts:=[x(1)...x(n),y(1)...y(n),z(1)...z(n)] c n ... local number of interpolation points c ifto ... transpose output (n,nfld)^T = (nfld,n) c itpts ... find interpolation points c ih ... interpolation handle c out: c fieldout ... packed list of interpolated values (n,nfld) c elid ... element on remote processor in which the point was found c proc ... remote processor on which the point was found c rst ... parametric coordinates for point include 'SIZE' include 'TOTAL' real fieldin(1), fieldout(1) real pts(1) real dist(lpart) ! squared distance real rst(lpart*ldim) integer rcode(lpart),elid(lpart),proc(lpart) integer nn(2) logical ifot,ifpts ifjac = .true. ! output transpose (of Jacobian) ifpts = .true. ! find points if(nio.eq.0) write(6,*) 'call intpts in interp_p(1)' if(n.gt.lpart) then write(6,*) $ 'ABORT: intpts() n>lpart, increase lpart in SIZE', n, lpart call exitt call exitt endif c ! locate points (iel,iproc,r,s,t) nfail = 0 if(ifpts) then if(nio.eq.0) write(6,*) 'call findpts in interp_p(2)' call findpts(ih,rcode,1, $ proc,1, $ elid,1, $ rst,ndim, $ dist,1, $ pts( 1),1, $ pts( n+1),1, $ pts(2*n+1),1,n) do in=1,n ! check return code if(rcode(in).eq.1) then if(dist(in).gt.1e-12) then nfail = nfail + 1 if (nfail.le.5) write(6,'(a,1p4e15.7)') $ ' WARNING: point on boundary or outside the mesh xy[z]d^2: ', $ (pts(in+k*n),k=0,ndim-1),dist(in) endif elseif(rcode(in).eq.2) then nfail = nfail + 1 if (nfail.le.5) write(6,'(a,1p3e15.7)') $ ' WARNING: point not within mesh xy[z]: !', $ (pts(in+k*n),k=0,ndim-1) endif enddo endif c ! evaluate inut field at given points ltot = lelt*lx1*ly1*lz1 do ifld = 1,nfld iin = (ifld-1)*ltot + 1 iout = (ifld-1)*n + 1 is_out = 1 if(ifot) then ! transpose output iout = ifld is_out = nfld endif call findpts_eval(ih,fieldout(iout),is_out, $ rcode,1, $ proc,1, $ elid,1, $ rst,ndim,n, $ fieldin(iin)) enddo nn(1) = iglsum(n,1) nn(2) = iglsum(nfail,1) if(nio.eq.0) then write(6,1) nn(1),nn(2) 1 format(' total number of points = ',i12,/,' failed = ' $ ,i12,/,' done :: intpts') endif return end c----------------------------------------------------------------------- subroutine interp_pp(elidp,procp,rr,ss,tt,xyz,n) !evaluate elid,rst for xyz c based on interp_v() include 'SIZE' include 'TOTAL' real rstp(lpart*ldim),rr(lpart),ss(lpart),tt(lpart) integer elidp(lpart),procp(lpart) real xyz(ldim,n) logical ifjac,ifpts parameter(nmax=lpart,nfldmax=ldim) common /rv_intp/ pts(ldim*nmax) common /iv_intp/ ihandle common /outtmp/ wrk(lx1*ly1*lz1*lelt,nfldmax) integer icalld,e save icalld data icalld /0/ nxyz = nx1*ny1*nz1 ntot = nxyz*nelt if (n.gt.nmax) call exitti ('ABORT: interp_v() n > nmax!$',n) if (nelgt.ne.nelgv) call exitti $ ('ABORT: interp_v() nelgt.ne.nelgv not yet supported!$',nelgv) do i=1,n ! ? not moving -> save? pts(i) = xyz(1,i) pts(i + n) = xyz(2,i) if (if3d) pts(i + n*2) = xyz(3,i) enddo if (icalld.eq.0) then ! interpolation setup !? intpts_done(ih_intp_v)? icalld = 1 tolin = 1.e-8 call intpts_setup(tolin,ihandle) endif nflds = ndim ! number of fields to interpolate ! pack working array call opcopy(wrk(1,1),wrk(1,2),wrk(1,3),vx,vy,vz) ! interpolate ifjac = .true. ! output transpose (of Jacobian) ifpts = .true. ! find points call interp_p(wrk,nflds,pts,n,uvw,ifjac,ifpts,ihandle $ ,elidp,procp,rstp) do i=1,n rr(i) = rstp(i+n) ss(i) = rstp(i+2*n) tt(i) = rstp(i+3*n) enddo return end Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sun Sep 16 10:56:57 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sun, 16 Sep 2018 15:56:57 +0000 Subject: [Nek5000-users] findpts Message-ID: Hi Jiang, Please see this article to see how findpts works: https://www.kth.se/social/files/5b14371756be5b899ec4af73/noorani_peplinski_schlatter_2015.pdf . Also see this: https://github.com/gslib/gslib/blob/master/src/findpts.c#L139 For a given point, findpts does not return ix,iy,iz inside an element.. That does not make sense because likely the sought point does not coincide with a GLL point. findpts returns the coordinates of the sought point in reference space i.e. r,s,t in [-1,1] inside that element. If you are not sure what reference space coordinates are, I would suggest reading up on it. Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Sep 17 03:05:02 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 17 Sep 2018 09:05:02 +0100 Subject: [Nek5000-users] findpts Message-ID: Dear Ketan, Thanks for your reply. May I ask why the element number and processor number output are not in a reasonable range? The code are https://lists.mcs.anl.gov/mailman/htdig/nek5000-users/2018-September/005631.html After calling 'call interp_pp(piel,pproc,pix,piy,piz,ppos0,lpart)' piel (element number): 3.695611030892524E-321 pproc (processor number): 1.432790372939615E-322 Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Sep 17 05:15:00 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 17 Sep 2018 11:15:00 +0100 Subject: [Nek5000-users] changing ffx Message-ID: Hi Nek, Do you know how to change ffx in different grid points using rst, element number and processor number obtained by findpts? Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Sep 17 07:31:52 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 17 Sep 2018 12:31:52 +0000 Subject: [Nek5000-users] Shear-periodic boundary conditions Message-ID: Hi Neks We try to simulate a homogeneous shear flow with shear-periodic boundary conditions. In x and z direction, the boundary conditions are periodic. But in y direction, the boundary conditions are f(x, L, z) = f(x - S*t*L, 0, z) where f(x,y,z) is any field, S is the shear rate and L is the domain size in y. How could we implement these boundary conditions? (In the moment, we specify Dirichlet boundary conditions at y = 0 and y = L and interpolate the fields with the spectral interpolation routine.) Thank you for your support. Best, Philipp -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Tue Sep 18 06:09:13 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Tue, 18 Sep 2018 12:09:13 +0100 Subject: [Nek5000-users] rst to ix iy iz Message-ID: Dear nek, Do you know how to convert r, s, t in [-1,1] to ix, iy, iz in an element? Actually I didn't see too many documents about reference space coordinates. Can you please provide some links for that? Thanks Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Tue Sep 18 07:10:45 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Tue, 18 Sep 2018 12:10:45 +0000 Subject: [Nek5000-users] rst to ix iy iz In-Reply-To: References: Message-ID: Dear Jian, There are an infinite number of r values but only a finite number of ix values, so there is no one-to-one mapping of r to ix. If you want the closest one, you could check against the positions of the Gauss-Lobatto-Legendre points, zgm1(ix,1), for ix=1,...,lx1. Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Tuesday, September 18, 2018 6:09 AM To: nek5000-users at lists.mcs.anl.gov Subject: [Nek5000-users] rst to ix iy iz Dear nek, Do you know how to convert r, s, t in [-1,1] to ix, iy, iz in an element? Actually I didn't see too many documents about reference space coordinates. Can you please provide some links for that? Thanks Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Tue Sep 18 08:50:16 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Tue, 18 Sep 2018 15:50:16 +0200 Subject: [Nek5000-users] rst to ix iy iz In-Reply-To: References: Message-ID: Hi, I guess the question is what you would like to do. From your other posts I assume that you would like to use interpolation to compute the value of the forcing that you can then input in userf at grid points ix,iy,iz. If this is correct, I guess what you would need to do is to regularise the forcing by spreading it among a number of actual grid points, e.g. using a Gaussian spreading which is centered in the actual point of the forcing which does not need to coincide with a grid point. philipp On 2018-09-18 14:10, nek5000-users at lists.mcs.anl.gov wrote: > > Dear Jian, > > > There are an infinite number of r values but only a finite number > > of ix values, so there is no one-to-one mapping of r to ix. > > > If you want the closest one, you could check against the positions of > > the Gauss-Lobatto-Legendre points, zgm1(ix,1), for ix=1,...,lx1. > > > Paul > > > > ------------------------------------------------------------------------ > *From:* Nek5000-users on > behalf of nek5000-users at lists.mcs.anl.gov > *Sent:* Tuesday, September 18, 2018 6:09 AM > *To:* nek5000-users at lists.mcs.anl.gov > *Subject:* [Nek5000-users] rst to ix iy iz > Dear nek, > > Do you know how to convert r, s, t in [-1,1] to ix, iy, iz in an element? > > Actually I didn't see too many documents about reference space > coordinates. Can you please provide some links for that? Thanks > > Kind regards, > > Jian > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Tue Sep 18 14:18:51 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Tue, 18 Sep 2018 20:18:51 +0100 Subject: [Nek5000-users] Mesh generation pretex Message-ID: Hello, I would like to create a mesh for a 3d annulus geometry. The figure for the annulus is attached. I am able to create a 2d trapezium. Is there an option for solid of revolution of a plane around an axis in pretex? Something like n2to3, but in the azimuthal direction. I checked the options 4 CURVE SIDES in pretex, but there isn't one. Any pointers on how to make this grid? Any help is apperciatedThank you in advance. Regards, Sandeep [image: Annulus.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Annulus.png Type: image/png Size: 21592 bytes Desc: not available URL: From nek5000-users at lists.mcs.anl.gov Tue Sep 18 21:37:29 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 19 Sep 2018 02:37:29 +0000 Subject: [Nek5000-users] Mesh generation pretex In-Reply-To: References: Message-ID: Dear Sandeep, The attached files will build the annulus mesh. Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Tuesday, September 18, 2018 2:18:51 PM To: nek5000-users at lists.mcs.anl.gov Subject: [Nek5000-users] Mesh generation pretex Hello, I would like to create a mesh for a 3d annulus geometry. The figure for the annulus is attached. I am able to create a 2d trapezium. Is there an option for solid of revolution of a plane around an axis in pretex? Something like n2to3, but in the azimuthal direction. I checked the options 4 CURVE SIDES in pretex, but there isn't one. Any pointers on how to make this grid? Any help is apperciatedThank you in advance. Regards, Sandeep [Annulus.png] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Annulus.png Type: image/png Size: 21592 bytes Desc: Annulus.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: t.tgz Type: application/x-gzip Size: 3493 bytes Desc: t.tgz URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 19 05:57:47 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 19 Sep 2018 11:57:47 +0100 Subject: [Nek5000-users] rst to ix iy iz Message-ID: Dear Philipp and Paul, Thanks for your replies. Yes, what I am trying to do is to change the force on every grid point. Now I have all the things without finding the closest grid point. So I am thinking to use rst and element numbers obtained from findpts to locate the closest grid point. If I got which grid point it is (specifically on having element number and ix iy iz in that element), that would be sorted. May I know further to the GLL points. As far as I can see, the zgm1(ix,1) ranges from -1 (ix=1) to 1 (ix=nx1). Also zgm1(iy,2) and zgm1(iz,3) are for y and z, respectively. (1). Are they corresponding to rst? If so, do you mean I can find the closest zgm1 and determine the ix iy iz. (2). How can I know which element zgm1 refers to? Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 19 06:28:11 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 19 Sep 2018 13:28:11 +0200 Subject: [Nek5000-users] rst to ix iy iz In-Reply-To: References: Message-ID: Hi, I really think you want to distribute your force among a number of grid points, rather than concentrate everything on just the nearest point. On 2018-09-19 12:57, nek5000-users at lists.mcs.anl.gov wrote: > Dear Philipp and Paul, > > Thanks for your replies. Yes, what I am trying to do is to change the > force on every grid point. Now I have all the things without finding the > closest grid point. So I am thinking to use rst and element numbers > obtained from findpts to locate the closest grid point. If I got which > grid point it is (specifically?on having element number and ix iy iz in > that element), that would be sorted. > > May I know further to the GLL points. As far as I can see, the > zgm1(ix,1) ranges from -1 (ix=1) to 1 (ix=nx1). Also zgm1(iy,2) and > zgm1(iz,3) are for y and z, respectively. > (1). Are they corresponding to rst? If so, do you mean I can find the > closest zgm1 and determine the ix iy iz. > (2). How can I know which element zgm1 refers to? > > Kind regards, > > Jian > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Wed Sep 19 14:06:53 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 19 Sep 2018 20:06:53 +0100 Subject: [Nek5000-users] rst to ix iy iz Message-ID: Dear Philipp, Thanks for your your suggestion to correct my model which I really appreciate. I want to explain what I am doing and could you please give me some advice. Thanks so much. Now I am doing a Lagrangian particle tracking model with two-way coupling. So the interaction between two phases in my mind, should be added to the Eulerian grid point where the particle is, and the grid point where it belongs to is what I was finding out. I am sorry to say that I don't totally understand the the regularising the force among a number of grid points. I read some IBM papers before. If I were correct, I think the regularised function is employed with interpolating velocity and regularising force among adjacent grid points to keep the interface smooth. Do you think for my piont-particle tracking the regularising is needed? Would you please give me any suggestions? Kind regards, Jian -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 19 21:52:56 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Thu, 20 Sep 2018 04:52:56 +0200 Subject: [Nek5000-users] rst to ix iy iz In-Reply-To: References: Message-ID: Hi, yes, I think that for Lagrangian particle tracking such a regularisation is absolutely needed such that the force field is smooth, and peaks in the actual location of the particle. Depending on the function one uses to distribute the force, one does not need to calculate the indices. There are a couple of references in the literature that do such an approach, see e.g. Zhao, Andersson and Gillissen, 2010. Philipp On 2018-09-19 21:06, nek5000-users at lists.mcs.anl.gov wrote: > Dear Philipp, > > Thanks for your your suggestion to correct my model which I really > appreciate. I want to explain what I am doing and could you please give > me some advice. Thanks so much. > > Now I am doing a Lagrangian particle tracking model with two-way > coupling. So the interaction between two phases in my mind, should be > added to the Eulerian grid point where the particle is, and the grid > point where it belongs to is what I was finding out. I am sorry to say > that I don't totally understand the the regularising the force among a > number of grid points. I read some IBM papers before. If I were correct, > I think the regularised function is employed with interpolating velocity > and regularising force among adjacent grid points to keep the interface > smooth. Do you think for my piont-particle tracking the regularising is > needed? Would you please give me any suggestions? > > Kind regards, > > Jian > > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Fri Sep 21 04:53:48 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 21 Sep 2018 10:53:48 +0100 Subject: [Nek5000-users] interpolation of solution on extra points Message-ID: Hello all, Is it possible to write out the solution from Nek5000 with more points than actually used for calculation? Say, if I ran the simulation with poly. order 7 (i.e. 8 points inside each element per direction), can I write out the solution with 16 sampling points (i.e. solution interpolated on 16 points inside each element per direction)? Thanks for your time. BR Vishal -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Fri Sep 21 05:50:53 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 21 Sep 2018 12:50:53 +0200 Subject: [Nek5000-users] interpolation of solution on extra points In-Reply-To: References: Message-ID: Probably the easiest is to read in and directly write out with nek compiled with higher nx1. Philipp On September 21, 2018 11:53:48 AM GMT+02:00, nek5000-users at lists.mcs.anl.gov wrote: >Hello all, > >Is it possible to write out the solution from Nek5000 with more points >than >actually used for calculation? >Say, if I ran the simulation with poly. order 7 (i.e. 8 points inside >each >element per direction), can I write out the solution with 16 sampling >points (i.e. solution interpolated on 16 points inside each element per >direction)? > >Thanks for your time. > >BR >Vishal -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sat Sep 22 06:44:09 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 22 Sep 2018 12:44:09 +0100 Subject: [Nek5000-users] invalid handle in findpts Message-ID: Hi nek, When findpts was called, the error happened when interpolated points are 90000, as shown below. But it is fine for interpolation of 10 points as tested. I tried different memory located as maximum as 768GB of memory which is still not working. Can anyone give some suggestion? /nek5_svn/trunk/nek/jl/findpts.c:319): findpts_eval: invalid handle -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Sep 24 11:52:23 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 24 Sep 2018 18:52:23 +0200 Subject: [Nek5000-users] invalid handle in findpts In-Reply-To: References: Message-ID: Hard to say what's going wrong. Please provide more details (logfile, code, ...). -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Saturday 22nd September 2018 13:44 > To: nek5000-users at lists.mcs.anl.gov > Subject: [Nek5000-users] invalid handle in findpts > > Hi nek, > > When findpts was called, the error happened when interpolated points are 90000, as shown below. But it is fine for interpolation of 10 points as tested. I tried different memory located as maximum as 768GB of memory which is still not working. Can anyone give some suggestion? > > /nek5_svn/trunk/nek/jl/findpts.c:319): findpts_eval: invalid handle > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Wed Sep 26 04:26:11 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 12:26:11 +0300 Subject: [Nek5000-users] =?utf-8?q?Question_about_tolerance?= Message-ID: Hi, Neks! I'm wonder why in .par file, for example in the turbChannel case, there are different tolerances for pressure and for velocity? I mean that velocity seems to be dependent on pressure field, and if it is 10^-4 tolerance for the pressure, it will automatically the same for the velocity? Best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 26 05:23:00 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 12:23:00 +0200 Subject: [Nek5000-users] Question about tolerance In-Reply-To: References: Message-ID: There are two solvers -> two tolerances. -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Wednesday 26th September 2018 11:27 > To: nek5000-users > Subject: [Nek5000-users] Question about tolerance > > Hi, Neks! > > I'm wonder why in .par file, for example in the turbChannel case, there are different tolerances for pressure and for velocity? I mean that velocity seems to be dependent on pressure field, and if it is 10^-4 tolerance for the pressure, it will automatically the same for the velocity? > > Best regards, > Vlad > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Wed Sep 26 11:05:42 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 16:05:42 +0000 Subject: [Nek5000-users] info Message-ID: Hi, I am having difficulties with installing the software on a Mac. I would like to know if somebody can provide some help please. I have read the documents already. Thank you. Federico From nek5000-users at lists.mcs.anl.gov Wed Sep 26 11:40:57 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 16:40:57 +0000 Subject: [Nek5000-users] info In-Reply-To: References: Message-ID: Have you installed either openmpi or mpich? Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Wednesday, September 26, 2018 11:05:42 AM To: nek5000-users at lists.mcs.anl.gov Subject: [Nek5000-users] info Hi, I am having difficulties with installing the software on a Mac. I would like to know if somebody can provide some help please. I have read the documents already. Thank you. Federico _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 26 11:42:34 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 19:42:34 +0300 Subject: [Nek5000-users] =?utf-8?q?Question_about_tolerance?= In-Reply-To: References: Message-ID: No, the answer is in why they are different, not why there are two tolerances. Because, it seems that tolerance 10^-4 for the pressure means the tolerance 10^-4 for the velocity, but in .par file it is 10^-6 for the velocity.? ?????, 26 ???????? 2018 ?., 17:24 +0700 ?? nek5000-users at lists.mcs.anl.gov : >There are two solvers -> two tolerances. > >-----Original message----- >> From:nek5000-users at lists.mcs.anl.gov < nek5000-users at lists.mcs.anl.gov > >> Sent: Wednesday 26th September 2018 11:27 >> To: nek5000-users < nek5000-users at lists.mcs.anl.gov > >> Subject: [Nek5000-users] Question about tolerance >> >> Hi, Neks! >> >> I'm wonder why in .par file, for example in the turbChannel case, there are different tolerances for pressure and for velocity? I mean that velocity seems to be dependent on pressure field, and if it is 10^-4 tolerance for the pressure, it will automatically the same for the velocity? >> >> Best regards, >> Vlad >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >_______________________________________________ >Nek5000-users mailing list >Nek5000-users at lists.mcs.anl.gov >https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 26 11:42:02 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 18:42:02 +0200 Subject: [Nek5000-users] info In-Reply-To: References: Message-ID: Can you please be more specific. How do the "difficulties" manifest itself? -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Wednesday 26th September 2018 18:39 > To: nek5000-users at lists.mcs.anl.gov > Subject: [Nek5000-users] info > > Hi, > I am having difficulties with installing the software on a Mac. > I would like to know if somebody can provide some help please. > I have read the documents already. > Thank you. > > Federico > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Wed Sep 26 14:31:36 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 19:31:36 +0000 Subject: [Nek5000-users] Question about tolerance In-Reply-To: References: , Message-ID: The units of pressure and velocity are not the same so there is no expectation that the tolerances would be the same. The velocity generally converges much more rapidly so there is no cost in making that tolerance tighter and doing so generally helps to keep things relatively smooth. Thus, the key is - what tolerance for the pressure. In non dimensional units based on convective timescales 1.e-6 is generally found to be adequate for PnPn-2; a bit looser is ok for PnPn. In some cases (e.g., if you are measuring very small lift values for flow past a cylinder, say) you might need tighter tolerances, but these work for most applications. For transition or for "exact" solution comparisons, you would also want tighter tolerances, e.g., 1.e-10/1.e-12 or even 1.e-12/1.e-14. You have to be careful to not get too close to machine precision because the condition numbers of the governing system matrices will prevent you from converging to tolerances that are too tight. hth, Paul ________________________________ From: Nek5000-users on behalf of nek5000-users at lists.mcs.anl.gov Sent: Wednesday, September 26, 2018 11:42:34 AM To: nek5000-users at lists.mcs.anl.gov Subject: Re: [Nek5000-users] Question about tolerance No, the answer is in why they are different, not why there are two tolerances. Because, it seems that tolerance 10^-4 for the pressure means the tolerance 10^-4 for the velocity, but in .par file it is 10^-6 for the velocity. ?????, 26 ???????? 2018 ?., 17:24 +0700 ?? nek5000-users at lists.mcs.anl.gov : There are two solvers -> two tolerances. -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > > Sent: Wednesday 26th September 2018 11:27 > To: nek5000-users > > Subject: [Nek5000-users] Question about tolerance > > Hi, Neks! > > I'm wonder why in .par file, for example in the turbChannel case, there are different tolerances for pressure and for velocity? I mean that velocity seems to be dependent on pressure field, and if it is 10^-4 tolerance for the pressure, it will automatically the same for the velocity? > > Best regards, > Vlad > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 26 15:24:29 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 20:24:29 +0000 Subject: [Nek5000-users] info In-Reply-To: References: , Message-ID: No I haven't installed the software below but I will do. I have tried to install it using the terminal but unsuccessfully, I do not have a great knowledge of it, I have tried with Homebrew.On-line I found a sequence of instructions about installation on ubuntu. Something like that would be really useful. Thank you for your replies. Kind Regards Federico > On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov" wrote: > > Can you please be more specific. How do the "difficulties" manifest itself? > > -----Original message----- >> From:nek5000-users at lists.mcs.anl.gov >> Sent: Wednesday 26th September 2018 18:39 >> To: nek5000-users at lists.mcs.anl.gov >> Subject: [Nek5000-users] info >> >> Hi, >> I am having difficulties with installing the software on a Mac. >> I would like to know if somebody can provide some help please. >> I have read the documents already. >> Thank you. >> >> Federico >> >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Wed Sep 26 15:33:46 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 22:33:46 +0200 Subject: [Nek5000-users] info Message-ID: To install the GNU C/Fortran compilers + MPI, just run brew install gcc brew install gfortran brew install open-mpi To compile and run Nek5000 just follow the instructions outlined in: http://nek5000.github.io/NekDoc/quickstart.html#running-your-very-first-simulation -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Wednesday 26th September 2018 22:26 > To: nek5000-users at lists.mcs.anl.gov > Subject: Re: [Nek5000-users] info > > No I haven't installed the software below but I will do. I have tried to install it using the terminal but unsuccessfully, I do not have a great knowledge of it, I have tried with Homebrew.On-line I found a sequence of instructions about installation on ubuntu. Something like that would be really useful. > Thank you for your replies. > Kind Regards > > Federico > > > > > On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov" wrote: > > > > Can you please be more specific. How do the "difficulties" manifest itself? > > > > -----Original message----- > >> From:nek5000-users at lists.mcs.anl.gov > >> Sent: Wednesday 26th September 2018 18:39 > >> To: nek5000-users at lists.mcs.anl.gov > >> Subject: [Nek5000-users] info > >> > >> Hi, > >> I am having difficulties with installing the software on a Mac. > >> I would like to know if somebody can provide some help please. > >> I have read the documents already. > >> Thank you. > >> > >> Federico > >> > >> _______________________________________________ > >> Nek5000-users mailing list > >> Nek5000-users at lists.mcs.anl.gov > >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > _______________________________________________ > > Nek5000-users mailing list > > Nek5000-users at lists.mcs.anl.gov > > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Wed Sep 26 16:57:08 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 21:57:08 +0000 Subject: [Nek5000-users] info In-Reply-To: References: Message-ID: I had gcc and gfortran installed already. I have installed open-mpi It still does not work. I have followed the instructions, however I had to change the second line dragging the tar file from the desktop to the terminal otherwise I would have got the following error: tar: Error opening archive: Failed to open ?Nek5000_X.Y.tar.gz' Is the software supposed to open after the line run? Kind regards Federico On 26 Sep 2018, at 21:33, nek5000-users at lists.mcs.anl.gov wrote: To install the GNU C/Fortran compilers + MPI, just run brew install gcc brew install gfortran brew install open-mpi To compile and run Nek5000 just follow the instructions outlined in: http://nek5000.github.io/NekDoc/quickstart.html#running-your-very-first-simulation -----Original message----- From:nek5000-users at lists.mcs.anl.gov Sent: Wednesday 26th September 2018 22:26 To: nek5000-users at lists.mcs.anl.gov Subject: Re: [Nek5000-users] info No I haven't installed the software below but I will do. I have tried to install it using the terminal but unsuccessfully, I do not have a great knowledge of it, I have tried with Homebrew.On-line I found a sequence of instructions about installation on ubuntu. Something like that would be really useful. Thank you for your replies. Kind Regards Federico On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov" wrote: Can you please be more specific. How do the "difficulties" manifest itself? -----Original message----- From:nek5000-users at lists.mcs.anl.gov Sent: Wednesday 26th September 2018 18:39 To: nek5000-users at lists.mcs.anl.gov Subject: [Nek5000-users] info Hi, I am having difficulties with installing the software on a Mac. I would like to know if somebody can provide some help please. I have read the documents already. Thank you. Federico _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 26 17:01:15 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Thu, 27 Sep 2018 00:01:15 +0200 Subject: [Nek5000-users] info In-Reply-To: References: Message-ID: First you download the code e.g. using wget https://github.com/Nek5000/Nek5000/releases/download/v17.0/Nek5000-v17.0.tar.gz Then untar the file tar -zxvf Nek5000-v17.0.tar.gz -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Wednesday 26th September 2018 23:56 > To: nek5000-users at lists.mcs.anl.gov > Subject: Re: [Nek5000-users] info > > I had gcc and gfortran installed already. >
> I have installed open-mpi >
> It still does not work. >
> I have followed the instructions, however I had to change the second line dragging the tar file from the desktop to the terminal otherwise I would have got the following error: > tar: Error opening archive: Failed to open ?Nek5000_X.Y.tar.gz' >
> Is the software supposed to open after the line run? >
> Kind regards? >
> Federico > On 26 Sep 2018, at 21:33, > nek5000-users at lists.mcs.anl.gov wrote: >
> To install the GNU C/Fortran compilers MPI, just run
>
> brew install gcc
> brew install gfortran
> brew install open-mpi
>
> To compile and run Nek5000 just follow the instructions outlined in:
> http://nek5000.github.io/NekDoc/quickstart.html#running-your-very-first-simulation
>
> -----Original message-----
> From:nek5000-users at lists.mcs.anl.gov
> Sent: Wednesday 26th September 2018 22:26
> To: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] info
>
> No I haven't installed the software below but I will do. I have tried to install it using the terminal but unsuccessfully, I do not have a great knowledge of it, I have tried with Homebrew.On-line I found a sequence of instructions about installation on ubuntu. > Something like that would be really useful.
> Thank you for your replies.
> Kind Regards
>
> Federico
>
>
>
> On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov" wrote:
>
> Can you please be more specific. How do the "difficulties" manifest itself?
>
> -----Original message-----
> From:nek5000-users at lists.mcs.anl.gov
> Sent: Wednesday 26th September 2018 18:39
> To: nek5000-users at lists.mcs.anl.gov
> Subject: [Nek5000-users] info
>
> Hi,
> I am having difficulties with installing the software on a Mac.
> I would like to know if somebody can provide some help please.
> I have read the documents already.
> Thank you.
>
> Federico
>
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>
> _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Wed Sep 26 17:16:00 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 22:16:00 +0000 Subject: [Nek5000-users] info In-Reply-To: References: Message-ID: I have used your version of the code and the on-line version and I get the same error. Last login: Wed Sep 26 23:07:14 on ttys000 MacBook-di-Federico-Antonelli:~ maitai2005$ cd ~ MacBook-di-Federico-Antonelli:~ maitai2005$ tar -xvzf Nek5000_X.Y.tar.gz tar: Error opening archive: Failed to open 'Nek5000_X.Y.tar.gz' MacBook-di-Federico-Antonelli:~ maitai2005$ tar -zxvf Nek5000-v17.0.tar.gz tar: Error opening archive: Failed to open 'Nek5000-v17.0.tar.gz' MacBook-di-Federico-Antonelli:~ maitai2005$ On 26 Sep 2018, at 23:01, nek5000-users at lists.mcs.anl.gov wrote: First you download the code e.g. using wget https://github.com/Nek5000/Nek5000/releases/download/v17.0/Nek5000-v17.0.tar.gz Then untar the file tar -zxvf Nek5000-v17.0.tar.gz -----Original message----- From:nek5000-users at lists.mcs.anl.gov > Sent: Wednesday 26th September 2018 23:56 To: nek5000-users at lists.mcs.anl.gov Subject: Re: [Nek5000-users] info I had gcc and gfortran installed already.
I have installed open-mpi
It still does not work.
I have followed the instructions, however I had to change the second line dragging the tar file from the desktop to the terminal otherwise I would have got the following error: tar: Error opening archive: Failed to open ?Nek5000_X.Y.tar.gz'
Is the software supposed to open after the line run?
Kind regards
Federico On 26 Sep 2018, at 21:33, nek5000-users at lists.mcs.anl.gov wrote:
To install the GNU C/Fortran compilers MPI, just run

brew install gcc
brew install gfortran
brew install open-mpi

To compile and run Nek5000 just follow the instructions outlined in:
http://nek5000.github.io/NekDoc/quickstart.html#running-your-very-first-simulation

-----Original message-----
From:nek5000-users at lists.mcs.anl.gov >
Sent: Wednesday 26th September 2018 22:26
To: nek5000-users at lists.mcs.anl.gov
Subject: Re: [Nek5000-users] info

No I haven't installed the software below but I will do. I have tried to install it using the terminal but unsuccessfully, I do not have a great knowledge of it, I have tried with Homebrew.On-line I found a sequence of instructions about installation on ubuntu. Something like that would be really useful.
Thank you for your replies.
Kind Regards

Federico



On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov" > wrote:

Can you please be more specific. How do the "difficulties" manifest itself?

-----Original message-----
From:nek5000-users at lists.mcs.anl.gov >
Sent: Wednesday 26th September 2018 18:39
To: nek5000-users at lists.mcs.anl.gov
Subject: [Nek5000-users] info

Hi,
I am having difficulties with installing the software on a Mac.
I would like to know if somebody can provide some help please.
I have read the documents already.
Thank you.

Federico

_______________________________________________
Nek5000-users mailing list
Nek5000-users at lists.mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
_______________________________________________
Nek5000-users mailing list
Nek5000-users at lists.mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
_______________________________________________
Nek5000-users mailing list
Nek5000-users at lists.mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

_______________________________________________
Nek5000-users mailing list
Nek5000-users at lists.mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

_______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Wed Sep 26 17:19:22 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Thu, 27 Sep 2018 00:19:22 +0200 Subject: [Nek5000-users] info In-Reply-To: References: Message-ID: did you download the file? On 2018-09-27 00:16, nek5000-users at lists.mcs.anl.gov wrote: > I have used your version of the code and the on-line version and I get > the same error. > >> Last login: Wed Sep 26 23:07:14 on ttys000 >> MacBook-di-Federico-Antonelli:~ maitai2005$ cd ~ >> MacBook-di-Federico-Antonelli:~ maitai2005$ tar -xvzf Nek5000_X.Y.tar.gz >> tar: Error opening archive: Failed to open 'Nek5000_X.Y.tar.gz' >> MacBook-di-Federico-Antonelli:~ maitai2005$ tar -zxvf Nek5000-v17.0.tar.gz >> tar: Error opening archive: Failed to open 'Nek5000-v17.0.tar.gz' >> MacBook-di-Federico-Antonelli:~ maitai2005$ >> On 26 Sep 2018, at 23:01, nek5000-users at lists.mcs.anl.gov >> wrote: >> >> First you download the code e.g. using >> wget >> https://github.com/Nek5000/Nek5000/releases/download/v17.0/Nek5000-v17.0.tar.gz >> >> Then untar the file >> tar -zxvf Nek5000-v17.0.tar.gz >> >> >> -----Original message----- >>> From:nek5000-users at lists.mcs.anl.gov >>> >>> >> > >>> Sent: Wednesday 26th September 2018 23:56 >>> To: nek5000-users at lists.mcs.anl.gov >>> >>> Subject: Re: [Nek5000-users] info >>> >>> I had gcc and gfortran installed already. >>>
>>> I have installed open-mpi >>>
>>> It still does not work. >>>
>>> I have followed the instructions, however I had to change the second >>> line dragging the tar file from the desktop to the terminal otherwise >>> I would have got the following error: >>> tar: Error opening archive: Failed to open ?Nek5000_X.Y.tar.gz' >>>
>>> Is the software supposed to open after the line run? >>>
>>> Kind regards >>>
>>> Federico >>> On 26 Sep 2018, at 21:33, >>> nek5000-users at lists.mcs.anl.gov >>> >>> wrote: >>>
>>> To install the GNU C/Fortran compilers ?MPI, just run
>>>
>>> brew install gcc
>>> brew install gfortran
>>> brew install open-mpi
>>>
>>> To compile and run Nek5000 just follow the instructions outlined >>> in:
>>> http://nek5000.github.io/NekDoc/quickstart.html#running-your-very-first-simulation >>>
>> class="" /> >>>
>>> -----Original message-----
>>> From:nek5000-users at lists.mcs.anl.gov >>> >>> >> >
>>> Sent: Wednesday 26th September 2018 22:26
>>> To: nek5000-users at lists.mcs.anl.gov >>>
>>> Subject: Re: [Nek5000-users] info
>>>
>>> No I haven't installed the software below but I will do. I have tried >>> to install it using the terminal but unsuccessfully, I do not have a >>> great knowledge of it, I have tried with Homebrew.On-line I found a >>> sequence of instructions about installation on ubuntu. >>> Something like that would be really useful.
>>> Thank you for your replies.
>>> Kind Regards
>>>
>>> Federico
>>>
>>>
>>>
>>> On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov >>> " >>> >> > wrote:
>>>
>>> Can you please be more specific. How do the "difficulties" manifest >>> itself?
>>>
>>> -----Original message-----
>>> From:nek5000-users at lists.mcs.anl.gov >>> >>> >> >
>>> Sent: Wednesday 26th September 2018 18:39
>>> To: nek5000-users at lists.mcs.anl.gov >>>
>>> Subject: [Nek5000-users] info
>>>
>>> Hi,
>>> I am having difficulties with installing the software on a Mac.
>> class="" /> >>> I would like to know if somebody can provide some help please.
>> class="" /> >>> I have read the documents already.
>>> Thank you.
>>>
>>> Federico
>>>
>>> _______________________________________________
>>> Nek5000-users mailing list
>>> Nek5000-users at lists.mcs.anl.gov >>>
>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>> _______________________________________________
>>> Nek5000-users mailing list
>>> Nek5000-users at lists.mcs.anl.gov >>>
>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>> _______________________________________________
>>> Nek5000-users mailing list
>>> Nek5000-users at lists.mcs.anl.gov >>>
>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>>
>>> _______________________________________________
>>> Nek5000-users mailing list
>>> Nek5000-users at lists.mcs.anl.gov >>>
>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>>
>>> _______________________________________________ >>> Nek5000-users mailing list >>> Nek5000-users at lists.mcs.anl.gov >>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Wed Sep 26 17:19:51 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Thu, 27 Sep 2018 00:19:51 +0200 Subject: [Nek5000-users] info In-Reply-To: References: Message-ID: We try hard to help beginners to get started but it seems like you have to learn some shell basics first. Did you check if the file exists? -----Original message----- > From:nek5000-users at lists.mcs.anl.gov > Sent: Thursday 27th September 2018 0:15 > To: nek5000-users at lists.mcs.anl.gov > Subject: Re: [Nek5000-users] info > > I have used your version of the code and the on-line version and I get the same error. >
> Last login: Wed Sep 26 23:07:14 on ttys000 > MacBook-di-Federico-Antonelli:~ maitai2005$ cd ~ > MacBook-di-Federico-Antonelli:~ maitai2005$ tar -xvzf Nek5000_X.Y.tar.gz > tar: Error opening archive: Failed to open 'Nek5000_X.Y.tar.gz' > MacBook-di-Federico-Antonelli:~ maitai2005$ tar -zxvf Nek5000-v17.0.tar.gz > tar: Error opening archive: Failed to open 'Nek5000-v17.0.tar.gz' > MacBook-di-Federico-Antonelli:~ maitai2005$? > On 26 Sep 2018, at 23:01, > nek5000-users at lists.mcs.anl.gov wrote: >
> First you download the code e.g. using
> wget > https://github.com/Nek5000/Nek5000/releases/download/v17.0/Nek5000-v17.0.tar.gz
>
> Then untar the file
> tar -zxvf Nek5000-v17.0.tar.gz
>
>
> -----Original message-----
> From:nek5000-users at lists.mcs.anl.gov >
> Sent: Wednesday 26th September 2018 23:56
> To: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] info
>
> I had gcc and gfortran installed already.
>

> I have installed open-mpi
>

> It still does not work.
>

> I have followed the instructions, however I had to change the second line dragging the tar file from the desktop to the terminal otherwise I would have got the following error:
> tar: Error opening archive: Failed to open ?Nek5000_X.Y.tar.gz'
>

> Is the software supposed to open after the line run?
>

> Kind regards?
>

> Federico
> On 26 Sep 2018, at 21:33,
> nek5000-users at lists.mcs.anl.gov > wrote:
>

> To install the GNU C/Fortran compilers ?MPI, just run

>

> brew install gcc

> brew install gfortran

> brew install open-mpi

>

> To compile and run Nek5000 just follow the instructions outlined in:

> http://nek5000.github.io/NekDoc/quickstart.html#running-your-very-first-simulation >
class="" />
>

> -----Original message-----

> From:nek5000-users at lists.mcs.anl.gov >

> Sent: Wednesday 26th September 2018 22:26

> To: nek5000-users at lists.mcs.anl.gov

> Subject: Re: [Nek5000-users] info

>

> No I haven't installed the software below but I will do. I have tried to install it using the terminal but unsuccessfully, I do not have a great knowledge of it, I have tried with Homebrew.On-line I found a sequence of instructions about installation on ubuntu.
> Something like that would be really useful.

> Thank you for your replies.

> Kind Regards

>

> Federico

>

>

>

> On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov " > wrote:

>

> Can you please be more specific. How do the "difficulties" manifest itself?

>

> -----Original message-----

> From:nek5000-users at lists.mcs.anl.gov >

> Sent: Wednesday 26th September 2018 18:39

> To: nek5000-users at lists.mcs.anl.gov

> Subject: [Nek5000-users] info

>

> Hi,

> I am having difficulties with installing the software on a Mac.

> I would like to know if somebody can provide some help please.

> I have read the documents already.

> Thank you.

>

> Federico

>

> _______________________________________________

> Nek5000-users mailing list

> Nek5000-users at lists.mcs.anl.gov

> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

> _______________________________________________

> Nek5000-users mailing list

> Nek5000-users at lists.mcs.anl.gov

> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

> _______________________________________________

> Nek5000-users mailing list

> Nek5000-users at lists.mcs.anl.gov

> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

>

> _______________________________________________

> Nek5000-users mailing list

> Nek5000-users at lists.mcs.anl.gov

> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

>

> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>
> _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Wed Sep 26 17:45:03 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 26 Sep 2018 22:45:03 +0000 Subject: [Nek5000-users] info In-Reply-To: References: , Message-ID: I have to learn what shells are first, then I can start with the basics. :-) We, humans, generally install programs using pkg files. If you wish to tell me where to look for info about shells It would be really nice. Thank you very much for your help and goodnight from UK, from Leeds to be precise. Kind Regards Federico > On 26 Sep 2018, at 23:21, "nek5000-users at lists.mcs.anl.gov" wrote: > > We try hard to help beginners to get started but it seems like you have to learn some shell basics first. > Did you check if the file exists? > > -----Original message----- >> From:nek5000-users at lists.mcs.anl.gov >> Sent: Thursday 27th September 2018 0:15 >> To: nek5000-users at lists.mcs.anl.gov >> Subject: Re: [Nek5000-users] info >> >> I have used your version of the code and the on-line version and I get the same error. >>
>> Last login: Wed Sep 26 23:07:14 on ttys000 >> MacBook-di-Federico-Antonelli:~ maitai2005$ cd ~ >> MacBook-di-Federico-Antonelli:~ maitai2005$ tar -xvzf Nek5000_X.Y.tar.gz >> tar: Error opening archive: Failed to open 'Nek5000_X.Y.tar.gz' >> MacBook-di-Federico-Antonelli:~ maitai2005$ tar -zxvf Nek5000-v17.0.tar.gz >> tar: Error opening archive: Failed to open 'Nek5000-v17.0.tar.gz' >> MacBook-di-Federico-Antonelli:~ maitai2005$ >> On 26 Sep 2018, at 23:01, >> nek5000-users at lists.mcs.anl.gov wrote: >>
>> First you download the code e.g. using
>> wget >> https://github.com/Nek5000/Nek5000/releases/download/v17.0/Nek5000-v17.0.tar.gz
>>
>> Then untar the file
>> tar -zxvf Nek5000-v17.0.tar.gz
>>
>>
>> -----Original message-----
>> From:nek5000-users at lists.mcs.anl.gov >
>> Sent: Wednesday 26th September 2018 23:56
>> To: nek5000-users at lists.mcs.anl.gov
>> Subject: Re: [Nek5000-users] info
>>
>> I had gcc and gfortran installed already.
>>

>> I have installed open-mpi
>>

>> It still does not work.
>>

>> I have followed the instructions, however I had to change the second line dragging the tar file from the desktop to the terminal otherwise I would have got the following error:
>> tar: Error opening archive: Failed to open ?Nek5000_X.Y.tar.gz'
>>

>> Is the software supposed to open after the line run?
>>

>> Kind regards
>>

>> Federico
>> On 26 Sep 2018, at 21:33,
>> nek5000-users at lists.mcs.anl.gov > wrote:
>>

>> To install the GNU C/Fortran compilers MPI, just run

>>

>> brew install gcc

>> brew install gfortran

>> brew install open-mpi

>>

>> To compile and run Nek5000 just follow the instructions outlined in:

>> http://nek5000.github.io/NekDoc/quickstart.html#running-your-very-first-simulation >
> class="" />
>>

>> -----Original message-----

>> From:nek5000-users at lists.mcs.anl.gov >

>> Sent: Wednesday 26th September 2018 22:26

>> To: nek5000-users at lists.mcs.anl.gov

>> Subject: Re: [Nek5000-users] info

>>

>> No I haven't installed the software below but I will do. I have tried to install it using the terminal but unsuccessfully, I do not have a great knowledge of it, I have tried with Homebrew.On-line I found a sequence of instructions about installation on ubuntu.
>> Something like that would be really useful.

>> Thank you for your replies.

>> Kind Regards

>>

>> Federico

>>

>>

>>

>> On 26 Sep 2018, at 17:44, "nek5000-users at lists.mcs.anl.gov " > wrote:

>>

>> Can you please be more specific. How do the "difficulties" manifest itself?

>>

>> -----Original message-----

>> From:nek5000-users at lists.mcs.anl.gov >

>> Sent: Wednesday 26th September 2018 18:39

>> To: nek5000-users at lists.mcs.anl.gov

>> Subject: [Nek5000-users] info

>>

>> Hi,

>> I am having difficulties with installing the software on a Mac.

>> I would like to know if somebody can provide some help please.

>> I have read the documents already.

>> Thank you.

>>

>> Federico

>>

>> _______________________________________________

>> Nek5000-users mailing list

>> Nek5000-users at lists.mcs.anl.gov

>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

>> _______________________________________________

>> Nek5000-users mailing list

>> Nek5000-users at lists.mcs.anl.gov

>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

>> _______________________________________________

>> Nek5000-users mailing list

>> Nek5000-users at lists.mcs.anl.gov

>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

>>

>> _______________________________________________

>> Nek5000-users mailing list

>> Nek5000-users at lists.mcs.anl.gov

>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

>>

>> _______________________________________________
>> Nek5000-users mailing list
>> Nek5000-users at lists.mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>> _______________________________________________
>> Nek5000-users mailing list
>> Nek5000-users at lists.mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>
>> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Thu Sep 27 08:07:31 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Thu, 27 Sep 2018 18:37:31 +0530 Subject: [Nek5000-users] Merging two meshes. Message-ID: Hello Everyone, I am a brand new user to NEK5000. I want to merge a circular mesh with a box mesh. Can someone guide me how to do this. Regards, Rohan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Fri Sep 28 08:08:45 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 28 Sep 2018 14:08:45 +0100 Subject: [Nek5000-users] Nekbone correctness Message-ID: Dear Neks, I was wondering if someone could help us to identify correctness tests for Nekbone. How can we make sure that after modifying the code we still get a valid output? Is there anything we can use from the printed output, for instance |/cg: ||0| |3||.9174E+||02| /|/comm time ||0||.2370E-||01| /|/cg: ||101| |2||.9280E-||05| |4||.3327E-||01| |7||.6135E-||01| /|/2||.5748E-||09| /|/cg: ||0| |4||.0900E+||02| /|/comm time ||0||.2232E-||01| /|/cg: ||101| |1||.0800E-||05| |4||.4507E-||01| |7||.4233E-||01| /|/3||.4828E-||10| /|/| /|/nelt = ||64||, np = ||8||, nx1 = ||10||, elements /= ||512| |/Tot MFlops = ||4||.0983E+||04||, MFlops = ||5||.1228E+||03| /|/Setup Flop = ||8||.8960E+||08||, Solver Flop = ||9||.6960E+||07| /|/Solve Time = ||0||.1926E+||00| /|/Avg MFlops = ||4||.0983E+||04| / Thanks for your help! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: not available URL: From nek5000-users at lists.mcs.anl.gov Sun Sep 30 06:32:19 2018 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sun, 30 Sep 2018 17:02:19 +0530 Subject: [Nek5000-users] Problems in getting eigenvalues using ARNOLDI Message-ID: Hi Nek, I am running ARNOLDI simulation case for a flow over a 2D cylinder using ARPACK . My domain is [-50,50] and Re=50. I have calculated base flow using SFD method. While calculating Eigen values I have to run the restart around 16-20 times to get an eigenvalues.txt file as an output. Also the eigenvalues that are coming most of them has zero rotation i.e., they are negative real value. Suppose I want to have 30 eigenvalues as an output and I am setting Krylov subspace as 90 then I am getting around 20 of eigenvalues having zero rotation. Kindly help as I am stuck in it for long. Thanks in advance. Regards, Alok Mishra Aerospace Department IIT Kanpur -------------- next part -------------- An HTML attachment was scrubbed... URL: