[Nek5000-users] run-time hang up in gs_setup
nek5000-users at lists.mcs.anl.gov
nek5000-users at lists.mcs.anl.gov
Wed Jan 11 11:40:10 CST 2012
Dear Stefan and Aleks;
Thanks for updating the wiki webpage regarding the AMG, although, I
persume there must be another step also there exist, namely: copy the
generated files from the amg_matlb to the running directory? (Or they
should be remained there and one puts the generated .dat files after
running the 3rd step?). By the way non of the versions I tried working
(even 707!) despite the fact that I had a range of matlab versions
tried. Hanging with the old version I had I compiled again and have
got the four files which was rather fast (with the message at the end:
Error contraction factor: 0.47...) I used them and every time during
the run-time it crashed simply:
###########################################################################
AMG: reading through row 144540, pass 121/121
AMG: reading 0.071106 MB of W
AMG: reading 0.115601 MB of AfP
AMG: reading 0.132477 MB of Aff
AMG level 1 F-vars: 440159
AMG level 2 F-vars: 55146
AMG level 3 F-vars: 28480
AMG level 4 F-vars: 17051
AMG level 5 F-vars: 7524
AMG level 6 F-vars: 5711
AMG level 7 F-vars: 5763
AMG level 8 F-vars: 28583
AMG level 9 F-vars: 5380
AMG level 10 F-vars: 5737
Application 731033 exit codes: 139
Application 731033 exit signals: Killed
Application 731033 resources: utime ~417s, stime ~3s
##########################################################################
Can you help me with that cause I believe this case still doable with
correct AMG scheme.
Many thanks
Azad
>
> Hi Azad,
>
> I believe old AMG files should work up to and including revision 707
> in case >you want to check AMG quickly.
>
> Best.
> Aleks
>
>
> ----- Original Message -----
> From: nek5000-users at lists.mcs.anl.gov
> To: nek5000-users at lists.mcs.anl.gov
> Sent: Tuesday, January 10, 2012 10:58:47 AM
> Subject: Re: [Nek5000-users] run-time hang up in gs_setup
>
> Hi Azad,
>
> your choice of lx1=8 is fine (it's our preferred sweet spot). If you
> have a large element count (say > 300'000) the factorization in the
> XXt setup phase may take hours. I guess that's why it looks like it's
> hanging. Again, there is a known bug which looks the same. So can't
> tell exactly what's causing your problem.
>
> I just updated the Wiki: https://nek5000.mcs.anl.gov/index.php/Amg_matlab
>
> Can you verify that it still fails.
>
> -Stefan
>
> Quoting nek5000-users-request at lists.mcs.anl.gov:
>
> Send Nek5000-users mailing list submissions to
> nek5000-users at lists.mcs.anl.gov
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> or, via email, send a message with subject or body 'help' to
> nek5000-users-request at lists.mcs.anl.gov
>
> You can reach the person managing the list at
> nek5000-users-owner at lists.mcs.anl.gov
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Nek5000-users digest..."
>
>
> Today's Topics:
>
> 1. Re: run-time hang up in gs_setup (nek5000-users at lists.mcs.anl.gov)
> 2. Re: run-time hang up in gs_setup (nek5000-users at lists.mcs.anl.gov)
> 3. Re: run-time hang up in gs_setup (nek5000-users at lists.mcs.anl.gov)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 10 Jan 2012 06:01:29 -0600 (CST)
> From: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] run-time hang up in gs_setup
> To: nek5000-users at lists.mcs.anl.gov
> Message-ID: <Pine.LNX.4.64.1201100557250.6026 at v8.mcs.anl.gov>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
>
> Hi Azad,
>
> You are in record-setting territory for element counts! :)
>
> Are you using the amg-based coarse-grid solver?
> It is certain that you will need to do this (and,
> therefore, you will need matlab to process the AMG
> operators). There is some discussion of the steps
> on the wiki page. We can walk you through this process
> if you have any questions.
>
> What value of lx1 are you using?
>
> I would recommend fewer elements and a higher value of lx1.
> I think it will be easier to manage the data, etc.
>
> Paul
>
>
>
>
>
> On Tue, 10 Jan 2012, nek5000-users at lists.mcs.anl.gov wrote:
>
>> Dear NEKs;
>>
>> I am trying to run a simulation of a turbulent flow in a straight pipe
>> in high Reynolds number (Re_tau = 1000). After generating the grid with
>> PRENEK and extrude it using n2to3, the mesh ended up with 4,495,920
>> elements. It compiled properly; however, trying to run it, hanged up in
>> the last stage:
>> ########################################################################
>>
>> verify mesh topology
>> -1.000000000000000 1.000000000000000 Xrange
>> -1.000000000000000 1.000000000000000 Yrange
>> 0.000000000000000 25.00000000000000 Zrange
>> done :: verify mesh topology
>>
>> E-solver strategy: 1 itr
>> mg_nx: 1 3
>> mg_ny: 1 3
>> mg_nz: 1 3
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 866937 unique labels shared
>> pairwise times (avg, min, max): 0.000241442 0.00019722 0.000265908
>> crystal router : 0.000458177 0.000445795 0.000471807
>> used all_to_all method: pairwise
>> setupds time 5.6048E-02 seconds 1 2 4565612 4495920
>> setvert3d: 4 86046564 122013924 86046564 86046564
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 8041169 unique labels shared
>> pairwise times (avg, min, max): 0.00050716 0.000427103 0.00056479
>> crystal router : 0.0040165 0.00392921 0.00411811
>> used all_to_all method: pairwise
>> setupds time 1.0465E+00 seconds 2 4 86046564 4495920
>> setup h1 coarse grid, nx_crs= 2
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 866937 unique labels shared
>> pairwise times (avg, min, max): 0.000233683 0.000197816 0.00024941
>> crystal router : 0.000466869 0.00045588 0.000478101
>> used all_to_all method: pairwise
>> ########################################################################
>>
>>
>> I was wondering if you could help me with that. I attached the run
>> logfile and also genmap.out.
>>
>> Many thanks
>> Azad
>>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 10 Jan 2012 13:35:22 +0100
> From: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] run-time hang up in gs_setup
> To: nek5000-users at lists.mcs.anl.gov
> Message-ID:
> <CAGTrLsaexkteQN1Y1NQ3FYz7Q2abb5YSLzOv+zeTwdvYXpD3Fw at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi Azad,
>
> We have seen similar situations. I think this has to do with a known
> bug. Unfortunately this bug is hard to reproduce and we haven't
> managed to fix it yet.
>
> -Stefan
>
> On 1/10/12, nek5000-users at lists.mcs.anl.gov
> <nek5000-users at lists.mcs.anl.gov> wrote:
>>
>> Hi Azad,
>>
>> You are in record-setting territory for element counts! :)
>>
>> Are you using the amg-based coarse-grid solver?
>> It is certain that you will need to do this (and,
>> therefore, you will need matlab to process the AMG
>> operators). There is some discussion of the steps
>> on the wiki page. We can walk you through this process
>> if you have any questions.
>>
>> What value of lx1 are you using?
>>
>> I would recommend fewer elements and a higher value of lx1.
>> I think it will be easier to manage the data, etc.
>>
>> Paul
>>
>>
>>
>>
>>
>> On Tue, 10 Jan 2012, nek5000-users at lists.mcs.anl.gov wrote:
>>
>>> Dear NEKs;
>>>
>>> I am trying to run a simulation of a turbulent flow in a straight pipe
>>> in high Reynolds number (Re_tau = 1000). After generating the grid with
>>> PRENEK and extrude it using n2to3, the mesh ended up with 4,495,920
>>> elements. It compiled properly; however, trying to run it, hanged up in
>>> the last stage:
>>> ########################################################################
>>>
>>> verify mesh topology
>>> -1.000000000000000 1.000000000000000 Xrange
>>> -1.000000000000000 1.000000000000000 Yrange
>>> 0.000000000000000 25.00000000000000 Zrange
>>> done :: verify mesh topology
>>>
>>> E-solver strategy: 1 itr
>>> mg_nx: 1 3
>>> mg_ny: 1 3
>>> mg_nz: 1 3
>>> call usrsetvert
>>> done :: usrsetvert
>>>
>>> gs_setup: 866937 unique labels shared
>>> pairwise times (avg, min, max): 0.000241442 0.00019722 0.000265908
>>> crystal router : 0.000458177 0.000445795 0.000471807
>>> used all_to_all method: pairwise
>>> setupds time 5.6048E-02 seconds 1 2 4565612 4495920
>>> setvert3d: 4 86046564 122013924 86046564 86046564
>>> call usrsetvert
>>> done :: usrsetvert
>>>
>>> gs_setup: 8041169 unique labels shared
>>> pairwise times (avg, min, max): 0.00050716 0.000427103 0.00056479
>>> crystal router : 0.0040165 0.00392921 0.00411811
>>> used all_to_all method: pairwise
>>> setupds time 1.0465E+00 seconds 2 4 86046564 4495920
>>> setup h1 coarse grid, nx_crs= 2
>>> call usrsetvert
>>> done :: usrsetvert
>>>
>>> gs_setup: 866937 unique labels shared
>>> pairwise times (avg, min, max): 0.000233683 0.000197816 0.00024941
>>> crystal router : 0.000466869 0.00045588 0.000478101
>>> used all_to_all method: pairwise
>>> ########################################################################
>>>
>>>
>>> I was wondering if you could help me with that. I attached the run
>>> logfile and also genmap.out.
>>>
>>> Many thanks
>>> Azad
>>>
>> _______________________________________________
>> Nek5000-users mailing list
>> Nek5000-users at lists.mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>>
>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 10 Jan 2012 17:58:47 +0100
> From: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] run-time hang up in gs_setup
> To: nek5000-users at lists.mcs.anl.gov
> Message-ID: <1326214727.2600.282.camel at damavand.mech.kth.se>
> Content-Type: text/plain; charset="UTF-8"
>
> Dear Paul and Stefan;
>
> Thanks very much for looking into it. I use polynomial order 7th
> (lx1=8). For the coarse-grid solver I actually used XXt. I also tried to
> use AMG, but unfortunately neither v619 nor the latest version could
> have compiled its matlab files and always gives me this error (in
> matlab/R2011a):
> ##############################################
> ...
> sparsification tolerance [1e-4]: stol = 0.0001
>
> ------------------------------------------------------------------------
> Segmentation violation detected at Tue Jan 10 15:56:46 2012
> ------------------------------------------------------------------------
> ....
> Abnormal termination:
> Segmentation violation
> ....
> #############################################
> I have been in the web page: "amg_matlab Matlab based tool to generate
> AMG solver inputfiles" (http://nek5000.mcs.anl.gov/index.php/Amg_matlab)
> which gives me an empty link.
>
> I had an old version of the .dat files needed to run AMG, which I tried
> those as (amg_Aff.dat, amgdmp_i.dat, amg.dat, amg_AfP.dat, amgdmp_p.dat,
> amgdmp_j.dat, amg_W.dat) and I have got this error:
>
> ############################################
> ...
> AMG: reading through row 142800, pass 119/121
> AMG: reading through row 144000, pass 120/121
> AMG: reading through row 144540, pass 121/121
> ERROR (proc
> 0000,
> /afs/pdc.kth.se/home/a/anoorani/codes/latest_nek/nek5_svn/trunk/nek/jl/amg.c:468): AMG: missing data for some
> rows
>
> call exitt: dying ...
> ############################################
>
> I think AMG could be a possibility to overcome this problem, though I
> could not manage to get a run with that one. I look into the problem
> with higher polynomial order to see if it reduces the number of elements
> dramatically, or at least resolve this issue.
>
> Best regards
> Azad
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>> Hi Azad,
>>
>> We have seen similar situations. I think this has to do with a known
>> bug. Unfortunately this bug is hard to reproduce and we haven't
>> managed to fix it yet.
>>
>> -Stefan
>>
>> On 1/10/12, nek5000-users at lists.mcs.anl.gov
>> <nek5000-users at lists.mcs.anl.gov> wrote:
>>
>>
>> Hi Azad,
>>
>> You are in record-setting territory for element counts! :)
>>
>> Are you using the amg-based coarse-grid solver?
>> It is certain that you will need to do this (and,
>> therefore, you will need matlab to process the AMG
>> operators). There is some discussion of the steps
>> on the wiki page. We can walk you through this process
>> if you have any questions.
>>
>> What value of lx1 are you using?
>>
>> I would recommend fewer elements and a higher value of lx1.
>> I think it will be easier to manage the data, etc.
>>
>> Paul
>>
>>
>>
>>
>>
>> On Tue, 10 Jan 2012, nek5000-users at lists.mcs.anl.gov wrote:
>>
>> Dear NEKs;
>>
>> I am trying to run a simulation of a turbulent flow in a straight pipe
>> in high Reynolds number (Re_tau = 1000). After generating the grid with
>> PRENEK and extrude it using n2to3, the mesh ended up with 4,495,920
>> elements. It compiled properly; however, trying to run it, hanged up in
>> the last stage:
>> ########################################################################
>>
>> verify mesh topology
>> -1.000000000000000 1.000000000000000 Xrange
>> -1.000000000000000 1.000000000000000 Yrange
>> 0.000000000000000 25.00000000000000 Zrange
>> done :: verify mesh topology
>>
>> E-solver strategy: 1 itr
>> mg_nx: 1 3
>> mg_ny: 1 3
>> mg_nz: 1 3
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 866937 unique labels shared
>> pairwise times (avg, min, max): 0.000241442 0.00019722 0.000265908
>> crystal router : 0.000458177 0.000445795 0.000471807
>> used all_to_all method: pairwise
>> setupds time 5.6048E-02 seconds 1 2 4565612 4495920
>> setvert3d: 4 86046564 122013924 86046564 86046564
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 8041169 unique labels shared
>> pairwise times (avg, min, max): 0.00050716 0.000427103 0.00056479
>> crystal router : 0.0040165 0.00392921 0.00411811
>> used all_to_all method: pairwise
>> setupds time 1.0465E+00 seconds 2 4 86046564 4495920
>> setup h1 coarse grid, nx_crs= 2
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 866937 unique labels shared
>> pairwise times (avg, min, max): 0.000233683 0.000197816 0.00024941
>> crystal router : 0.000466869 0.00045588 0.000478101
>> used all_to_all method: pairwise
>> ########################################################################
>>
>>
>> I was wondering if you could help me with that. I attached the run
>> logfile and also genmap.out.
>>
>> Many thanks
>> Azad
>>
>
>
>
> ------------------------------
>
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>
>
> End of Nek5000-users Digest, Vol 35, Issue 5
> ********************************************
>
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
More information about the Nek5000-users
mailing list