[Nek5000-users] run-time hang up in gs_setup

Tue Jan 10 12:13:45 CST 2012

Hi Azad,

your choice of lx1=8 is fine (it's our preferred sweet spot). If you
have a large element count (say > 300'000) the factorization in the
XXt setup phase may take hours. I guess that's why it looks like it's
hanging. Again, there is a known bug which looks the same. So can't
tell exactly what's causing your problem.

I just updated the Wiki: https://nek5000.mcs.anl.gov/index.php/Amg_matlab

Can you verify that it still fails.

-Stefan

On 1/10/12, nek5000-users at lists.mcs.anl.gov
<nek5000-users at lists.mcs.anl.gov> wrote:
> Dear Paul and Stefan;
>
> Thanks very much for looking into it. I use polynomial order 7th
> (lx1=8). For the coarse-grid solver I actually used XXt. I also tried to
> use AMG, but unfortunately neither v619 nor the latest version could
> have compiled its matlab files and always gives me this error (in
> matlab/R2011a):
> ##############################################
> ...
> sparsification tolerance [1e-4]: stol = 0.0001
>
> ------------------------------------------------------------------------
>        Segmentation violation detected at Tue Jan 10 15:56:46 2012
> ------------------------------------------------------------------------
> ....
> Abnormal termination:
> Segmentation violation
> ....
> #############################################
> I have been in the web page: "amg_matlab Matlab based tool to generate
> AMG solver inputfiles" (http://nek5000.mcs.anl.gov/index.php/Amg_matlab)
> which gives me an empty link.
>
> I had an old version of the .dat files needed to run AMG, which I tried
> those as (amg_Aff.dat, amgdmp_i.dat, amg.dat, amg_AfP.dat, amgdmp_p.dat,
> amgdmp_j.dat, amg_W.dat) and I have got this error:
>
> ############################################
> ...
> AMG: reading through row 142800, pass 119/121
> AMG: reading through row 144000, pass 120/121
> AMG: reading through row 144540, pass 121/121
> ERROR (proc
> 0000,
> /afs/pdc.kth.se/home/a/anoorani/codes/latest_nek/nek5_svn/trunk/nek/jl/amg.c:468):
> AMG: missing data for some rows
>
> call exitt: dying ...
> ############################################
>
> I think AMG could be a possibility to overcome this problem, though I
> could not manage to get a run with that one. I look into the problem
> with higher polynomial order to see if it reduces the number of elements
> dramatically, or at least resolve this issue.
>
> Best regards
> Azad
>
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
>>Hi Azad,
>>
>>We have seen similar situations. I think this has to do with a known
>>bug. Unfortunately this bug is hard to reproduce and we haven't
>>managed to fix it yet.
>>
>>-Stefan
>>
>>On 1/10/12, nek5000-users at lists.mcs.anl.gov
>><nek5000-users at lists.mcs.anl.gov> wrote:
>>
>>
>>Hi Azad,
>>
>>You are in record-setting territory for element counts! :)
>>
>>Are you using the amg-based coarse-grid solver?
>>It is certain that you will need to do this (and,
>>therefore, you will need matlab to process the AMG
>>operators).   There is some discussion of the steps
>>on the wiki page.  We can walk you through this process
>>if you have any questions.
>>
>>What value of lx1 are you using?
>>
>>I would recommend fewer elements and a higher value of lx1.
>>I think it will be easier to manage the data, etc.
>>
>>Paul
>>
>>
>>
>>
>>
>>On Tue, 10 Jan 2012, nek5000-users at lists.mcs.anl.gov wrote:
>>
>> Dear NEKs;
>>
>> I am trying to run a simulation of a turbulent flow in a straight pipe
>> in high Reynolds number (Re_tau = 1000). After generating the grid with
>> PRENEK and extrude it using n2to3, the mesh ended up with 4,495,920
>> elements. It compiled properly; however, trying to run it, hanged up in
>> the last stage:
>> ########################################################################
>>
>> verify mesh topology
>>   -1.000000000000000         1.000000000000000       Xrange
>>   -1.000000000000000         1.000000000000000       Yrange
>>    0.000000000000000         25.00000000000000       Zrange
>> done :: verify mesh topology
>>
>>  E-solver strategy:  1 itr
>> mg_nx:            1            3
>> mg_ny:            1            3
>> mg_nz:            1            3
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 866937 unique labels shared
>>   pairwise times (avg, min, max): 0.000241442 0.00019722 0.000265908
>>   crystal router                : 0.000458177 0.000445795 0.000471807
>>   used all_to_all method: pairwise
>>   setupds time 5.6048E-02 seconds   1  2     4565612     4495920
>>   setvert3d:   4    86046564   122013924    86046564    86046564
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 8041169 unique labels shared
>>   pairwise times (avg, min, max): 0.00050716 0.000427103 0.00056479
>>   crystal router                : 0.0040165 0.00392921 0.00411811
>>   used all_to_all method: pairwise
>>   setupds time 1.0465E+00 seconds   2  4    86046564     4495920
>> setup h1 coarse grid, nx_crs=            2
>> call usrsetvert
>> done :: usrsetvert
>>
>> gs_setup: 866937 unique labels shared
>>   pairwise times (avg, min, max): 0.000233683 0.000197816 0.00024941
>>   crystal router                : 0.000466869 0.00045588 0.000478101
>>   used all_to_all method: pairwise
>> ########################################################################
>>
>>
>> I was wondering if you could help me with that. I attached the run
>> logfile and also genmap.out.
>>
>> Many thanks
>> Azad
>>
>
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
>