[Nek5000-users] mesh problems

nek5000-users at lists.mcs.anl.gov nek5000-users at lists.mcs.anl.gov
Thu Dec 27 05:13:33 CST 2012


Dear Neks,

I generated a big mesh with 7948800 elements for a cylindrical cell of aspect ratio 3, i.e.

x between -1.5 and 1.5
y between -1.5 and 1.5
z between 0 and 1

prex, n2to3, reatore2, amgdump and genmap worked. The AMG preconditioning with go.m worked as well (runtime 3.5 weeks).

prex started out of 3 basic elements which were refined with multi split into 

Element1:  90, 90 subelements
Element2:  70, 90
Element3:  70, 90
stretch, r, factor 3         

I started the job on BG/Q and get the following error messages in the output file:

bash-4.1$ m G3_96.out.0
/----------------------------------------------------------\
|      _   __ ______ __ __  ______  ____    ____    ____   |
|     / | / // ____// //_/ / ____/ / __ \ / __ \ / __ \ |
|    /  |/ // __/  / ,<   /___ \  / / / // / / // / / /   |
|   / /|  // /___ / /| | ____/ / / /_/ // /_/ // /_/ /     |
|  /_/ |_//_____//_/ |_|/_____/  \____/ \____/ \____/   |
|                                                          |
|----------------------------------------------------------|
|                                                          |
| NEK5000:  Open Source Spectral Element Solver            |
| COPYRIGHT (c) 2008-2010 UCHICAGO ARGONNE, LLC            |
| Version:  1.0rc1 / SVN r838                              |
| Web:      http://nek5000.mcs.anl.gov                     |
|                                                          |
\----------------------------------------------------------/
                                                              

 Number of processors: 65536
 REAL    wdsize      : 8
 INTEGER wdsize      : 4


  Beginning session:
  /homec/hil02/hil020/nek5_svn/g396_r097_Lx1_08_Ra1e9/G3_96.rea                 

 
 timer accuracy:   1.4962500E-07 sec

 read .rea file
 read .re2 file
 byte swap: T -0.2931277218E+36 6.543210030
 nelgt/nelgv/lelt:  7948800  7948800      150
 lx1  /lx2  /lx3 :        4        2        4

 mapping elements to processors
 element load imbalance:  1 121 122
 done :: mapping elements to processors
  
   reading mesh 
   reading curved sides 
   reading bc for ifld 1
   reading bc for ifld 2
 done :: read .re2 file
  
 0  objects found
 done :: read .rea file    496.34     sec

 Reset the target Courant number to .5
 setup mesh topology
   Right-handed check complete for 7948800 elements. OK.
   setvert3d:   4   152084809   215675209   152084809   152084809
 call usrsetvert
 done :: usrsetvert

gs_setup: 50336837 unique labels shared
   pairwise times (avg, min, max): 0.00330717 0.000218656 0.00853093
   crystal router                : 0.0077495 0.00761262 0.00818637
   used all_to_all method: crystal router
   setupds time 6.1645E-01 seconds   0  4   152084809     7948800
 8  max multiplicity
 done :: setup mesh topology
  
 call usrdat
 done :: usrdat

 generate geomerty data
 vol_t,vol_v: 7.06857958483581861 7.06857958483581861
 done :: generate geomerty data
  
 call usrdat2
 done :: usrdat2

 regenerate geomerty data 1
 vol_t,vol_v: 7.06857958483581861 7.06857958483581861
 done :: regenerate geomerty data 1
  
 verify mesh topology
 -1.50001142304138035 1.50001142290878886  Xrange
 -1.50001142298026835 1.50001142292693612  Yrange
 0.000000000000000000E+00 1.00000000000000000  Zrange
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    4    2    1   954634
 i,j,k,ie:    1    2    1   916744
 i,j,k,ie:    2    4    2   972970
 i,j,k,ie:    2    1    1   931570
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    4    2    1   957874
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    4    2    1   996664
 Near X =  -.12308168E-01  -.34092836       .60036000E-01, d:   .12947845E-04   .00000000E+00   81.225474    
 WARNING1 Element mesh mismatch at:
 Near X =   .11336871E-01  -.82826699       .61838637E-01, d:   .00000000E+00  -.11766071E-04   88.069931    
 Near X =   .97128875E-05   .82845591       .60036000E-01, d:   .00000000E+00  -.20015559E-04   88.069931    
 i,j,k,ie:    2    1    1   981007
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    2    1    1   939607
 i,j,k,ie:    4    2    4   995854
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    2    4    3   972989
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    2    4    4   931590
 Near X =   .12193251E-01   .42841310       .66558003E-01, d:   .00000000E+00  -.12947474E-04   81.991313    
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    4    2    4   916204
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    1    2    4   958144
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 Near X =   .12320624E-01   .31593135       .73282003E-01, d:   .12573222E-04   .00000000E+00   81.079569    
 i,j,k,ie:    3    2    4   541894
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    2    1    1   981003
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    2    4    1   939603
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 i,j,k,ie:    2    4    4   890151
 WARNING1 Element mesh mismatch at:
 Near X =  -.62894639E-05  -1.1769232       .60036000E-01, d:   .15284684E-04   .00000000E+00   92.459135    
 WARNING1 Element mesh mismatch at:
 Near X =   .63923475E-05   1.1769232       .60036000E-01, d:   .00000000E+00  -.15284684E-04   92.458255    
 i,j,k,ie:    3    2    4     2524
 i,j,k,ie:    3    2    4   624964
 Near X =  -.10302672E-04   .82845591       .60036000E-01, d:   .20015559E-04   .00000000E+00   88.062609    
 i,j,k,ie:    2    4    4    62171
 i,j,k,ie:    2    4    4   600363
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
 WARNING1 Element mesh mismatch at:
---------------------------------------------------------
and so on…….


The job hangs at this position and does not continue. Can it be that the precision of the coordinate ranges is not sufficient?
For a job that runs well (here an aspect ratio one cell as an example) I got in the output file the following coordinates.

……

 verify mesh topology
 -0.500000000000000000 0.500000000000000000  Xrange
 -0.500000000000000000 0.500000000000000000  Yrange
 0.000000000000000000E+00 1.00000000000000000  Zrange
 done :: verify mesh topology

……

i.e. precise bounds down to the last digit. 
Note also that I had to adapt the programs n2to3 and reatore2, e.g.:

[josc1234 at reynolds n2to3]$ diff n2to3.f  n2to3.f_orig
31c31
<       parameter(nelm=99999)
---
>       parameter(nelm=9999)

[josc1234 at reynolds reatore2]$ diff reatore2.f  reatore2.f_orig
19c19
<       parameter(lelt=10000000)
---
>       parameter(lelt=1000000)
118c118


I wonder if you have an idea where things went wrong? Is it already the prex?
Thanks for your help in advance.

Best regards, Joerg. 




More information about the Nek5000-users mailing list