[Nek5000-users] Interpolation between two meshes

nek5000-users at lists.mcs.anl.gov nek5000-users at lists.mcs.anl.gov
Tue Mar 28 16:14:56 CDT 2017


Can you also call nekgsync() before and after findpts. Just trying to understand if it really hangs there. 
 
-----Original message-----
> From:nek5000-users at lists.mcs.anl.gov <nek5000-users at lists.mcs.anl.gov>
> Sent: Tuesday 28th March 2017 22:59
> To: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] Interpolation between two meshes
> 
> We have tested glfdr() using different cases/machines and it worked fine. With the information below I cannot say what's going wrong. 
> 
> - Is your source fld in double precision?
> - Do have any additional information where it hangs or crashes?
> - Can you please test gfldr() say with turbChannel (NekExamples).
> 
> Stefan
>  
> -----Original message-----
> > From:nek5000-users at lists.mcs.anl.gov <nek5000-users at lists.mcs.anl.gov>
> > Sent: Tuesday 28th March 2017 22:46
> > To: nek5000-users at lists.mcs.anl.gov
> > Subject: Re: [Nek5000-users] Interpolation between two meshes
> > 
> > Hi Stefan,
> > 
> > In case the attachments didn't go through, please see below for the .usr file and log file.
> > I only used 'userchk' and 'usrdat' in the .usr file for interpolation. The other subroutines are empty.
> > (I added several print lines in 'gfldr.f' to see where the interpolation halted.)
> > 
> > Best regards,
> > Tony
> > 
> > .USR FILE:
> > c-----------------------------------------------------------------------
> >       subroutine userchk()
> >       include 'SIZE'
> >       include 'TOTAL'
> >       include 'ZPER'
> >       include 'CTIMER'
> >       include 'RESTART'
> > 
> >       character*132 sourcefld
> > 
> >       sourcefld='source.f00001'
> >       call gfldr(sourcefld)
> >       call outpost(vx,vy,vz,pr,t,'rst')
> >       call exitt()
> > 
> >       return
> >       end
> > c-----------------------------------------------------------------------
> >       subroutine usrdat()   ! This routine to modify element vertices
> >       include 'SIZE'
> >       include 'TOTAL'
> > 
> >       parameter (ielx=150)
> >       parameter (iely=22)
> >       parameter (ielz=142)
> >       integer i,e,eg,ex,ey,ez
> >       real cony
> > 
> >       cony = 2.36d0
> > 
> >       do e=1,nelv
> >         eg = lglel(e)
> >        if(eg<=ielx*iely*ielz) then
> >         call get_exyz(ex,ey,ez,eg,ielx,iely,ielz)
> > 
> >         do i=1,8
> >          if(i<=2 .or. i==5 .or. i==6) then
> >           if((yc(i,e)>0.0e0.and.yc(i,e)<1.0e0).OR.
> >      &      (yc(i,e)>1.0e0.and.yc(i,e)<2.0e0)) then
> >            yc(i,e) = 2.0e0*Real(ey-1)/Real(iely)
> >           end if
> >          else if(i==3 .or. i==4 .or. i>=7) then
> >           if((yc(i,e)>0.0e0.and.yc(i,e)<1.0e0).OR.
> >      &      (yc(i,e)>1.0e0.and.yc(i,e)<2.0e0)) then
> >            yc(i,e) = 2.0e0*Real(ey)/Real(iely)
> >           end if
> >          end if
> >         end do
> > 
> >        end if
> >       end do
> > 
> >       do e=1,nelv
> >         eg = lglel(e)
> >         if(eg==1) print*,yc(1,e),yc(2,e),yc(5,e),yc(6,e)
> >         if(eg==1) print*,yc(3,e),yc(4,e),yc(7,e),yc(8,e)
> >         do i=1,8
> >          if(yc(i,e)>0.0e0 .AND. yc(i,e)<1.0e0) then
> >            yc(i,e) = yc(i,e)-1.0e0
> >            yc(i,e) = tanh(cony*yc(i,e))/tanh(cony)
> >            yc(i,e) = yc(i,e)1.0e0
> >          else if(yc(i,e)>1.0e0 .AND. yc(i,e)<2.0e0) then
> >            yc(i,e) = yc(i,e)-1.0e0
> >            yc(i,e) = tanh(cony*yc(i,e))/tanh(cony)
> >            yc(i,e) = yc(i,e)1.0e0
> >          end if
> >         end do
> >       end do
> > 
> >       return
> >       end
> > c-----------------------------------------------------------------------
> > 
> > 
> > LOG FILE:
> > 
> > /----------------------------------------------------------\\
> > |      _   __ ______ __ __  ______  ____   ____   ____     |
> > |     / | / // ____// //_/ / ____/ / __ \\/ __ \\/ __ \\   |
> > |    /  |/ // __/  / ,<   /___ \\ / / / // / / // / / /    |
> > |   / /|  // /___ / /| | ____/ / / /_/ // /_/ // /_/ /     |
> > |  /_/ |_//_____//_/ |_|/_____/  \\___/ \\___/ \\___/      |
> > |                                                          |
> > |----------------------------------------------------------|
> > |                                                          |
> > | NEK5000:  Open Source Spectral Element Solver            |
> > | COPYRIGHT (c) 2008-2017 UCHICAGO ARGONNE, LLC            |
> > | Version:  17.0.0                                         |
> > | Web:      http://nek5000.mcs.anl.gov                     |
> > |                                                          |
> > \\----------------------------------------------------------/
> > 
> > 
> >  Number of processors:         720
> >  REAL    wdsize      :           8
> >  INTEGER wdsize      :           4
> >  Timer accuracy      : 9.54E-08
> >  Reading /work/e01/e01/tony/geometry_nonosci_cavity/interp.rea
> > 
> >  mapping elements to processors
> >  Reading /work/e01/e01/tony/geometry_nonosci_cavity/interp.map
> >  element load imbalance:            1         652         653
> >  done :: mapping elements to processors
> > 
> >            0  objects found
> >  118   Parameters from file:
> >    1      1.00000     p001 DENSITY
> >    2     -3150.00     p002 VISCOS
> >   12     -0.11750E-01 p012 DT
> >   15      500.000     p015 IOSTEP
> >   21     0.100000E-07 p021 DIVERGENCE
> >   22     0.100000E-07 p022 HELMHOLTZ
> >   26      2.00000     p026 COURANT/NTAU
> >   27      3.00000     p027 TORDER
> >   59      0.00000     p059 !=0 --> full Jac. eval. for each el.
> >   63      8.00000     p063 =8 --> force 8-byte output
> >   65      1.00000     p065 #iofiles (eg, 0 or 64); <0 --> sep. dirs
> >   66      6.00000     p066 output : <0=ascii, else binary
> >   67      6.00000     p067 restart: <0=ascii, else binary
> >   93      20.0000     p093 Number of previous pressure solns saved
> >   94      5.00000     p094 start projecting velocity after p94 step
> >   95      5.00000     p095 start projecting pressure after p95 step
> >   99      3.00000     p099 dealiasing: <0--> off/3--> old/4--> new
> >  103      0.10000E-01 p103 weight of stabilizing filter (.01)
> > 
> >  nelgt/nelgv/lelt:      469500      469500         653
> >  lx1  /lx2  /lx3 :           8           8           8
> >  done :: read .rea file    45.222     sec
> > 
> >  setup mesh topology
> >    Right-handed check complete for  469500 elements. OK.
> >    setvert3d:   8    60855000   162267000    60855000    60855000
> >  call usrsetvert
> >  done :: usrsetvert
> > 
> > gs_setup: 10325502 unique labels shared
> >    pairwise times (avg, min, max): 0.00044986 0.00033021 0.000500798
> >    crystal router                : 0.00145524 0.00143301 0.0014781
> >    used all_to_all method: pairwise
> >    handle bytes (avg, min, max): 2.64212e06 2463020 2796252
> >    buffer bytes (avg, min, max): 491593 352288 632640
> >    setupds time 5.3504E-01 seconds   0  8    60855000      469500
> >            8  max multiplicity
> >  done :: setup mesh topology
> > 
> >   0.000000000000000E000  0.000000000000000E000  0.000000000000000E000
> >   0.000000000000000E000
> >   9.090909090909091E-002  9.090909090909091E-002  9.090909090909091E-002
> >   9.090909090909091E-002
> >  call usrdat
> >  done :: usrdat
> > 
> >  generate geometry data
> >  NOTE: All elements deformed , param(59) ^=0
> >  done :: generate geometry data
> > 
> >  call usrdat2
> >  done :: usrdat2
> > 
> >  regenerate geometry data           1
> >  NOTE: All elements deformed , param(59) ^=0
> >  done :: regenerate geometry data           1
> > 
> >  verify mesh topology
> >   0.000000000000000E000   6.00000000000000       Xrange
> >  -0.120000000000000        2.00000000000000       Yrange
> >   0.000000000000000E000   49.7000000000000       Zrange
> >  done :: verify mesh topology
> > 
> >  IFTRAN   = T
> >  IFFLOW   = T
> >  IFHEAT   = F
> >  IFSPLIT  = T
> >  IFLOMACH = F
> >  IFUSERVP = F
> >  IFUSERMV = F
> >  IFSTRS   = F
> >  IFCHAR   = T
> >  IFCYCLIC = F
> >  IFAXIS   = F
> >  IFMVBD   = F
> >  IFMELT   = F
> >  IFMODEL  = F
> >  IFKEPS   = F
> >  IFMOAB   = F
> >  IFNEKNEK = F
> >  IFSYNC   = T
> > 
> >  IFVCOR   = F
> >  IFINTQ   = F
> >  IFCWUZ   = F
> >  IFSWALL  = F
> >  IFGEOM   = F
> >  IFSURT   = F
> >  IFWCNO   = F
> > 
> >  IFTMSH for field           1    =  F
> >  IFADVC for field           1    =  T
> >  IFNONL for field           1    =  F
> > 
> >  Dealiasing enabled, lxd=          12
> >  verify mesh topology
> >   0.000000000000000E000   6.00000000000000       Xrange
> >  -0.120000000000000        2.00000000000000       Yrange
> >   0.000000000000000E000   49.7000000000000       Zrange
> >  done :: verify mesh topology
> > 
> >  call usrdat3
> >  done :: usrdat3
> > 
> >  set initial conditions
> >  nekuic (1) for ifld            1
> >  call nekuic for vel
> >  xyz min     0.0000     -0.12000       0.0000
> >  uvwpt min   0.0000       0.0000       0.0000       0.0000       0.0000
> >  xyz max     6.0000       2.0000       49.700
> >  uvwpt max   0.0000       0.0000       0.0000       0.0000       0.0000
> >  done :: set initial conditions
> > 
> > gridpoints unique/tot:     162267000    240384000
> >   dofs:             159972750             162104250
> > 
> >  Initialization successfully completed    46.588     sec
> > 
> >  nsteps=0 -> skip time loop
> >  running solver in post processing mode
> > 
> >  call userchk
> >  call gfldr
> >  byte swap: F   6.543210     -2.9312772E35
> >  sourcefld X  0.000000000000000E000   6.00000000000000
> >  sourcefld Y -0.120000000000000        2.00000000000000
> >  sourcefld Z  0.000000000000000E000   49.7000000000000
> >  CHECK interp prepare
> >  CHECK interp read
> > 
> > -----------
> > From: nek5000-users-bounces at lists.mcs.anl.gov <nek5000-users-bounces at lists.mcs.anl.gov> on behalf of nek5000-users-request at lists.mcs.anl.gov <nek5000-users-request at lists.mcs.anl.gov>
> > Sent: 28 March 2017 20:51
> > To: nek5000-users at lists.mcs.anl.gov
> > Subject: Nek5000-users Digest, Vol 97, Issue 33 
> > 
> > Message: 3
> > Date: Tue, 28 Mar 2017 19:51:04 0000
> > From: nek5000-users at lists.mcs.anl.gov
> > To: "nek5000-users at lists.mcs.anl.gov"
> >         <nek5000-users at lists.mcs.anl.gov>
> > Subject: Re: [Nek5000-users] Interpolation between two meshes
> > Message-ID:
> >         <mailman.3915.1490730701.2967.nek5000-users at lists.mcs.anl.gov>
> > Content-Type: text/plain; charset="iso-8859-1"
> > 
> > Hi Stefan,
> > 
> > 
> > Many thanks for your reply. I'm running my simulations on ARCHER (Cray XC30 supercomputer) and I'm using the Intel compiler.
> > 
> > 
> > The current simulation has 469500 elements with polynomial order of 7. The source field I used for interpolation has 383700 elements with the same polynomial order. The size of the source field is about 10GB. I'm not sure if it can be easily transferred. But
> >  I have attached my .usr and log files herewith (also the gfldr.f with few added checking lines). I ran the interpolation using 720 processors.
> > 
> > 
> > Please have a look and let me know if you need further information about the case. Thank you very much for you help.
> > 
> > 
> > Best regards,
> > 
> > Tony
> > _______________________________________________
> > Nek5000-users mailing list
> > Nek5000-users at lists.mcs.anl.gov
> > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> 


More information about the Nek5000-users mailing list