[Parvis-users] Performance of the parncl.

[Jayesh Krishna]

Hi,
 Please let us know more information regarding the dataset that you are using in your scripts (Attach "ncdump -h" of one of the netcdf files) so that we can try out more tests.

Regards,
Jayesh

----- Original Message -----
From: "Jayesh Krishna" <jayesh at mcs.anl.gov>
To: "Matrix" <sov_matrix at 126.com>
Cc: parvis-users at lists.mcs.anl.gov
Sent: Saturday, April 13, 2013 12:49:49 PM
Subject: Re: [Parvis-users] Performance of the parncl.

Hi,
 We usually test performance of ParNCL on a system with a parallel file system (like PVFS). I will try some tests on my laptop (with no parallel file system) and let you know the results.
 Please try the vorticity test, plot_vort.ncl, in the example scripts page (http://trac.mcs.anl.gov/projects/parvis/wiki/ParNCLDeveloperInfo/SimpleParNCLScripts) and let us know the results.

Regards,
Jayesh

----- Original Message -----
From: "Matrix" <sov_matrix at 126.com>
To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
Sent: Saturday, April 13, 2013 12:54:40 AM
Subject: Re:Re: [Parvis-users] Performance of the parncl.


yes, I did try running with 4-8cores and more. 
here is my result: 

1 process uses 7492 ms 
2 processes use 17531 ms 
3 processes use 16345 ms 
4 processes use 16542 ms 
5 processes use 17814 ms 
6 processes use 19272 ms 
7 processes use 22437 ms 
8 processes use 22507 ms 
9 processes use 26080 ms 
10 processes use 25356 ms 
11 processes use 26817 ms 
12 processes use 25638 ms 
13 processes use 29558 ms 
14 processes use 27958 ms 
15 processes use 28537 ms 
16 processes use 30162 ms 
The running environment: 16 cores,RAID,but we don't have a parallel file system. 
I don't know why running with only 1 process is the best , and why the more processes I run with , the poorer performance I get. what ' s wrong with my job? 
I wonder if you have tested the ParNCL on a machine with parallel file system.and can you give me a script calculating vorticity, divergence ? thank you very much. 







At 2013-04-12 23:37:27,"Jayesh Krishna" <jayesh at mcs.anl.gov> wrote:
>Hi,
> How many processors are you using to run your script? Did you try running your job with 4-8 cores/processors?
> The grid-aware-parallelism in ParNCL/ParGAL comes with its costs (read/distribute data across processes; gather results; etc) and for simpler calculations like dim_avg_n() these costs show up in performance results. For more complex calculations like calculating vorticity, divergence etc ParNCL/ParGAL perform an order of magnitude better than NCL (serial version). So you might have to play with the number of processors to get better results.
> Let us know the results.
>
>(PS: We are also looking into making these simpler calculations more efficient. I have created a ticket on this issue - http://trac.mcs.anl.gov/projects/parvis/ticket/71)
>
>Regards,
>Jayesh
>
>----- Original Message -----
>From: "Matrix" <sov_matrix at 126.com>
>To: "Jayesh Krishna" <jayesh at mcs.anl.gov>
>Sent: Friday, April 12, 2013 10:12:18 AM
>Subject: Re:Re: [Parvis-users] Performance of the parncl.
>
>
>Here is my script .Thank you. 
>
>
>load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl" 
>load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl" 
>load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/contributed.ncl" 
>load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/shea_util.ncl" 
>begin 
>fili = systemfunc("cat listname") ; 50 yrs of cam data. 
>
>setfileoption("nc","MissingToFillValue",True) ; alter default mode 
>
>f = addfiles(fili, "r") ; add the files 
>
>t = f[:]->T ; read temp 
>
>t_avg = dim_avg_n(t, 0) ; calc dim_avg 
>
>wks=gsn_open_wks("ps", "parvis_cam_eulspec_plotter_parncl") 
>gsn_define_colormap(wks,"gui_default") 
>
>res = True 
>res at cnFillOn = True 
>res at gsnSpreadColors = True 
>res at mpOceanFillColor = 0 
>res at gsnStringFontHeightF = 0.02 
>res at gsnLeftStringFontHeightF = 0.02 
>res@ gsnRightStringFontHeightF = 0.02 
>res at tmXBLabelFontHeightF = 0.017 
>res at tmYLLabelFontHeightF = 0.017 
>res at lbLabelFontHeightF = 0.017 
>res at cnLinesOn = False 
>res at lbOrientation = "vertical" 
>res at mpCenterLonF = 180 
>res at cnLevelSelectionMode = "ExplicitLevels" 
>
>plot=gsn_csm_contour_map_ce(wks, t_avg(0,:,:), res) 
>
>end 
>
>
>
>
>
>
>
>
>At 2013-04-12 22:29:45,"Jayesh Krishna" <jayesh at mcs.anl.gov> wrote:
>>Hi,
>> How are you running ParNCL ? Can you send us the NCL scripts that you are using?
>>
>>(PS: Note that some computations that are trivial might require higher number of processors for better performance.)
>>Regards,
>>Jayesh
>>
>>----- Original Message -----
>>From: "Matrix" <sov_matrix at 126.com>
>>Cc: parvis-users at lists.mcs.anl.gov
>>Sent: Friday, April 12, 2013 3:44:13 AM
>>Subject: [Parvis-users] Performance of the parncl.
>>
>>
>>
>>Dear Professor: 
>>I met with some troubles when using the ParNCL,I've tested the performance of the ParNCL, and it is not better than NCL, I want to know if I made something wrong with the ParNCL. So, I beg you to give me some information about the running environment when you testing the ParNCL. Thank you . 
>>Best wishes, 
>>Ji Xu 
>>Tsinghua University. 
>>
>>
>>
>>
>>_______________________________________________
>>Parvis-users mailing list
>>Parvis-users at lists.mcs.anl.gov
>>https://lists.mcs.anl.gov/mailman/listinfo/parvis-users 
> 

_______________________________________________
Parvis-users mailing list
Parvis-users at lists.mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/parvis-users