[Swift-user] Query regarding optimal configuration in Swift for running code in Midway.

[Yadu Nand]

Hi Kushal,

In the discussion we had off-list, I'd recommended the following changes:

* Set maxWallTime to ~9hours and maxJobTime to ~36hours.
* Set initialParallelTasks and maxParallelTasks to the number of app
invocation necessary.
* Since the application does not use multiple-cores, set tasksPerNode to
the number of cores on the target machines
* Add pools for other queues (westmere, sandyb, and bigmem)
* maxNodesPerJob=1

Were you able to complete the runs with these changes ?

Thanks,
Yadu

On Fri, Jan 9, 2015 at 10:19 AM, kushal kumar dey <kshldey at gmail.com> wrote:

> Hi,
>
> I am a graduate student in the University of Chicago and I am currently
> trying to run a parallel script using Swift-lang on the Midway cluster. I
> need your help regarding optimally choosing the configuration to run my
> program.  I am running 100 parallel scripts at a time in the "bigmem"
> machine, and each task takes around *5 hours *to complete. I have been
> asked in the Swift Configuration file (the program I am using to run these
> parallel scripts)  to set the following in the configuration file
>
> max Nodes per Job :
> maxJobs:
> tasks per Node:
> max Job time:  (This I am setting as 36 hours currently)
> max Wall time : (this i am setting as 34 hours right now)
> max Parallel Task
> Initial Parallel Task
>
> Can you please let me know what numbers I should be putting in these
> options so that I can optimally run my 100 jobs in parallel and given that
> each of these tasks takes around 5 hours, whether it would be possible to
> complete all the 100 tasks in 36 hours?
>
> If you could give me the optimal configuration or at least give some
> intuition about how to fix it, it would be really helpful..This is the
> first time I am running jobs in parallel, so I have very little idea about
> it...
>
>
> Thanks a lot
>
> Kushal
>
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>



-- 
Yadu Nand B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150113/90933d14/attachment.html>