[Swift-devel] bug 53

[Mihael Hategan]

On Thu, 2007-09-13 at 17:35 -0500, Veronika Nefedova wrote:
> Thanks, Mihael! I could try submitting now some 20 molecules to tg-uc  
> (directly to GRAM) -- just to be on a safe side. If no GRAM problems  
> will be reported, I'll increase the number to 244.
> Of, course the performance will suffer greatly -- but I hope it would  
> enable to get the whole workflow to go throw. Are there any throttles  
> that could be set to increase a bit the performance (given that I set  
> the maxSubmitRate to 0.2) ?'

I'd use the defaults for the file stuff. The job throttle probably
doesn't matter, so you can probably set it to off. The submit throttle
is not relevant in this case (since it's less powerful than the above),
so you can probably set that one to off too.

Mihael

> 
> Nika
> 
> On Sep 13, 2007, at 4:41 PM, Mihael Hategan wrote:
> 
> > Ok, so there's something in.
> > There are some discussions that can be had on certain aesthetic  
> > topics.
> > In any event, in sites.xml, you can add, for a site, something like
> > this:
> >
> > <profile namespace="karajan" key="maxSubmitRate">0.1</profile>
> >
> > The rate is in jobs per second. The above would mean one job every ten
> > seconds.
> >
> > Mihael
> >
> > On Thu, 2007-09-13 at 15:23 +0000, Ben Clifford wrote:
> >> Yes?
> >>
> >> On Thu, 13 Sep 2007, Mihael Hategan wrote:
> >>
> >>> May I still fix that bug though?
> >>>
> >>> On Thu, 2007-09-13 at 09:54 -0500, Ioan Raicu wrote:
> >>>> Hi,
> >>>> I am still working on the new feature for Falkon to avoid  
> >>>> submitting
> >>>> tasks to known bad nodes, and to perhaps do its own retries for  
> >>>> failed
> >>>> jobs with certain known errors (i.e. stale NFS handle).  I  
> >>>> should have
> >>>> that ready for next week to try out.  Once this new feature is  
> >>>> in, we
> >>>> could try MolDyn again to see how it behaves.
> >>>>
> >>>> About avoiding Falkon of MolDyn, I recall something about the
> >>>> scalability/policies of GRAM/PBS to handle many con current jobs,
> >>>> having to throttle job submissions to something around 1 job  
> >>>> every 10
> >>>> seconds (for sustained periods of time, short bursts could send
> >>>> faster), and the fact that only a few 10s of nodes would be used
> >>>> concurrently, even though the sites that it was running on had more
> >>>> free nodes.  I also think that MolDyn through GRAM/PBS was running
> >>>> only 1 job per node, in essence only using 1 processor of the 2 per
> >>>> node.  I think the largest workflow Nika was able to run over  
> >>>> GRAM/PBS
> >>>> was 5 molecules, 421 jobs (but only 340 jobs in the large stage).
> >>>> Nika, were there other problems you encountered?
> >>>>
> >>>> Ioan
> >>>>
> >>>> Mihael Hategan wrote:
> >>>>> Very well Sir. I shall see to the priority of the issue being  
> >>>>> raised.
> >>>>>
> >>>>> On Thu, 2007-09-13 at 14:09 +0000, Ben Clifford wrote:
> >>>>>
> >>>>>> I think one of the main impediments to moldyn running with  
> >>>>>> GRAM directly
> >>>>>> is bug 53 which is a request for sumission rate limiting.
> >>>>>>
> >>>>>> It might be relatively easy to implement that and see how the  
> >>>>>> MolDyn
> >>>>>> workflow behaves then.
> >>>>>>
> >>>>>> I'm interested to see if Falkon can be avoided for this workflow.
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Swift-devel mailing list
> >>>>> Swift-devel at ci.uchicago.edu
> >>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >>>>>
> >>>>>
> >>>>
> >>>> -- 
> >>>> ============================================
> >>>> Ioan Raicu
> >>>> Ph.D. Student
> >>>> ============================================
> >>>> Distributed Systems Laboratory
> >>>> Computer Science Department
> >>>> University of Chicago
> >>>> 1100 E. 58th Street, Ryerson Hall
> >>>> Chicago, IL 60637
> >>>> ============================================
> >>>> Email: iraicu at cs.uchicago.edu
> >>>> Web:   http://www.cs.uchicago.edu/~iraicu
> >>>>        http://dsl.cs.uchicago.edu/
> >>>> ============================================
> >>>> ============================================
> >>>
> >>>
> >>
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >
>