[Swift-devel] Re: [VDL2-devel] Re: test v0.1rc1

Mon Feb 26 12:12:15 CST 2007

On Mon, 2007-02-26 at 11:49 -0600, Ian Foster wrote:
> Mike:
> 
> My big concern continues to be that we diffuse our efforts by trying to 
> do too many things at once. I asked how things were going last week, and 
> someone (can't recall who) replied: "we are consumed getting new grid 
> sites to work."

And whether that statement is actually accurate and reflects the state
of things is another story. Also whether it refers to something that is
done to swift or it's about figuring out where Globus services are
installed on the sites, and whether they work, is yet a different
dimension.

What I know:
- I haven't changed the scheduler code in a while. The only change was
to support file operation throttling, but that was necessary even for
one site.
- The nightly tests have been running for quite a while on two sites,
and it seems to work fine (the % of successful workflows is >50% even if
one of the sites is down).
- I am "consumed" by updating documentation and polishing various rough
edges in Swift and thinking about addressing various problems that seem
to be of high importance to certain applications and with long term
benefits. Not much new there.

So I would like to ask folks to be careful about generic statements, and
be clear about what the problems are.

>  Meanwhile, there is no documentation on the Swift web 
> site of a single application that runs end to end, showing the code run 
> and the performance achieved. That doesn't seem good to me, if we want 
> to deliver value to users.
> 
> We of course do want to run across multiple sites, and soon--but I 
> continue to hope that we can be disciplined about approaching things one 
> step at a time.

The multiple sites step has been done for a long time now. Mostly
because Karajan comes with it by default. I think it may be time to
retire this discussions. To me it looks that most of the times when it
occurs, it is triggered by bits and pieces of information, like the
above, which are either not accurate or easy to misinterpret.

Mihael

> 
> Ian.
> 
> Mike Wilde wrote:
> > Ian Foster wrote, On 2/26/2007 11:14 AM:
> >> I am puzzled why we are testing on multiple sites, rather than a 
> >> single site.
> >
> > I believe that we need to, that the code is there to do this, and 
> > needs testing. This is not an unreasonable time to start trying this.
> >
> > If we are to be taken seriously by OSG, and as a Grid, we need to use 
> > multiple sites.
> >
> > We can set the priority low for the last anomaly that Tibi pointed out 
> > below (using 3 sites out of 4).  We decided in 0.1 that we would 
> > enable more sites to run (although we said there "not multiple").  0.1 
> > is virtually done; we have not yet frozen the feature set of 0.2.  
> > This is a reasonable candidate feature to consider for 0.2.
> >
> > People can be focusing and still mention (and file as bugs) things 
> > that they encounter in testing. They should be able to try things a 
> > bit out of the box as long as we are making good progress, which we are.
> >
> > - Mike
> >
> >>
> >> Ben Clifford wrote:
> >>> is this different behaviour from what you've observed with other 
> >>> versions?
> >>>
> >>> On Mon, 26 Feb 2007, Tiberiu Stef-Praun wrote:
> >>>
> >>>  
> >>>> Some early observations:
> >>>>
> >>>> I'm running the SIDGrid workflow from teraport, and some early
> >>>> observations: ( I have not finished a full run yet):
> >>>>
> >>>> - data seemed to be delayed in transferring back (from the NCSA
> >>>> site). I waited 5 minutes after the execution apparently finished on
> >>>> the remote site (I was logged in and was monitoring the output files)
> >>>> then stopped the workflow. Still investigating
> >>>> - the workflow chose 3 sites (I had 4 available) and it started 6
> >>>> parallel jobs on each site. Strange not to choose all the available
> >>>> sites. Still investigating
> >>>>
> >>>> Tibi
> >>>>
> >>>>
> >>>> On 2/26/07, Tiberiu Stef-Praun <tiberius at ci.uchicago.edu> wrote:
> >>>>    
> >>>>> I have to do measurements of the SIDGrid, so I'll use this new 
> >>>>> release
> >>>>> to do that.
> >>>>> You'll hear from me.
> >>>>>
> >>>>> Tibi
> >>>>>
> >>>>> On 2/26/07, Ben Clifford <benc at hawaga.org.uk> wrote:
> >>>>>      
> >>>>>> On Mon, 26 Feb 2007, Ben Clifford wrote:
> >>>>>>
> >>>>>>        
> >>>>>>> v0.1rc1 was built at the end of last week. please spend some time
> >>>>>>>           
> >>>>> testing
> >>>>>      
> >>>>>> here's the URL for download:
> >>>>>>
> >>>>>> http://www.ci.uchicago.edu/swift/tests/vdsk-0.1rc1.tar.gz
> >>>>>>
> >>>>>> -- 
> >>>>>>
> >>>>>>         
> >>>>> -- 
> >>>>> Tiberiu (Tibi) Stef-Praun, PhD
> >>>>> Research Staff, Computation Institute
> >>>>> 5640 S. Ellis Ave, #405
> >>>>> University of Chicago
> >>>>> http://www-unix.mcs.anl.gov/~tiberius/
> >>>>>
> >>>>>       
> >>>>
> >>>>     
> >>>
> >>> _______________________________________________
> >>> Swift-devel mailing list
> >>> Swift-devel at ci.uchicago.edu
> >>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >>>
> >>>   
> >>
> >> -- 
> >>
> >>    Ian Foster, Director, Computation Institute
> >> Argonne National Laboratory & University of Chicago
> >> Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
> >> Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
> >> Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
> >>       Globus Alliance: www.globus.org.
> >>
> >>
> >> ------------------------------------------------------------------------
> >>
> >> _______________________________________________
> >> Swift-devel mailing list
> >> Swift-devel at ci.uchicago.edu
> >> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >
> 
> -- 
> 
>    Ian Foster, Director, Computation Institute
> Argonne National Laboratory & University of Chicago
> Argonne: MCS/221, 9700 S. Cass Ave, Argonne, IL 60439
> Chicago: Rm 405, 5640 S. Ellis Ave, Chicago, IL 60637
> Tel: +1 630 252 4619.  Web: www.ci.uchicago.edu.
>       Globus Alliance: www.globus.org.
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 