[Swift-devel] Re: Swift Performance Data

[Mike Wilde]

OK, thanks for responding so quickly to your own request, Ian. :)

I agree that we need this and that this is a good spec to start from.

Ive been pushing to do all runs from the swift lab machines where we can readily 
collect this info.

We can create a set of scripts that collects the data in a uniform way, and 
makes it easy to send them to a central place.

Well start a campaign for this and move the discussion there.

I think that the stats should get gathered by default as part of the swift 
execution command; we may need some provision for collecting the data if swift 
dies.  Perhaps the swift shell wrapper can catch such errors and try to report 
in most cases.

I believe we also want to run everything under kickstart. Its been hard to get 
traction on that but we should discuss and keep on pushing on this.

We need to designate one person to lead on this and certainly others to contribute.

I hessitate to say "jump" until we review current work in progress and 
everyone's todo list.

Mike





Ian Foster wrote, On 6/21/2007 8:33 AM:
> Or maybe that is clear. I'd suggest that we want a tool that, after a 
> run, one of us can run to:
> 
> * Generate the three plots that Ioan has created
> * Generate a file containing as much information as we can about the run 
> and its parameters--maybe a name=value format?--and some derived values 
> such as those I mentioned in earlier email
> * Move these things to a known place
> * Create a Web page with pointers to these information and stick it 
> somewhere [or add it to an existing web page?]
> 
> Ian.
> 
> 
> 
> Ian Foster wrote:
>> Mike:
>>
>> It seems important to define what the specific goals and milestones 
>> are here, as it seems that simply asking for it doesn't get it done. 
>> Perhaps we need a brief specification?
>>
>> Ian.
>>
>> Mike Wilde wrote:
>>> Yes, this is what Ganglia has been using.
>>>
>>> Regarding the auto-publishing - Jens has a machanism that regularly 
>>> posted info in rrd format on the state of the VDS lab machines, using 
>>> a perl mechanism like what Ian described.  Perhaps we can find and 
>>> adapt that for Ioan's numbers.
>>> It was running on gainly I think. But its not hard to develop from 
>>> scratch.
>>>
>>> It would be good to see the same numbers for all the Swift apps being 
>>> worked on, driven initially by kickstart summaries and digesting the 
>>> swift logfile.
>>> We've long had this as a goal - now is a good time to push forward 
>>> and do this.
>>>
>>> Nika and Tibi, could you work with Ioan on this?
>>>
>>> - Mike
>>>
>>>
>>>
>>> Ian Foster wrote, On 6/20/2007 11:17 PM:
>>>> Hi,
>>>>
>>>> I was pointed at http://oss.oetiker.ch/rrdtool/, has anyone seen 
>>>> this? Seems nice to me.
>>>>
>>>> Ian.
>>>>
>>>>
>>>
>>
> 

-- 
Mike Wilde
Computation Institute, University of Chicago
Math & Computer Science Division
Argonne National Laboratory
Argonne, IL   60439    USA
tel 630-252-7497 fax 630-252-1997