[Swift-devel] Re: swift testing of gram5 on teraport

[Stuart Martin]

On Jun 19, 2009, at Jun 19, 11:08 AM, Michael Wilde wrote:

> Is what we're looking to see here the ability to run Swift with a  
> full or wide throttle to Gram5, directly, without Condor-G, and the  
> ability to have (a) lots of jobs in the queue and (b) many more jobs  
> running at once, while watching the gatekeepr host for CPU stress  
> and memory pressure?

Yes - exactly.

> Where say (a) is a few thousand jobs and (b) is the full cluster busy?

Yes and Yes.

>
> I wonder if we can get a full-system reservation on TeraPort to test  
> this?

I don't know.

>
> We're also testing Swift via Condor-G at the moment on UNL's new  
> cluster "Firefly" which has 6000 cores of which 3000 are accessible  
> to OSG. As its a new and lightly loaded cluster, perhaps Brian  
> Bockelman would be willing to test GRAM5 on it? (its a PBS cluster)

Ok - I'll check with Brian.

> So, now that I think about, as long as there's a GRAM5 gatekeeper we  
> can use, since it should Just Work, Im sure we can give it some  
> informal usage as soon as its available.

Cool.

> Stu, do you have plans for testing beyon Teraport on larger clusters?

Yes.  CMS will be doing the initial test for OSG.  If that goes well,  
then it could be used throughout OSG.  So, CMS using GRAM5 will be a  
good test.

>
> I wonder, maybe we could test it in AWS at large scales too on a  
> Nimbus workspace?

I suppose.  What would that entail?  You'd want an image with a gram5  
service running that interfaces with some LRM (PBS, SGE, Condor, ...)  
system.  Then that LRM managing a set of worker VMs?  What have you  
done with AWS/Nimbus sofar?  Anything like this?

>
> - Mike
>
>
> On 6/19/09 10:58 AM, Ben Clifford wrote:
>> On Fri, 19 Jun 2009, Michael Wilde wrote:
>>> In parallel, we should discuss on the list what ifany Swift  
>>> changes are needed
>>> to use it. It dont have my head around the issue at the moment.  
>>> Where can we
>>> read the specs of how it affects the user?
>> Theoretically it will Just Work with the GRAM2 provider. Evidence  
>> thus far suggests this might be true (for example, apparently the  
>> gram2 cog stuff can submit to gram5 ok) but there hasn't been any  
>> swift-level testing to see how it all fits together.