[Swift-devel] Fwd: [gram-user] GRAM5 Alpha2

[Michael Wilde]

Stu,

Glen Hocky has started testing a protein folding app called "OOPS" on 
QueenBee under GRAM5. Initial tiny sanity tests look good; we'll move on 
to running 100+ job runs, then larger.

We needed to figure out how to get Swift to use all 8 cores of the 
QueenBee compute nodes, which we did.

Now we can start scaling up. Glen hopes to test more there shortly.

So far, no problems; no observed differences (in interface) with the new 
GRAM.

Any chance of getting GRAM5 on the firefly host at UNL?

- Mike


On 7/28/09 11:17 AM, Stuart Martin wrote:
> On Jul 28, 2009, at Jul 28, 10:56 AM, Michael Wilde wrote:
> 
>> Allan Espinosa will try to test AMPL workflows for the SEE project 
>> there this week.
>>
>> I may try a few others time permitting, but likely not this week.
>>
>> Questions, Stu:
>> - do you want testing through Condor-G with the grid_monitor as well 
>> as native?
> 
> I'd say to use GRAM5 as is best for you/your users.  We've done some 
> condor-g testing with and without the grid-monitor.  We did with, just 
> for backward compatibility.  But without is recommended.  The 
> grid-monitor is no longer needed with GRAM5.
> 
> So, if you have users that use condor-g, then submit GRAM5 jobs with 
> that.  But, turn off using the grid-monitor.
>     http://dev.globus.org/wiki/GRAM5_Scalability_Results#Test_7:_gram5-condor-g 
> 
> 
> But if it is "better" to submit them natively, through cog API I 
> assume(?), then do that.
> 
>> - for native testing of GRAM5 (ie through the plain pre-WS GRAM 
>> interface) are then any guidelines for how many jobs we can safely 
>> submit at once, or should we not worry about limits? (ie sending a few 
>> thousand jobs is OK?)
> 
> Don't worry about it and submit away.  We need to know the 
> limits/breaking points.
> 
> But, to show what we've done in our testing, here are the results from 
> our 5 client tests (each running in a separate VM) hitting the same 
> GRAM5 service.
>     http://dev.globus.org/wiki/GRAM5_Scalability_Results#Test_4:_5-client-seg_2 
> 
>     http://dev.globus.org/wiki/GRAM5_Scalability_Results#Test_5:_5-client-seg-diffusers_2 
> 
> 
> They submitted 5000 jobs over a 1 hour window to the same GRAM5 
> service.  The load on the head node never went above 4 on the first and 
> 7 on the second.
> 
>>
>> Allan: I just remembered that since Queenbee has 8-core hosts like 
>> Abe, coasters is the only reasonable approach for large-scale testing. 
>> But testing just a few AMPL jobs through plain GRAM5 seems a 
>> reasonable step to do first.
>>
>> I realize that coaster testing, also, wont give good CPU utilization 
>> until the current "low demand" problem is solved.
>>
>> - Mike
>>
>>
>> On 7/28/09 9:26 AM, Stuart Martin wrote:
>>> Hi Mike,
>>> Just following up on this.  Will there be some swift use of GRAM5 on 
>>> queen bee this week?
>>> -Stu
>>> On Jul 21, 2009, at Jul 21, 5:23 PM, Michael Wilde wrote:
>>>> Yes, there are a few we can run on QueenBee.
>>>>
>>>> Can try to test next week.
>>>>
>>>> Allan, we can test SEE/AMPL, OOPS, and PTMap there.
>>>>
>>>> - Mike
>>>>
>>>>
>>>> On 7/21/09 10:58 AM, Stuart Martin wrote:
>>>>> Are there any swift apps that can use queen bee?  There is a GRAM5 
>>>>> service setup there for testing.
>>>>> -Stu
>>>>> Begin forwarded message:
>>>>>> From: Stuart Martin <smartin at mcs.anl.gov>
>>>>>> Date: July 21, 2009 10:56:04 AM CDT
>>>>>> To: gateways at teragrid.org
>>>>>> Cc: Stuart Martin <smartin at mcs.anl.gov>, Lukasz Lacinski 
>>>>>> <lukas at cct.lsu.edu>
>>>>>> Subject: Fwd: [gram-user] GRAM5 Alpha2
>>>>>>
>>>>>> Hi Gateways,
>>>>>>
>>>>>> Any gateways that use (or can use) Queen Bee, it would be great if 
>>>>>> you could target this new GRAM5 service that Lukasz deployed.  I 
>>>>>> heard from Lukasz that Jim has submitted a gateway user (SAML) job 
>>>>>> and that went through fine and populate the gram audit DB 
>>>>>> correctly.  Thanks Jim!  It would be nice to have some gateway 
>>>>>> push the service to test scalability.
>>>>>>
>>>>>> Let us know if you plan to do this.
>>>>>>
>>>>>> Thanks,
>>>>>> Stu
>>>>>>
>>>>>> Begin forwarded message:
>>>>>>
>>>>>>> From: Lukasz Lacinski <lukas at cct.lsu.edu>
>>>>>>> Date: July 21, 2009 1:18:05 AM CDT
>>>>>>> To: gram-user at lists.globus.org
>>>>>>> Subject: [gram-user] GRAM5 Alpha2
>>>>>>>
>>>>>>> I've installed GRAM5 Alpha2 on Queen Bee.
>>>>>>>
>>>>>>> queenbee.loni-lsu.teragrid.org:2120/jobmanager-fork
>>>>>>> queenbee.loni-lsu.teragrid.org:2120/jobmanager-pbs
>>>>>>>
>>>>>>> -seg-module pbs works fine.
>>>>>>> GRAM audit with PostgreSQL works fine.
>>>>>>>
>>>>>>> Can someone submit jobs as a gateway user? I'd like to check if 
>>>>>>> the gateway_user field is written to our audit database.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Lukasz
>>>>>>
>>>>> _______________________________________________
>>>>> Swift-devel mailing list
>>>>> Swift-devel at ci.uchicago.edu
>>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>