<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7654.12">
<TITLE>RE: [Swift-user] using swift on a cluster</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>Here is the output:<BR>
<BR>
<BR>
[hodgess@grid bin]$ swift -tc.file tc.data -sites.file sites.xml firstR.swift<BR>
Swift 0.9 swift-r2860 cog-r2388<BR>
<BR>
RunID: 20091021-0901-aku7y862<BR>
Progress:<BR>
Execution failed:<BR>
No service contacts available<BR>
[hodgess@grid bin]$<BR>
<BR>
<BR>
<BR>
Erin M. Hodgess, PhD<BR>
Associate Professor<BR>
Department of Computer and Mathematical Sciences<BR>
University of Houston - Downtown<BR>
mailto: hodgesse@uhd.edu<BR>
<BR>
<BR>
<BR>
-----Original Message-----<BR>
From: Michael Wilde [<A HREF="mailto:wilde@mcs.anl.gov">mailto:wilde@mcs.anl.gov</A>]<BR>
Sent: Wed 10/21/2009 7:02 AM<BR>
To: Hodgess, Erin<BR>
Cc: swift-user@ci.uchicago.edu<BR>
Subject: Re: [Swift-user] using swift on a cluster<BR>
<BR>
For running Swift locally on a Condor cluster, use a sites.xml based on<BR>
this example:<BR>
<BR>
<execution provider="condor" url="none" /><BR>
<BR>
<config><BR>
<BR>
<pool handle="localhost"><BR>
<gridftp url="local://localhost" /><BR>
<execution provider="local" url="none" /><BR>
<workdirectory>/home/erin/swiftwork</workdirectory><BR>
<profile namespace="karajan" key="jobThrottle">.03</profile><BR>
<profile namespace="karajan" key="initialScore">10000</profile><BR>
</pool><BR>
<BR>
<pool handle="condor"><BR>
<execution provider="condor" url="none"/><BR>
<gridftp url="local://localhost"/><BR>
<workdirectory>/home/erin/swiftwork</workdirectory><BR>
<profile namespace="karajan" key="jobThrottle">.19</profile><BR>
<profile namespace="karajan" key="initialScore">10000</profile><BR>
</pool><BR>
<BR>
</config><BR>
<BR>
The jobThrottle values above will enable Swift to run up to 4 jobs at a<BR>
time on localhost and 20 jobs at a time on the Condor cluster.<BR>
<BR>
Use tc.data to catalog applications on pool or the other.<BR>
<BR>
Set jobThrottle as desired to control execution parallelism.<BR>
<BR>
#jobs run in parallel is (jobThrottle * 100)+1<BR>
<BR>
initialScore=10000 overrides Swift's "start slow" approach to sensing<BR>
the site's responsiveness.<BR>
<BR>
- Mike<BR>
<BR>
On 10/21/09 3:17 AM, Hodgess, Erin wrote:<BR>
> Aha!<BR>
><BR>
> I needed the universe=vanilla line.<BR>
><BR>
><BR>
><BR>
> Erin M. Hodgess, PhD<BR>
> Associate Professor<BR>
> Department of Computer and Mathematical Sciences<BR>
> University of Houston - Downtown<BR>
> mailto: hodgesse@uhd.edu<BR>
><BR>
><BR>
><BR>
> -----Original Message-----<BR>
> From: swift-user-bounces@ci.uchicago.edu on behalf of Hodgess, Erin<BR>
> Sent: Wed 10/21/2009 3:07 AM<BR>
> To: Michael Wilde<BR>
> Cc: swift-user@ci.uchicago.edu<BR>
> Subject: RE: [Swift-user] using swift on a cluster<BR>
><BR>
> Hello!<BR>
><BR>
> We are indeed using condor.<BR>
><BR>
> I wanted to try a small test run, but am running into trouble:<BR>
><BR>
> [hodgess@grid bin]$ cat myjob.submit<BR>
> executable=/usr/bin/id<BR>
> output=results.output<BR>
> error=results.error<BR>
> log=results.log<BR>
> queue<BR>
> [hodgess@grid bin]$ condor_submit myjob.submit<BR>
> Submitting job(s).<BR>
> Logging submit event(s).<BR>
> 1 job(s) submitted to cluster 15.<BR>
> [hodgess@grid bin]$ ls results*<BR>
> results.error results.log results.output<BR>
> You have new mail in /var/spool/mail/hodgess<BR>
> [hodgess@grid bin]$ cat results.log<BR>
> 000 (015.000.000) 10/21 03:06:03 Job submitted from host:<BR>
> <192.168.1.11:46274><BR>
> ...<BR>
> 001 (015.000.000) 10/21 03:06:05 Job executing on host: <10.1.255.244:44508><BR>
> ...<BR>
> 002 (015.000.000) 10/21 03:06:05 (1) Job not properly linked for Condor.<BR>
> ...<BR>
> 009 (015.000.000) 10/21 03:06:05 Job was aborted by the user.<BR>
> ...<BR>
> [hodgess@grid bin]$<BR>
><BR>
> I'm not sure why the job is not linked.<BR>
><BR>
> Any suggestions would be much appreciated.<BR>
><BR>
> Thanks,<BR>
> Erin<BR>
><BR>
><BR>
> Erin M. Hodgess, PhD<BR>
> Associate Professor<BR>
> Department of Computer and Mathematical Sciences<BR>
> University of Houston - Downtown<BR>
> mailto: hodgesse@uhd.edu<BR>
><BR>
><BR>
><BR>
> -----Original Message-----<BR>
> From: Michael Wilde [<A HREF="mailto:wilde@mcs.anl.gov">mailto:wilde@mcs.anl.gov</A>]<BR>
> Sent: Tue 10/20/2009 10:49 PM<BR>
> To: Hodgess, Erin<BR>
> Cc: swift-user@ci.uchicago.edu<BR>
> Subject: Re: [Swift-user] using swift on a cluster<BR>
><BR>
> Hi Erin,<BR>
><BR>
> I'm assuming you meant "use Swift to run jobs on the compute nodes of<BR>
> the cluster"?<BR>
><BR>
> If so, you first need to find out what scheduler (also called "batch<BR>
> system" or "local resource manager") the cluster is running.<BR>
><BR>
> Thats typical one of these: PBS, Condor, or SGE.<BR>
><BR>
> Either ask your system administrator, or see if the "man" command or<BR>
> similar probes give you a clue:<BR>
><BR>
> Condor: condor_q -version<BR>
><BR>
> condor_q -version<BR>
> $CondorVersion: 7.2.4 Jun 16 2009 BuildID: 159529 $<BR>
> $CondorPlatform: I386-LINUX_RHEL5 $<BR>
><BR>
> PBS: man qstat:<BR>
><BR>
> qstat(1B) PBS<BR>
><BR>
> SGE: man qstat:<BR>
><BR>
> QSTAT(1) Sun Grid Engine User Commands<BR>
><BR>
><BR>
> If its PBS or Condor, then the Swift user guide gives the sites.xml<BR>
> entries to use.<BR>
><BR>
> Tell us what you find, then try following the instructions in the user<BR>
> guide, and follow up with questions as needed.<BR>
><BR>
> - Mike<BR>
><BR>
><BR>
> On 10/20/09 9:41 PM, Hodgess, Erin wrote:<BR>
> > Hi Swift Users:<BR>
> ><BR>
> > I'm on a cluster and would like to use swift on the different sites on<BR>
> > the cluster.<BR>
> ><BR>
> > How would I do that, please?<BR>
> ><BR>
> > Thanks,<BR>
> > Erin<BR>
> ><BR>
> ><BR>
> > Erin M. Hodgess, PhD<BR>
> > Associate Professor<BR>
> > Department of Computer and Mathematical Sciences<BR>
> > University of Houston - Downtown<BR>
> > mailto: hodgesse@uhd.edu<BR>
> ><BR>
> ><BR>
> > ------------------------------------------------------------------------<BR>
> ><BR>
> > _______________________________________________<BR>
> > Swift-user mailing list<BR>
> > Swift-user@ci.uchicago.edu<BR>
> > <A HREF="http://mail.ci.uchicago.edu/mailman/listinfo/swift-user">http://mail.ci.uchicago.edu/mailman/listinfo/swift-user</A><BR>
><BR>
><BR>
<BR>
</FONT>
</P>
</BODY>
</HTML>