[Swift-devel] proper way to run swift on resource using condor?

[Michael Wilde]

Hi Mike,

Let me try to clarify.

I think you used scripts to generate Condor submit files when you used 
the Purdue site without Swift.

Using GRAM, you could have written scripts that were independent of the 
site scheduler: they would work work on Condor, PBS, LSF etc. Thats the 
purpose of GRAM, to provide a scheduler-independent means of submitting 
jobs.

One can use GRAM through Condor-G as well: you write submit files, 
Condor sends jobs to GRAM, and GRAM sends them to the local scheduler.

Swift talks (mainly) to GRAM, so when you use Swift, you're also 
scheduler-independent. And in most cases, thats the way we suggest 
running Swift.

- Mike


On 5/20/08 9:31 PM, Ian Foster wrote:
> Again, why don't you use a GRAM submit as you would at any other TG site?
> 
> 
> On May 20, 2008, at 9:23 PM, Mike Kubal wrote:
> 
>> I've been having swift invoke a perl script on Purdue.
>> The perl script writes the the descriptor file that is
>> passed as an argument to the condor_submit command
>> within the perl script. The condor_submit command
>> actually launches the job on a compute node. Every 30
>> seconds the perl script checks to see if the launched
>> job has finished. When the launched job finishes the
>> perl script ends, and the result files are returned
>> via swift.  The problem with this approach is that I
>> am either piling many perl scripts onto the logon node
>> or wasting a compute node to just check the status of
>> the job.
>>
>> I have experimented with having swift call
>> condor_submit directly with the idea of having a
>> single separate swift function check the status of all
>> jobs and returned the result files when finished. The
>> problem here is that once the the job has been
>> launched successfully, swift removes the  job
>> sub-directory where the results of the job launched
>> via condor_submit are written.
>>
>> Suggestions welcome,
>>
>> Mike
>>
>> --- Mihael Hategan <hategan at mcs.anl.gov> wrote:
>>
>>> Are you referring to the arguments-with-spaces
>>> issue?
>>>
>>> On Tue, 2008-05-20 at 14:46 -0700, Mike Kubal wrote:
>>>> I have been using some 'work around' methods to
>>> use
>>>> Swift to run jobs on  the Purdue resource that
>>> uses
>>>> Condor. Is there a proper way?
>>>>
>>>> Thanks,
>>>>
>>>> Mike
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Swift-devel mailing list
>>>> Swift-devel at ci.uchicago.edu
>>>>
>>>
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>>
>>>
>>
>>
>>
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel