[Swift-devel] Items needed for Grid/OSG Condor-G support

[Mihael Hategan]

On Thu, 2010-09-02 at 07:55 -0600, Michael Wilde wrote:
> Glen, following up on these:
> 
> ----- "Glen Hocky" <hockyg at gmail.com> wrote:

> > 2) tagging the jobs submitted or at least making sure they get pulled
> > out when a job fails or is canceled with the condor provider
> 
> I was going to put a "run #" tag on all condor-g jobs from a run, and
> create a script that removes all the jobs from a run (swiftrm) using
> the right condor_rm patter matching spell.

Condor has an automatic removal mode. However, it is explicitly disabled
because of the way the current condor provider figures out status: it
needs to see the job in a done or failed state. After that happens, the
job is marked as auto-remove. However if swift is killed in the middle
of the process, that doesn't happen.

Now, if the condor provider was to use log files for tracking status
instead, this would not be a problem. So I think that's the right way to
go. Otherwise we could add some JVM cleanup tasks to mark the jobs as
removable when the JVM is shut down.

Mihael