[Swift-devel] Cache error in Swift

[Michael Wilde]

Jon, can you log your mappings? Or can you verify them from the Swift logs?

- Mike

----- Original Message -----
> From: "Jonathan Monette" <jonmon at mcs.anl.gov>
> To: "Mihael Hategan" <hategan at mcs.anl.gov>
> Cc: "swift-devel Devel" <swift-devel at ci.uchicago.edu>, "Michael Wilde" <wilde at mcs.anl.gov>
> Sent: Wednesday, September 14, 2011 11:03:11 PM
> Subject: Re: Cache error in Swift
> I am pretty sure I am not. All the input files are unique and the
> output files are all mapped using the regexp mapper and appending
> proj_ to the front.
> 
> But I will check anyways. I seem to be having the error without
> replications as well. Let me try a couple more runs to see if I added
> something erroneously.
> 
> On Sep 14, 2011, at 10:43 PM, Mihael Hategan wrote:
> 
> > I suppose it might be possible to have a race condition when using
> > replication such that two jobs in the same replication group
> > complete.
> >
> > But before I go and dig into that, can you double-check that you are
> > not
> > mapping two things to the same file?
> >
> > On Wed, 2011-09-14 at 17:33 -0500, Jonathan Monette wrote:
> >> Hello,
> >>   I just ran my SwiftMontage scripts again with the most recent
> >>   build of 0.93 source. I received this error
> >> Execution failed:
> >> 	The cache already contains
> >> 	pads:montage-20110914-1717-nrothtg3/shared/proj_dir/proj_2mass-atlas-000713s-j0870197.fits.
> >>
> >> I received this error after 3652 tasks. All the files are located
> >> in /home/jonmon/PADS/Swift/SwiftMontage/big/run.0013 on the CI
> >> machines.
> >>
> >> I wanted to try replications. I have noticed in my scripts that
> >> when PADS is full of jobs Swift doesn't try to resubmit to Beagle
> >> even though Beagle shows through showq that it has room for jobs. I
> >> thought replications would use both sites more efficiently. What I
> >> mean is I thought that replications would replicate a jobs onto
> >> Beagle since PADS is taking so long just sitting in the queue.
> >>
> >> Please educate me if this is not what I should be doing and if in
> >> fact there is no work around to this problem.
> >
> >

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory