[Swift-devel] trunk-cobalt block task ended prematurely

[Ketan Maheshwari]

Slow network looks unlikely to be a cause:

I tried with 1 app call, total I/O size less than 20KB and a job wall-time
of 40 minutes. I still see the hang. The output files produced by the app
do end up in the outdir.

Another observation is that despite 40 minutes of walltime, the application
crashes in 2 minutes with a message saying walltime exceeded, as follows:


exception @ swift-int-staging.k, line: 160
Caused by: Walltime exceeded

k:assign @ swift.k, line: 174
Caused by: Exception in bgsh:
    Arguments: [/home/ketan/SwiftApps/subjobs/mpicatsnsleep/mpicatnap,
/gpfs/mira-home/ketan/SwiftApps/subjobs/mpicatsnsleep/./data.txt,
/gpfs/mira-home/ketan/SwiftApps/subjobs/mpicatsnsleep/./outdir/f.0001.out,
1]
    Host: cluster
    Directory: catsnsleepmpi-run001/jobs/b/bgsh-k7exhe5m
exception @ swift-int-staging.k, line: 165

--Ketan


On Tue, Mar 3, 2015 at 2:51 PM, Hategan-Marandiuc, Philip M. <
hategan at mcs.anl.gov> wrote:

> With direct "staging" and a slow network FS, the application run time
> will go up. This is why in many cases "avoid NFS/gpfs" is a good
> strategy.
>
> What happens if you increase the walltime for your jobs?
>
> Mihael
>
> On Tue, 2015-03-03 at 14:01 -0600, Ketan Maheshwari wrote:
> > Hi,
> >
> > Continuing the discussion on devel. It seems that the run worked after I
> > changed the staging method from "direct" to "swift".
> >
> > I am trying to narrow down the cause why "direct" staging does not work.
> > Any pointers to possible causes will help.
> >
> > Thanks,
> > Ketan
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20150303/677036b6/attachment.html>