[Swift-user] Block task failed on Tukey/Cobalt

[David Kelly]

In 0.95, when a scheduler submit command fails with a non-zero exit code,
the task will fail and you should see output from the submit command. When
I try to run with an invalid project on Tukey using 0.95, it displays this:

Progress: Wed, 12 Feb 2014 21:20:59+0000
Progress: Wed, 12 Feb 2014 21:21:00+0000  Submitted:1

Could not submit job (cqsub reported an exit code of 1).
The allocation for ATPESC2013 on tukey has expired.
Projects available: ATPESC2013 ExM
For assistance, contact support at alcf.anl.gov
Filter /soft/cobalt/scripts/clusterbank-account failed
...




On Tue, Feb 11, 2014 at 4:38 PM, Ketan Maheshwari <ketan at mcs.anl.gov> wrote:

> Filed bug 1199 for Cobalt:
> https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1199
>
>
> On Tue, Feb 11, 2014 at 4:21 PM, Mihael Hategan <hategan at mcs.anl.gov>wrote:
>
>> That's for PBS which is only marginally related to the Cobalt provider.
>>
>> Mihael
>>
>> On Tue, 2014-02-11 at 12:35 -0600, Ketan Maheshwari wrote:
>> > There is bug 1185 filed already:
>> > https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1185
>> >
>> >
>> > On Tue, Feb 11, 2014 at 11:23 AM, Justin M Wozniak <wozniak at mcs.anl.gov
>> >wrote:
>> >
>> > >
>> > > Let's add a bugzilla entry to make sure this error is reported
>> clearly.
>> > >
>> > > On 02/11/2014 11:12 AM, Ketan Maheshwari wrote:
>> > >
>> > > This was fixed after changing the default project from the expired
>> atpesc
>> > > to ExM.
>> > >
>> > >  Thanks!
>> > >
>> > >
>> > > On Tue, Feb 11, 2014 at 10:51 AM, David Kelly <
>> davidkelly at uchicago.edu>wrote:
>> > >
>> > >>  The Cobalt provider submits jobs via the command line rather than a
>> > >> submit script, so I believe empty submit scripts are normal. From
>> the log
>> > >> it looks like jobs are getting submitted and getting a job number.
>> Are you
>> > >> running from a filesystem that is shared on worker nodes?
>> > >>
>> > >>
>> > >>  On Tue, Feb 11, 2014 at 10:12 AM, Ketan Maheshwari <
>> ketan at mcs.anl.gov>wrote:
>> > >>
>> > >>> Swift Tutorial scripts worked on Tukey as of last August during the
>> > >>> ATPESC tutorials.
>> > >>>
>> > >>>  I find that empty Cobalt submit script files are created.
>> > >>>
>> > >>>
>> > >>> On Tue, Feb 11, 2014 at 9:27 AM, Justin M Wozniak <
>> wozniak at mcs.anl.gov>wrote:
>> > >>>
>> > >>>>
>> > >>>> Has anyone else run successfully on Tukey?
>> > >>>>
>> > >>>> Can you inspect/post the Swift-generated Cobalt submit script? You
>> may
>> > >>>> also want to inspect/post the Cobalt-generated
>> > >>>> *.submit.stdout/*.submit.stderr logs.
>> > >>>>
>> > >>>> On 02/10/2014 08:23 PM, Ketan Maheshwari wrote:
>> > >>>>
>> > >>>>  Hi,
>> > >>>>
>> > >>>>  Trying some Swift test for Swift-Galaxy demo on Tukey. Using
>> > >>>> local:cobalt coaster provider. I am getting the following error:
>> > >>>>
>> > >>>>  $ swift -sites.file sites.xml -tc.file tc -config cf script.swift
>> > >>>> Swift 0.94 swift-r6637 cog-r3742
>> > >>>>
>> > >>>>  RunID: 20140211-0216-ryo7slj7
>> > >>>> Progress:  time: Tue, 11 Feb 2014 02:16:53 +0000
>> > >>>> Progress:  time: Tue, 11 Feb 2014 02:17:23 +0000  Submitted:3
>> > >>>> Progress:  time: Tue, 11 Feb 2014 02:17:53 +0000  Submitted:3
>> > >>>> Progress:  time: Tue, 11 Feb 2014 02:17:58 +0000  Submitted:2
>>  Active:1
>> > >>>> Execution failed:
>> > >>>>  Exception in sh:
>> > >>>>     Arguments:
>> > >>>>
>> [gpfs/mira-home/ketan/galaxy-dist/database/files/000/dataset_1.dat, 0]
>> > >>>>     Host: tukey
>> > >>>>     Directory: script-20140211-0216-ryo7slj7/jobs/u/sh-uv25u9ml
>> > >>>> Caused by:
>> > >>>>  Block task failed:
>> > >>>>
>> > >>>>
>> > >>>>  anapp, script.swift, line 13
>> > >>>>
>> > >>>>  Tried many different options in swift.properties to no avail.
>> > >>>>
>> > >>>>  Also tried the ATPESC 2013 tutorial setup but scripts fail with
>> same
>> > >>>> pattern/error messages.
>> > >>>>
>> > >>>>  Attaching the tarball with config, sites file.
>> > >>>>
>> > >>>>  Thanks for any suggestions.
>> > >>>>
>> > >>>>  Ketan
>> > >>>>
>> > >>>>
>> > >>>>   _______________________________________________
>> > >>>> Swift-user mailing listSwift-user at ci.uchicago.eduhttps://
>> lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Justin M Wozniak
>> > >>>>
>> > >>>>
>> > >>>> _______________________________________________
>> > >>>> Swift-user mailing list
>> > >>>> Swift-user at ci.uchicago.edu
>> > >>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> > >>>>
>> > >>>
>> > >>>
>> > >>> _______________________________________________
>> > >>> Swift-user mailing list
>> > >>> Swift-user at ci.uchicago.edu
>> > >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> > >>>
>> > >>
>> > >>
>> > >> _______________________________________________
>> > >> Swift-user mailing list
>> > >> Swift-user at ci.uchicago.edu
>> > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> > >>
>> > >
>> > >
>> > >
>> > > _______________________________________________
>> > > Swift-user mailing listSwift-user at ci.uchicago.eduhttps://
>> lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> > >
>> > >
>> > >
>> > > --
>> > > Justin M Wozniak
>> > >
>> > >
>> > > _______________________________________________
>> > > Swift-user mailing list
>> > > Swift-user at ci.uchicago.edu
>> > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>> > >
>> > _______________________________________________
>> > Swift-user mailing list
>> > Swift-user at ci.uchicago.edu
>> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>>
>>
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>>
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20140212/9c4cb4db/attachment.html>