[Swift-user] how to run swift script against a local cluster

[Wilde, Michael J.]

Hi Jiao,

While I look into the instructions you followed from the blog post below, could you try the tutorial examples at http://swift-lang.org/docs? We try to keep these examples up to date and tested. They should either work on your cluster or be readily adaptable to it.

To help us resolve your problem, please send the full log file (scriptname-date-*.log) to swift-support at ci.uchicago.edu and describe your cluster to us (scheduler, mounted filesystems, etc).

One possible issue I see in the blog post is this line in the example sites.xml file:

   <workdirectory>/scratch/{env.USER}/.swiftwork</workdirectory>

That directory needs to be shared (i.e. mounted) on both the host on which your are running the swift comment (e.g. the login host) as well as on every cluster node on which a Swift app might run. The ssl-cl provider should automatically generate internal certificates for you (no separate Globus install needed), but perhaps they cant be accessed by the worker nodes.

While you debug this, there's also a new, un-announced tutorial facility at http://swift-lang.org/tryswift which lets you try tutorial examples from the web on cloud nodes, and lets you experiment with small Swift scripts with no others setup.

- Mike
--
Michael Wilde
Mathematics and Computer Science          Computation Institute
Argonne National Laboratory                    The University of Chicago

________________________________
From: swift-user-bounces at ci.uchicago.edu [swift-user-bounces at ci.uchicago.edu] on behalf of Jiao Lin [linjiao at caltech.edu]
Sent: Sunday, February 02, 2014 1:50 PM
To: swift-user at ci.uchicago.edu
Subject: [Swift-user] how to run swift script against a local cluster

Hello,

We just learned about swift and are trying to use swift to run computations on a local cluster. I was trying to follow the note at http://rmcgibbo.github.io/blog/2013/06/03/setting-up-swift/

When trying to run a simple echo command at a local cluster using coaster provider, I got an error:

$ swift echo.swift
Swift 0.94.1 swift-r7114 cog-r3803

RunID: 20140202-1140-yyytkso6
Progress:  time: Sun, 02 Feb 2014 11:40:58 -0800
Execution failed:
        Exception in echo:
    Arguments: [hello]
    Host: fram
    Directory: echo-20140202-1140-yyytkso6/jobs/0/echo-0dqo8vll

Caused by:
        Could not submit job
Caused by:
        Could not start coaster service
Caused by:
        java.lang.NullPointerException
        at org.globus.cog.abstraction.impl.execution.coaster.AutoCA.ensureCACertsExist(AutoCA.java:143)
        at org.globus.cog.abstraction.impl.execution.coaster.AutoCA.createProxy(AutoCA.java:128)
        at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.setupGSIProxy(ServiceManager.java:238)
        at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.startService(ServiceManager.java:194)
        at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.reserveService(ServiceManager.java:132)
        at org.globus.cog.abstraction.impl.execution.coaster.ServiceManager.reserveService(ServiceManager.java:151)
        at org.globus.cog.abstraction.impl.execution.coaster.JobSubmissionTaskHandler.getChannel(JobSubmissionTaskHandler.java:119)
        at org.globus.cog.abstraction.impl.execution.coaster.JobSubmissionTaskHandler.submit(JobSubmissionTaskHandler.java:105)
        at org.globus.cog.abstraction.impl.common.AbstractTaskHandler.submit(AbstractTaskHandler.java:45)
        at org.globus.cog.karajan.scheduler.submitQueue.NonBlockingSubmit.run(NonBlockingSubmit.java:97)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)

        echo, echo.swift, line 11

It seems it failed when globus is trying to get certificates? Is it necessary to install some kind of globus service on the cluster?

I wonder what kind of requirements are there for the cluster? The cluster has java 1.7 installed:

java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

Your help is much appreciated.

Jiao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20140202/73379ce5/attachment.html>