[Swift-user] Setting up Swift at Stanford

[Mihael Hategan]

Unfortunately the pbs provider only works locally and the ssh provider
only runs remote ssh commands through bash (not queuing things through
PBS). I can see the benefit of tunneling what the pbs provider does
through ssh, but that's not there now.

Your choices are to either install Globus on your clusters or use
coasters which install their own service in the background on the
cluster. For the latter you need to say:
<execution provider="coaster" jobmanager="ssh:pbs"
url="vsp-compute-01.stanford.edu"/>

Mihael

On Mon, 2013-06-03 at 17:16 -0700, Robert McGibbon wrote:
> Hey,
> 
> We just heard about the swift project from some colleagues at U Chicago, and we're interested in trying it out with some of our compute resources at Stanford to run parallel molecular dynamics  and x-ray scatting simulations. Currently, I'm most interested in setting up the environment such that I can submit my swift script on a local workstation, with execution on a few different clusters. The head nodes of our local clusters are accessible via ssh, and then job execution is scheduled with pbs.
> 
> When I run swift, it can't seem to find qsub on the cluster.
> 
> rmcgibbo at Roberts-MacBook-Pro-2 ~/projects/swift
> $ swift -sites.file sites.xml hello.swift -tc.file tc.data
> Swift 0.94 swift-r6492 cog-r3658
> 
> RunID: 20130603-1704-5xii8svc
> Progress:  time: Mon, 03 Jun 2013 17:04:10 -0700
> 2013-06-03 17:04:10.735 java[77051:1f07] Loading Maximizer into bundle: com.apple.javajdk16.cmd
> 2013-06-03 17:04:11.410 java[77051:1f07] Maximizer: Unsupported window created of class: CocoaAppWindow
> Progress:  time: Mon, 03 Jun 2013 17:04:13 -0700  Stage in:1
> Execution failed:
> 	Exception in uname:
>    Arguments: [-a]
>    Host: vsp-compute
>    Directory: hello-20130603-1704-5xii8svc/jobs/y/uname-ydyn5fal
> Caused by:
> 	Cannot submit job: Cannot run program "qsub": error=2, No such file or directory
> 	uname, hello.swift, line 8
> 
> When I switch the execution provider from pbs to ssh, the hob runs successfully, but only on the head node of the vsp-compute cluster. I'd like to run instead using the cluster's pbs queue. Any help would be greatly appreciated.
> 
> -Robert
> Graduate Student, Pande Lab
> Stanford University, Department of Chemistry
> 
> p.s.
> 
> My sitess.xml file is 
> ```
> <config>
> <pool handle="vsp-compute">
>  <filesystem provider="ssh" url="vsp-compute-01.stanford.edu"/>
>  <execution provider="pbs" jobmanager="ssh:pbs" url="vsp-compute-01.stanford.edu"/>
> 
>  <profile namespace="globus" key="maxtime">750</profile>
>  <profile namespace="globus" key="jobsPerNode">1</profile>
>  <profile namespace="globus" key="queue">default</profile>
>  <profile namespace="swift"   key="stagingMethod">file</profile>
> 
>  <workdirectory>/scratch/rmcgibbo/swiftwork</workdirectory>  
> </pool>
> 
> <!-- End -->
> </config>
> ```
> 
> My SwiftScript is
> ```
> #hello.swift
> type file;
> 
> app (file o) uname() {
>  uname "-a" stdout=@o;
> }
> file outfile <"uname.txt">;
> 
> outfile = uname();
> ```
> Hey,
> 
> We just heard about the swift project from some colleagues at U
> Chicago, and we're interested in trying it out with some of our
> compute resources at Stanford to run parallel molecular dynamics  and
> x-ray scatting simulations. Currently, I'm most interested in setting
> up the environment such that I can submit my swift script on a local
> workstation, with execution on a few different clusters. The head
> nodes of our local clusters are accessible via ssh, and then job
> execution is scheduled with pbs.
> 
> When I run swift, it can't seem to find qsub on the cluster.
> 
> rmcgibbo at Roberts-MacBook-Pro-2 ~/projects/swift
> $ swift -sites.file sites.xml hello.swift -tc.file tc.data
> Swift 0.94 swift-r6492 cog-r3658
> 
> RunID: 20130603-1704-5xii8svc
> Progress:  time: Mon, 03 Jun 2013 17:04:10 -0700
> 2013-06-03 17:04:10.735 java[77051:1f07] Loading Maximizer into
> bundle: com.apple.javajdk16.cmd
> 2013-06-03 17:04:11.410 java[77051:1f07] Maximizer: Unsupported window
> created of class: CocoaAppWindow
> Progress:  time: Mon, 03 Jun 2013 17:04:13 -0700  Stage in:1
> Execution failed:
> Exception in uname:
>    Arguments: [-a]
>    Host: vsp-compute
>    Directory: hello-20130603-1704-5xii8svc/jobs/y/uname-ydyn5fal
> Caused by:
> Cannot submit job: Cannot run program "qsub": error=2, No such file or
> directory
> uname, hello.swift, line 8
> 
> When I switch the execution provider from pbs to ssh, the hob runs
> successfully, but only on the head node of the vsp-compute cluster.
> I'd like to run instead using the cluster's pbs queue. Any help would
> be greatly appreciated.
> 
> -Robert
> Graduate Student, Pande Lab
> Stanford University, Department of Chemistry
> 
> p.s.
> 
> My sitess.xml file is 
> ```
> <config>
> <pool handle="vsp-compute">
>  <filesystem provider="ssh" url="vsp-compute-01.stanford.edu"/>
>  <execution provider="pbs" jobmanager="ssh:pbs"
> url="vsp-compute-01.stanford.edu"/>
> 
>  <profile namespace="globus" key="maxtime">750</profile>
>  <profile namespace="globus" key="jobsPerNode">1</profile>
>  <profile namespace="globus" key="queue">default</profile>
>  <profile namespace="swift"   key="stagingMethod">file</profile>
> 
>  <workdirectory>/scratch/rmcgibbo/swiftwork</workdirectory>  
> </pool>
> 
> <!-- End -->
> </config>
> ```
> 
> My SwiftScript is
> ```
> #hello.swift
> type file;
> 
> app (file o) uname() {
>  uname "-a" stdout=@o;
> }
> file outfile <"uname.txt">;
> 
> outfile = uname();
> ```
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user