[Swift-user] Sites configuration examples

[Yadu Nand Babuji]

Hi Aurelien,

We are in between efforts to improve the swift documentation set, 
including those for the site-configuration.
The latest tutorial documentation covers configuration in limited 
detail, but comes with configurations
for several sites which you could use as reference.
http://swift-lang.org/swift-tutorial/doc/tutorial.html

In the meanwhile, we can help with any questions you might have over the 
swift-user list.

Here are some of the configs used for the nightly tests that could also 
be useful:
a) Run remotely via ssh:
https://github.com/swift-lang/swift-k/blob/master/tests/sites/bridled/swift.conf
https://github.com/swift-lang/swift-k/blob/master/tests/sites/mcs/swift.conf

b) Run remotely on a cluster that uses the Slurm scheduler:
https://github.com/swift-lang/swift-k/blob/master/tests/sites/midway/swift.conf
https://github.com/swift-lang/swift-k/blob/master/tests/sites/stampede/swift.conf

c) Run remotely on a system that uses the Condor scheduler:
https://github.com/swift-lang/swift-k/blob/master/tests/sites/osgconnect/swift.conf

If you do have configurations in the older *xml formats, you could use 
swift-convert-config.

The sites.<site_name>.execution.jobManager property has a colon 
separated value as that
was simply inherited from the older style. It follows this format:
<Method to connect to remote site> : <Scheduler option to use in order 
to start workers>
This is true if you are using execution type "coasters" as will be the 
case for most workflows.

local:local  -> Run on the local machine directly. (uses fork to start 
workers)
local:slurm -> Run on the local machine,and use slurm scheduler 
directives to start workers
ssh-cl:slurm -> Use commandline ssh to connect to the provided URL, and 
use slurm
                         directives on that machine to start workers.

I do not think we have templates for the configurations you have asked 
for, but that is
something that could be done. The closest we have are the configs that 
come with the
tutorial and the configs I've linked to from the test-suite.

Thanks,
Yadu
On 07/13/2015 11:52 AM, Aurélien Mazurie wrote:
> Dear all,
> I'm still struggling with the existing Swift documentation to understand how to deploy a Swift-based test workflow outside of the desktop computer it currently (successfully) runs in.
>
> What I'm trying to achieve is a small series of test cases:
> (a) execution of the same workflow on a remote machine to which I have SSH access
> (b) execution of the same workflow on a SLURM-powered compute cluster
> (c) execution of the same workflow on a HTCondor compute cluster (EC2-based, and created with StarCluster)
>
> My trouble comes from the fact that most of the material I can find on the Net is pre-0.96, and refers to XML description of site configuration that I haven't successfully converted to the new aggregative, vaguely JSON-like configuration format (which is very nice, by the way; I wish other toolkits I use could have the same).
>
> I'm also confused by some snippets having compound values for the "site.<site_name>.execution.jobManager" property, such as "ssh-cl:local". The documentation is not very clear on these cases, nor how this variable relates to "site.<site_name>.execution.type".
>
> Is there such a thing as a template site configuration for the test cases listed above?
>
> Best,
> Aurélien
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user