[Swift-user] Problem with running swift and condor

[Ben Clifford]

On Thu, 2 Jul 2009, Hodgess, Erin wrote:

> In the sites file, I am referring to an OSG site to be run on condor 
> rather than gram. Here is the sites file: <config>

You are specifying a jobmanager and an execution provider - you can only 
specify one (although swift does not properly detect if you do both).

The jobmanager element means to use plain GRAM which is conficting with 
your additional specification of condor. Use only one.

>     <jobmanager universe="vanilla" url="osgce.cs.clemson.edu:2119/jobmanager-condor" major="2" />

>  <execution provider="condor" />
>  <profile namespace="globus" key="gridResource">gt2 osgce.cs.clemson.edu/jobmanager-condor</profile>



> 
>     <workdirectory >/home/osg</workdirectory>
>   </pool>
> 
> </config>
> 
> The swift program is very simple; running a word count on a file:
> [erin at communicado swift1]$ cat wc1a.swift
> 
> type file;  
>   
> app (file o) count(file i) {  
>   wc @i stdout=@o;  
> }  
>   
> file i <"input.txt">;  
> 
> foreach j in [1:15]
> 	{
> file o <single_file_mapper;file=@strcat("results/",j,".out")>;
> 	(o)=count(i);
> 	}
> 
> 
> But when I run the swift job, I get a "parameter not supported".  Note: this job works fine on localhost, teraport, and on several OSG sites when set up using gram.
> 
> 
> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file sites1i.xml wc1a.swift
> Swift svn swift-r2950 cog-r2406
> 
> RunID: 20090702-2009-sk2d7m89
> Progress:
> Progress:  Selecting site:13  Initializing site shared directory:1  Stage in:1
> Progress:  Selecting site:13  Submitting:1  Submitted:1
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/8 on Clemson-ciTeam
> Progress:  Selecting site:12  Stage in:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 on Clemson-ciTeam
> Progress:  Selecting site:12  Submitted:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b on Clemson-ciTeam
> Progress:  Selecting site:11  Stage in:1 Failed but can retry:3
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/e on Clemson-ciTeam
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:10  Stage in:1 Failed but can retry:4
> Progress:  Selecting site:10  Submitted:1 Failed but can retry:4
> Progress:  Selecting site:10 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/g on Clemson-ciTeam
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:9  Stage in:1 Failed but can retry:5
> Progress:  Selecting site:9  Submitted:1 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/i on Clemson-ciTeam
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:8  Stage in:1 Failed but can retry:6
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/k on Clemson-ciTeam
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:7  Stage in:1 Failed but can retry:7
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/m on Clemson-ciTeam
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:6  Stage in:1 Failed but can retry:8
> Progress:  Selecting site:6  Submitted:1 Failed but can retry:8
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/o on Clemson-ciTeam
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:5  Stage in:1 Failed but can retry:9
> Progress:  Selecting site:5  Submitted:1 Failed but can retry:9
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/q on Clemson-ciTeam
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:4  Stage in:1 Failed but can retry:10
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/s on Clemson-ciTeam
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:3  Stage in:1 Failed but can retry:11
> Progress:  Selecting site:3  Submitted:1 Failed but can retry:11
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/u on Clemson-ciTeam
> Progress:  Selecting site:3 Failed but can retry:12
> 
> 
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:2  Stage in:1 Failed but can retry:12
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/x on Clemson-ciTeam
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:1  Stage in:1 Failed but can retry:13
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/z on Clemson-ciTeam
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/1 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/3 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/5 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/7 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> 
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/d on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/f on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/h on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/j on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/l on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/n on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/p on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/r on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/t on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/v on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/y on Clemson-ciTeam
> Progress:  Failed:1 Failed but can retry:14
> Execution failed:
> 	Exception in wc:
> Arguments: [input.txt]
> Host: Clemson-ciTeam
> Directory: wc1a-20090702-2009-sk2d7m89/jobs/y/wc-y2zfj4dj
> stderr.txt: 
> 
> stdout.txt: 
> 
> ----
> 
> Caused by:
> 	Cannot submit job
> Caused by:
> 	Parameter not supported
> [erin at communicado swift1]$ 
> 
> Thank you for your kind attention.
> 
> Erin
> 
> 
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
> 
>