From HodgessE at uhd.edu  Wed Jul  1 13:40:24 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Wed, 1 Jul 2009 13:40:24 -0500
Subject: [Swift-user] Using multiple sites
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C37148@BALI.uhd.campus>

Hi Swift Users:

When your sites.xml file has multiple sites, you can see in the log file which jobs went to which sites.  Okay.

Now, is there a way to control which jobs go to which sites, please?

Thanks,
Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090701/511044b5/attachment.html>

From tiberius at ci.uchicago.edu  Wed Jul  1 13:46:52 2009
From: tiberius at ci.uchicago.edu (Tiberiu Stef-Praun)
Date: Wed, 1 Jul 2009 13:46:52 -0500
Subject: [Swift-user] Using multiple sites
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C37148@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37148@BALI.uhd.campus>
Message-ID: <fec1351f0907011146n38883f7fua53d71fe506f7d8a@mail.gmail.com>

In tc.data, specify for each job, only the site entries that you want
to have that job sent to.


e.g

teraport executeJob /bin/execute bla bla

localhost mergeJob /home/user/merge.sh


On Wed, Jul 1, 2009 at 1:40 PM, Hodgess, Erin<HodgessE at uhd.edu> wrote:
> Hi Swift Users:
>
> When your sites.xml file has multiple sites, you can see in the log file
> which jobs went to which sites.? Okay.
>
> Now, is there a way to control which jobs go to which sites, please?
>
> Thanks,
> Erin
>
>
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
>
>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
>
>



-- 
Tiberiu (Tibi) Stef-Praun, PhD
Computational Sciences Researcher
Computation Institute
5640 S. Ellis Ave, #405
University of Chicago
http://www-unix.mcs.anl.gov/~tiberius/


From HodgessE at uhd.edu  Wed Jul  1 16:17:30 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Wed, 1 Jul 2009 16:17:30 -0500
Subject: [Swift-user] running a test in R on Clemson-ciTeam
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C37150@BALI.uhd.campus>

I'm trying to run a test in Swift using R that runs fine on localhost and teraport.

When I switched to Clemson-ciTeam, this is what I got:

$ swift -tc.file tc.data -sites.file sites1g.xml fun3.swift
Swift svn swift-r2950 cog-r2406

RunID: 20090701-1614-abynf6p5
Progress:
Progress:  Selecting site:8  Initializing site shared directory:1  Stage in:1
Progress:  Selecting site:8  Submitting:1  Submitted:1
Progress:  Selecting site:8  Active:1  Checking status:1
Progress:  Selecting site:8 Failed but can retry:2
Progress:  Selecting site:7  Stage in:1 Failed but can retry:2
Progress:  Selecting site:6  Submitted:2 Failed but can retry:2
Progress:  Selecting site:6  Submitted:2 Failed but can retry:2
[erin at tp-login2 bin]$ 

Here is the sites1g.xml file:
$ cat sites1g.xml
<config>

  <pool handle="Clemson-ciTeam" >
    <gridftp  url="gsiftp://osgce.cs.clemson.edu/" />
    <jobmanager universe="vanilla" url="osgce.cs.clemson.edu:2119/jobmanager-condor" major="2" />
    <workdirectory >/home/osg/swiftwork</workdirectory>
  </pool>



</config>

And here is tc.data
[erin at tp-login2 bin]$ cat tc.data
#NOTE WELL: fields in this file must be separated by tabs, not spaces
#           and there must be no trailing whitespace at the end of each line.
#
# sitename      app     pathname        (ignored)       (ignored)       profiles
localhost       echo    /bin/echo       INSTALLED       INTEL32::LINUX  null
teraport        echo    /bin/echo       INSTALLED       INTEL32::LINUX  null
localhost	translate	/usr/bin/tr	INSTALLED	INTEL32::LINUX	null
localhost	R	/home/erin/R-2.9.0/bin/R	INSTALLED	INTEL32::LINUX	null
localhost       wc	/usr/bin/wc     INSTALLED       INTEL32::LINUX	null
localhost       convert	/usr/bin/convert	INSTALLED	INTEL32::LINUX	null
localhost	RInvoke	/home/erin/R-2.9.0/bin/RInvoke.sh	INSTALLED	INTEL32::LINUX	null
teraport	RInvoke /home/erin/R-2.9.0/bin/RInvoke.sh       INSTALLED	INTEL32::LINUX  null
localhost	RPermInv1	/home/erin/R-2.9.0/bin/RPermInv1.sh	INSTALLED	INTEL32::LINUX	null
localhost	RInvoke1	/home/erin/R-2.9.0/bin/RInvoke1.sh	INSTALLED	INTEL32::LINUX	null
Clemson-ciTeam	RInvoke	/home/erin/R-2.9.0/bin/RInvoke.sh	INSTALLED	INTEL32::LINUX	null
[erin at tp-login2 bin]$ 

Does anyone have any suggestions, please?

Thanks,
Erin



Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090701/3c98b841/attachment.html>

From HodgessE at uhd.edu  Wed Jul  1 16:18:43 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Wed, 1 Jul 2009 16:18:43 -0500
Subject: [Swift-user] info on Throttle
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C37151@BALI.uhd.campus>

Hi again!

Where do you get info on using Throttle please?

thanks,
Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090701/63900f29/attachment.html>

From wilde at mcs.anl.gov  Wed Jul  1 17:07:16 2009
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Wed, 01 Jul 2009 17:07:16 -0500
Subject: [Swift-user] info on Throttle
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C37151@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37151@BALI.uhd.campus>
Message-ID: <4A4BDE14.10207@mcs.anl.gov>

The starting place is to read the throttle property descriptions in the 
users guide, and in the default properties file in the swift etc/ dir.

Then you can read the swift-devel email archives, but unfortunately 
these are not easily searchable.

There are some papers I or others need to send you - from Xi Li, 
Zhengxiong Hou, and maybe others.

In the meantime maybe others on the list can provide initial suggestions.

For now, just use the default values, observe the behavior, and write to 
the list regarding behavior you feel is unexpected or poor.

- Mike

On 7/1/09 4:18 PM, Hodgess, Erin wrote:
> Hi again!
> 
> Where do you get info on using Throttle please?
> 
> thanks,
> Erin
> 
> 
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user


From HodgessE at uhd.edu  Thu Jul  2 09:32:25 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 09:32:25 -0500
Subject: [Swift-user] setting throttle options
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>

Hi all!

I'm trying to experiment with the throttle option but am having trouble setting them.  I tried the following (and several variations), but here is what I receive:

swift -tc.file tc.test.data -sites.file sites1g.xml throttle.host.submit 4 wc1a.swift
SwiftScript program does not exist: throttle.host.submit

Any help much appreciated.

Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/c8944eea/attachment.html>

From HodgessE at uhd.edu  Thu Jul  2 10:16:30 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 10:16:30 -0500
Subject: [Swift-user] setting throttle options
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715A@BALI.uhd.campus>

I found it...it's in a file called swift.properties in the etc directory.

Thanks though!


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu



-----Original Message-----
From: swift-user-bounces at ci.uchicago.edu on behalf of Hodgess, Erin
Sent: Thu 7/2/2009 9:32 AM
To: swift-user at ci.uchicago.edu
Subject: [Swift-user] setting throttle options
 
Hi all!

I'm trying to experiment with the throttle option but am having trouble setting them.  I tried the following (and several variations), but here is what I receive:

swift -tc.file tc.test.data -sites.file sites1g.xml throttle.host.submit 4 wc1a.swift
SwiftScript program does not exist: throttle.host.submit

Any help much appreciated.

Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/f55227ae/attachment.html>

From hategan at mcs.anl.gov  Thu Jul  2 10:20:17 2009
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Thu, 02 Jul 2009 10:20:17 -0500
Subject: [Swift-user] setting throttle options
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>
Message-ID: <1246548017.28190.24.camel@localhost>

On Thu, 2009-07-02 at 09:32 -0500, Hodgess, Erin wrote:
> Hi all!
> 
> I'm trying to experiment with the throttle option but am having
> trouble setting them.

Please don't experiment with explosives unless in a tightly controlled
environment. They have the potential to blow things up.

May I ask what prompted you to want to try to experiment with the
throttle options?

>   I tried the following (and several variations), but here is what I
> receive:
> 
> swift -tc.file tc.test.data -sites.file sites1g.xml
> throttle.host.submit 4 wc1a.swift
> SwiftScript program does not exist: throttle.host.submit

swift -tc.file tc.test.data -sites.file sites1g.xml
-throttle.host.submit 4 wcla.swift

(note the hyphen before the option).




From HodgessE at uhd.edu  Thu Jul  2 10:24:44 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 10:24:44 -0500
Subject: [Swift-user] setting throttle options
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>
	<1246548017.28190.24.camel@localhost>
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715B@BALI.uhd.campus>

We are interested in timing and "who gets what jobs", etc.



Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu



-----Original Message-----
From: Mihael Hategan [mailto:hategan at mcs.anl.gov]
Sent: Thu 7/2/2009 10:20 AM
To: Hodgess, Erin
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] setting throttle options
 
On Thu, 2009-07-02 at 09:32 -0500, Hodgess, Erin wrote:
> Hi all!
> 
> I'm trying to experiment with the throttle option but am having
> trouble setting them.

Please don't experiment with explosives unless in a tightly controlled
environment. They have the potential to blow things up.

May I ask what prompted you to want to try to experiment with the
throttle options?

>   I tried the following (and several variations), but here is what I
> receive:
> 
> swift -tc.file tc.test.data -sites.file sites1g.xml
> throttle.host.submit 4 wc1a.swift
> SwiftScript program does not exist: throttle.host.submit

swift -tc.file tc.test.data -sites.file sites1g.xml
-throttle.host.submit 4 wcla.swift

(note the hyphen before the option).



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/4c3295f3/attachment.html>

From HodgessE at uhd.edu  Thu Jul  2 10:27:31 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 10:27:31 -0500
Subject: [Swift-user] setting throttle options
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>
	<1246548017.28190.24.camel@localhost>
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715C@BALI.uhd.campus>

Here what I got with the dash

[erin at tp-login2 swift1]$ swift -tc.file tc.test.data -sites.file sites1g.xml -throttle.host.submit 4 wc1a.swift
SwiftScript program does not exist: -throttle.host.submit

For usage information:  swift -help

[erin at tp-login2 swift1]$ 


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu



-----Original Message-----
From: Mihael Hategan [mailto:hategan at mcs.anl.gov]
Sent: Thu 7/2/2009 10:20 AM
To: Hodgess, Erin
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] setting throttle options
 
On Thu, 2009-07-02 at 09:32 -0500, Hodgess, Erin wrote:
> Hi all!
> 
> I'm trying to experiment with the throttle option but am having
> trouble setting them.

Please don't experiment with explosives unless in a tightly controlled
environment. They have the potential to blow things up.

May I ask what prompted you to want to try to experiment with the
throttle options?

>   I tried the following (and several variations), but here is what I
> receive:
> 
> swift -tc.file tc.test.data -sites.file sites1g.xml
> throttle.host.submit 4 wc1a.swift
> SwiftScript program does not exist: throttle.host.submit

swift -tc.file tc.test.data -sites.file sites1g.xml
-throttle.host.submit 4 wcla.swift

(note the hyphen before the option).



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/3062d2c0/attachment.html>

From HodgessE at uhd.edu  Thu Jul  2 10:32:30 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 10:32:30 -0500
Subject: [Swift-user] error message question
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715D@BALI.uhd.campus>

Hi all.

This might seem very obvious, but here goes anyway:  The message "Failed but can retry"

Does that mean it failed on a particular node?  Or something else, please?

Thanks,
Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/a0b2f8ab/attachment.html>

From hategan at mcs.anl.gov  Thu Jul  2 10:36:55 2009
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Thu, 02 Jul 2009 10:36:55 -0500
Subject: [Swift-user] setting throttle options
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715B@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>
	<1246548017.28190.24.camel@localhost>
	<70A5AC06FDB5E54482D19E1C04CDFCF307C3715B@BALI.uhd.campus>
Message-ID: <1246549015.28742.13.camel@localhost>

On Thu, 2009-07-02 at 10:24 -0500, Hodgess, Erin wrote:
> We are interested in timing and "who gets what jobs", etc.

Very odd. It's a rather obscure and difficult issue. Yet, for some
strange reason, whenever there is a student or a new person temporarily
on the project, they somehow jump exactly to that. And the results have
so far invariably been lacking.




From hategan at mcs.anl.gov  Thu Jul  2 10:37:47 2009
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Thu, 02 Jul 2009 10:37:47 -0500
Subject: [Swift-user] setting throttle options
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715C@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus>
	<1246548017.28190.24.camel@localhost>
	<70A5AC06FDB5E54482D19E1C04CDFCF307C3715C@BALI.uhd.campus>
Message-ID: <1246549067.28742.14.camel@localhost>

On Thu, 2009-07-02 at 10:27 -0500, Hodgess, Erin wrote:
> Here what I got with the dash
> 
> [erin at tp-login2 swift1]$ swift -tc.file tc.test.data -sites.file
> sites1g.xml -throttle.host.submit 4 wc1a.swift
> SwiftScript program does not exist: -throttle.host.submit
> 
> For usage information:  swift -help

Ah, right. That one isn't available on the command line.




From hategan at mcs.anl.gov  Thu Jul  2 10:39:54 2009
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Thu, 02 Jul 2009 10:39:54 -0500
Subject: [Swift-user] error message question
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715D@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715D@BALI.uhd.campus>
Message-ID: <1246549194.28742.17.camel@localhost>

On Thu, 2009-07-02 at 10:32 -0500, Hodgess, Erin wrote:
> Hi all.
> 
> This might seem very obvious, but here goes anyway:  The message
> "Failed but can retry"
> 
> Does that mean it failed on a particular node?

Define "node".

>   Or something else, please?

It means a job failed and it will be re-scheduled for another try as
opposed to having been re-rescheduled for a large enough number of times
("execution.retries" option) that swift has decided to give up.




From HodgessE at uhd.edu  Thu Jul  2 10:46:32 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 10:46:32 -0500
Subject: [Swift-user] setting throttle options
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37159@BALI.uhd.campus><1246548017.28190.24.camel@localhost><70A5AC06FDB5E54482D19E1C04CDFCF307C3715B@BALI.uhd.campus>
	<1246549015.28742.13.camel@localhost>
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3715E@BALI.uhd.campus>

But what's a girl to do?


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu



-----Original Message-----
From: Mihael Hategan [mailto:hategan at mcs.anl.gov]
Sent: Thu 7/2/2009 10:36 AM
To: Hodgess, Erin
Cc: swift-user at ci.uchicago.edu
Subject: RE: [Swift-user] setting throttle options
 
On Thu, 2009-07-02 at 10:24 -0500, Hodgess, Erin wrote:
> We are interested in timing and "who gets what jobs", etc.

Very odd. It's a rather obscure and difficult issue. Yet, for some
strange reason, whenever there is a student or a new person temporarily
on the project, they somehow jump exactly to that. And the results have
so far invariably been lacking.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/3d16b9e7/attachment.html>

From HodgessE at uhd.edu  Thu Jul  2 12:39:00 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 12:39:00 -0500
Subject: [Swift-user] condor on communicado
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3716D@BALI.uhd.campus>

Is there a condor pool on communicado, please?

This is what I get:

[erin at communicado ~]$ condor_q
Error: 

Extra Info: You probably saw this error because the condor_schedd is not 
running on the machine you are trying to query. If the condor_schedd is not 
running, the Condor system will not be able to find an address and port to 
connect to and satisfy this request. Please make sure the Condor daemons are 
running and try again.
 
Extra Info: If the condor_schedd is running on the machine you are trying to 
query and you still see the error, the most likely cause is that you have 
setup a personal Condor, you have not defined SCHEDD_NAME in your 
condor_config file, and something is wrong with your SCHEDD_ADDRESS_FILE 
setting. You must define either or both of those settings in your config 
file, or you must use the -name option to condor_q. Please see the Condor 
manual for details on SCHEDD_NAME and SCHEDD_ADDRESS_FILE. 
[erin at communicado ~]$ 


Here is my .soft file:
[erin at communicado ~]$ cat .soft
#
# This is your SoftEnv configuration run control file.
#
#   It is used to tell SoftEnv how to customize your environment by
#   setting up variables such as PATH and MANPATH.  To learn more
#   about this file, do a "man softenv".
#
@python-2.5
+java-sun                                                                      @swift
+apache-ant
+gx-map
+condor
+gx-map
@globus-4
+R
+torque
+maui
+matlab-7.7
+osg-client
#+osg-client-1.0.0-r1
@osg
+apache-ant
+gx-map
@default
[erin at communicado ~]$ 



Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/69538f4d/attachment.html>

From HodgessE at uhd.edu  Thu Jul  2 21:04:26 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Thu, 2 Jul 2009 21:04:26 -0500
Subject: [Swift-user] Problem with running swift and condor
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717E@BALI.uhd.campus>

Hello all.

I am running swift on communicado.

In the sites file, I am referring to an OSG site to be run on condor rather than gram.
Here is the sites file:
<config>

  <!-- Clemson-ciTeam -->
  <pool handle="Clemson-ciTeam" >
    <gridftp  url="gsiftp://osgce.cs.clemson.edu/" />
    <jobmanager universe="vanilla" url="osgce.cs.clemson.edu:2119/jobmanager-condor" major="2" />
 <execution provider="condor" />
 <profile namespace="globus" key="jobType">grid</profile>
 <profile namespace="globus" key="gridResource">gt2 osgce.cs.clemson.edu/jobmanager-condor</profile>

    <workdirectory >/home/osg</workdirectory>
  </pool>

</config>

The swift program is very simple; running a word count on a file:
[erin at communicado swift1]$ cat wc1a.swift

type file;  
  
app (file o) count(file i) {  
  wc @i stdout=@o;  
}  
  
file i <"input.txt">;  

foreach j in [1:15]
	{
file o <single_file_mapper;file=@strcat("results/",j,".out")>;
	(o)=count(i);
	}


But when I run the swift job, I get a "parameter not supported".  Note: this job works fine on localhost, teraport, and on several OSG sites when set up using gram.


[erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file sites1i.xml wc1a.swift
Swift svn swift-r2950 cog-r2406

RunID: 20090702-2009-sk2d7m89
Progress:
Progress:  Selecting site:13  Initializing site shared directory:1  Stage in:1
Progress:  Selecting site:13  Submitting:1  Submitted:1
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/8 on Clemson-ciTeam
Progress:  Selecting site:12  Stage in:1 Failed but can retry:2
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 on Clemson-ciTeam
Progress:  Selecting site:12  Submitted:1 Failed but can retry:2
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b on Clemson-ciTeam
Progress:  Selecting site:11  Stage in:1 Failed but can retry:3
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/e on Clemson-ciTeam
Progress:  Selecting site:11 Failed but can retry:4
Progress:  Selecting site:11 Failed but can retry:4
Progress:  Selecting site:11 Failed but can retry:4
Progress:  Selecting site:10  Stage in:1 Failed but can retry:4
Progress:  Selecting site:10  Submitted:1 Failed but can retry:4
Progress:  Selecting site:10 Failed but can retry:5
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/g on Clemson-ciTeam
Progress:  Selecting site:10 Failed but can retry:5
Progress:  Selecting site:10 Failed but can retry:5
Progress:  Selecting site:10 Failed but can retry:5
Progress:  Selecting site:9  Stage in:1 Failed but can retry:5
Progress:  Selecting site:9  Submitted:1 Failed but can retry:5
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/i on Clemson-ciTeam
Progress:  Selecting site:9 Failed but can retry:6
Progress:  Selecting site:9 Failed but can retry:6
Progress:  Selecting site:9 Failed but can retry:6
Progress:  Selecting site:8  Stage in:1 Failed but can retry:6
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/k on Clemson-ciTeam
Progress:  Selecting site:8 Failed but can retry:7
Progress:  Selecting site:8 Failed but can retry:7
Progress:  Selecting site:8 Failed but can retry:7
Progress:  Selecting site:7  Stage in:1 Failed but can retry:7
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/m on Clemson-ciTeam
Progress:  Selecting site:7 Failed but can retry:8
Progress:  Selecting site:7 Failed but can retry:8
Progress:  Selecting site:7 Failed but can retry:8
Progress:  Selecting site:6  Stage in:1 Failed but can retry:8
Progress:  Selecting site:6  Submitted:1 Failed but can retry:8
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/o on Clemson-ciTeam
Progress:  Selecting site:6 Failed but can retry:9
Progress:  Selecting site:6 Failed but can retry:9
Progress:  Selecting site:6 Failed but can retry:9
Progress:  Selecting site:6 Failed but can retry:9
Progress:  Selecting site:5  Stage in:1 Failed but can retry:9
Progress:  Selecting site:5  Submitted:1 Failed but can retry:9
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/q on Clemson-ciTeam
Progress:  Selecting site:5 Failed but can retry:10
Progress:  Selecting site:5 Failed but can retry:10
Progress:  Selecting site:5 Failed but can retry:10
Progress:  Selecting site:4  Stage in:1 Failed but can retry:10
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/s on Clemson-ciTeam
Progress:  Selecting site:4 Failed but can retry:11
Progress:  Selecting site:4 Failed but can retry:11
Progress:  Selecting site:4 Failed but can retry:11
Progress:  Selecting site:3  Stage in:1 Failed but can retry:11
Progress:  Selecting site:3  Submitted:1 Failed but can retry:11
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/u on Clemson-ciTeam
Progress:  Selecting site:3 Failed but can retry:12


Progress:  Selecting site:3 Failed but can retry:12
Progress:  Selecting site:3 Failed but can retry:12
Progress:  Selecting site:2  Stage in:1 Failed but can retry:12
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/x on Clemson-ciTeam
Progress:  Selecting site:2 Failed but can retry:13
Progress:  Selecting site:2 Failed but can retry:13
Progress:  Selecting site:2 Failed but can retry:13
Progress:  Selecting site:2 Failed but can retry:13
Progress:  Selecting site:1  Stage in:1 Failed but can retry:13
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/z on Clemson-ciTeam
Progress:  Selecting site:1 Failed but can retry:14
Progress:  Selecting site:1 Failed but can retry:14
Progress:  Selecting site:1 Failed but can retry:14
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/1 on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/3 on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/5 on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/7 on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15

Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/d on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/f on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/h on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/j on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/l on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/n on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/p on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/r on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/t on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Progress:  Submitted:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/v on Clemson-ciTeam
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress: Failed but can retry:15
Progress:  Stage in:1 Failed but can retry:14
Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/y on Clemson-ciTeam
Progress:  Failed:1 Failed but can retry:14
Execution failed:
	Exception in wc:
Arguments: [input.txt]
Host: Clemson-ciTeam
Directory: wc1a-20090702-2009-sk2d7m89/jobs/y/wc-y2zfj4dj
stderr.txt: 

stdout.txt: 

----

Caused by:
	Cannot submit job
Caused by:
	Parameter not supported
[erin at communicado swift1]$ 

Thank you for your kind attention.

Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090702/92d2e657/attachment.html>

From wilde at mcs.anl.gov  Thu Jul  2 21:16:05 2009
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Thu, 02 Jul 2009 21:16:05 -0500
Subject: [Swift-user] Problem with running swift and condor
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717E@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717E@BALI.uhd.campus>
Message-ID: <4A4D69E5.6010308@mcs.anl.gov>

Erin, Im not sure where "parameter not supported" is coming from, but 
check if this is a problem:

your pool element for clemson says /home/osg for its work directory, but 
for clemson-ciTeam in the RESS output I see:

     <workdirectory 
 >/export/osg/data/engage/tmp/Clemson-ciTeam</workdirectory>

Unless you know /home/osg to be a good value for that site, I would try 
the value above.

- Mike


On 7/2/09 9:04 PM, Hodgess, Erin wrote:
> Hello all.
> 
> I am running swift on communicado.
> 
> In the sites file, I am referring to an OSG site to be run on condor 
> rather than gram.
> Here is the sites file:
> <config>
> 
>   <!-- Clemson-ciTeam -->
>   <pool handle="Clemson-ciTeam" >
>     <gridftp  url="gsiftp://osgce.cs.clemson.edu/" />
>     <jobmanager universe="vanilla" 
> url="osgce.cs.clemson.edu:2119/jobmanager-condor" major="2" />
>  <execution provider="condor" />
>  <profile namespace="globus" key="jobType">grid</profile>
>  <profile namespace="globus" key="gridResource">gt2 
> osgce.cs.clemson.edu/jobmanager-condor</profile>
> 
>     <workdirectory >/home/osg</workdirectory>
>   </pool>
> 
> </config>
> 
> The swift program is very simple; running a word count on a file:
> [erin at communicado swift1]$ cat wc1a.swift
> 
> type file; 
>  
> app (file o) count(file i) { 
>   wc @i stdout=@o; 
> } 
>  
> file i <"input.txt">; 
> 
> foreach j in [1:15]
>         {
> file o <single_file_mapper;file=@strcat("results/",j,".out")>;
>         (o)=count(i);
>         }
> 
> 
> But when I run the swift job, I get a "parameter not supported".  Note: 
> this job works fine on localhost, teraport, and on several OSG sites 
> when set up using gram.
> 
> 
> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file 
> sites1i.xml wc1a.swift
> Swift svn swift-r2950 cog-r2406
> 
> RunID: 20090702-2009-sk2d7m89
> Progress:
> Progress:  Selecting site:13  Initializing site shared directory:1  
> Stage in:1
> Progress:  Selecting site:13  Submitting:1  Submitted:1
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/8 
> on Clemson-ciTeam
> Progress:  Selecting site:12  Stage in:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 
> on Clemson-ciTeam
> Progress:  Selecting site:12  Submitted:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b 
> on Clemson-ciTeam
> Progress:  Selecting site:11  Stage in:1 Failed but can retry:3
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/e 
> on Clemson-ciTeam
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:10  Stage in:1 Failed but can retry:4
> Progress:  Selecting site:10  Submitted:1 Failed but can retry:4
> Progress:  Selecting site:10 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/g 
> on Clemson-ciTeam
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:9  Stage in:1 Failed but can retry:5
> Progress:  Selecting site:9  Submitted:1 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/i 
> on Clemson-ciTeam
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:8  Stage in:1 Failed but can retry:6
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/k 
> on Clemson-ciTeam
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:7  Stage in:1 Failed but can retry:7
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/m 
> on Clemson-ciTeam
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:6  Stage in:1 Failed but can retry:8
> Progress:  Selecting site:6  Submitted:1 Failed but can retry:8
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/o 
> on Clemson-ciTeam
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:5  Stage in:1 Failed but can retry:9
> Progress:  Selecting site:5  Submitted:1 Failed but can retry:9
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/q 
> on Clemson-ciTeam
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:4  Stage in:1 Failed but can retry:10
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/s 
> on Clemson-ciTeam
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:3  Stage in:1 Failed but can retry:11
> Progress:  Selecting site:3  Submitted:1 Failed but can retry:11
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/u 
> on Clemson-ciTeam
> Progress:  Selecting site:3 Failed but can retry:12
> 
> 
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:2  Stage in:1 Failed but can retry:12
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/x 
> on Clemson-ciTeam
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:1  Stage in:1 Failed but can retry:13
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/z 
> on Clemson-ciTeam
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/1 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/3 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/5 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/7 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> 
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/d 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/f 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/h 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/j 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/l 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/n 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/p 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/r 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/t 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/v 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/y 
> on Clemson-ciTeam
> Progress:  Failed:1 Failed but can retry:14
> Execution failed:
>         Exception in wc:
> Arguments: [input.txt]
> Host: Clemson-ciTeam
> Directory: wc1a-20090702-2009-sk2d7m89/jobs/y/wc-y2zfj4dj
> stderr.txt:
> 
> stdout.txt:
> 
> ----
> 
> Caused by:
>         Cannot submit job
> Caused by:
>         Parameter not supported
> [erin at communicado swift1]$
> 
> Thank you for your kind attention.
> 
> Erin
> 
> 
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user


From benc at hawaga.org.uk  Fri Jul  3 04:33:23 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Fri, 3 Jul 2009 09:33:23 +0000 (GMT)
Subject: [Swift-user] Problem with running swift and condor
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717E@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717E@BALI.uhd.campus>
Message-ID: <Pine.LNX.4.64.0907030931170.32243@dildano.hawaga.org.uk>


On Thu, 2 Jul 2009, Hodgess, Erin wrote:

> In the sites file, I am referring to an OSG site to be run on condor 
> rather than gram. Here is the sites file: <config>

You are specifying a jobmanager and an execution provider - you can only 
specify one (although swift does not properly detect if you do both).

The jobmanager element means to use plain GRAM which is conficting with 
your additional specification of condor. Use only one.

>     <jobmanager universe="vanilla" url="osgce.cs.clemson.edu:2119/jobmanager-condor" major="2" />

>  <execution provider="condor" />
>  <profile namespace="globus" key="gridResource">gt2 osgce.cs.clemson.edu/jobmanager-condor</profile>



> 
>     <workdirectory >/home/osg</workdirectory>
>   </pool>
> 
> </config>
> 
> The swift program is very simple; running a word count on a file:
> [erin at communicado swift1]$ cat wc1a.swift
> 
> type file;  
>   
> app (file o) count(file i) {  
>   wc @i stdout=@o;  
> }  
>   
> file i <"input.txt">;  
> 
> foreach j in [1:15]
> 	{
> file o <single_file_mapper;file=@strcat("results/",j,".out")>;
> 	(o)=count(i);
> 	}
> 
> 
> But when I run the swift job, I get a "parameter not supported".  Note: this job works fine on localhost, teraport, and on several OSG sites when set up using gram.
> 
> 
> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file sites1i.xml wc1a.swift
> Swift svn swift-r2950 cog-r2406
> 
> RunID: 20090702-2009-sk2d7m89
> Progress:
> Progress:  Selecting site:13  Initializing site shared directory:1  Stage in:1
> Progress:  Selecting site:13  Submitting:1  Submitted:1
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/8 on Clemson-ciTeam
> Progress:  Selecting site:12  Stage in:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 on Clemson-ciTeam
> Progress:  Selecting site:12  Submitted:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b on Clemson-ciTeam
> Progress:  Selecting site:11  Stage in:1 Failed but can retry:3
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/e on Clemson-ciTeam
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:10  Stage in:1 Failed but can retry:4
> Progress:  Selecting site:10  Submitted:1 Failed but can retry:4
> Progress:  Selecting site:10 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/g on Clemson-ciTeam
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:9  Stage in:1 Failed but can retry:5
> Progress:  Selecting site:9  Submitted:1 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/i on Clemson-ciTeam
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:8  Stage in:1 Failed but can retry:6
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/k on Clemson-ciTeam
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:7  Stage in:1 Failed but can retry:7
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/m on Clemson-ciTeam
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:6  Stage in:1 Failed but can retry:8
> Progress:  Selecting site:6  Submitted:1 Failed but can retry:8
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/o on Clemson-ciTeam
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:5  Stage in:1 Failed but can retry:9
> Progress:  Selecting site:5  Submitted:1 Failed but can retry:9
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/q on Clemson-ciTeam
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:4  Stage in:1 Failed but can retry:10
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/s on Clemson-ciTeam
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:3  Stage in:1 Failed but can retry:11
> Progress:  Selecting site:3  Submitted:1 Failed but can retry:11
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/u on Clemson-ciTeam
> Progress:  Selecting site:3 Failed but can retry:12
> 
> 
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:2  Stage in:1 Failed but can retry:12
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/x on Clemson-ciTeam
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:1  Stage in:1 Failed but can retry:13
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/z on Clemson-ciTeam
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/1 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/3 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/5 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/7 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> 
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/d on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/f on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/h on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/j on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/l on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/n on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/p on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/r on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/t on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/v on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/y on Clemson-ciTeam
> Progress:  Failed:1 Failed but can retry:14
> Execution failed:
> 	Exception in wc:
> Arguments: [input.txt]
> Host: Clemson-ciTeam
> Directory: wc1a-20090702-2009-sk2d7m89/jobs/y/wc-y2zfj4dj
> stderr.txt: 
> 
> stdout.txt: 
> 
> ----
> 
> Caused by:
> 	Cannot submit job
> Caused by:
> 	Parameter not supported
> [erin at communicado swift1]$ 
> 
> Thank you for your kind attention.
> 
> Erin
> 
> 
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
> 
> 


From HodgessE at uhd.edu  Fri Jul  3 06:17:47 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Fri, 3 Jul 2009 06:17:47 -0500
Subject: [Swift-user] Problem with running swift and condor
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717E@BALI.uhd.campus>
	<4A4D69E5.6010308@mcs.anl.gov>
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717F@BALI.uhd.campus>

/home/osg is a good site.  I have used it with several other applications.

Thank you,
Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu



-----Original Message-----
From: Michael Wilde [mailto:wilde at mcs.anl.gov]
Sent: Thu 7/2/2009 9:16 PM
To: Hodgess, Erin
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] Problem with running swift and condor
 
Erin, Im not sure where "parameter not supported" is coming from, but 
check if this is a problem:

your pool element for clemson says /home/osg for its work directory, but 
for clemson-ciTeam in the RESS output I see:

     <workdirectory 
 >/export/osg/data/engage/tmp/Clemson-ciTeam</workdirectory>

Unless you know /home/osg to be a good value for that site, I would try 
the value above.

- Mike


On 7/2/09 9:04 PM, Hodgess, Erin wrote:
> Hello all.
> 
> I am running swift on communicado.
> 
> In the sites file, I am referring to an OSG site to be run on condor 
> rather than gram.
> Here is the sites file:
> <config>
> 
>   <!-- Clemson-ciTeam -->
>   <pool handle="Clemson-ciTeam" >
>     <gridftp  url="gsiftp://osgce.cs.clemson.edu/" />
>     <jobmanager universe="vanilla" 
> url="osgce.cs.clemson.edu:2119/jobmanager-condor" major="2" />
>  <execution provider="condor" />
>  <profile namespace="globus" key="jobType">grid</profile>
>  <profile namespace="globus" key="gridResource">gt2 
> osgce.cs.clemson.edu/jobmanager-condor</profile>
> 
>     <workdirectory >/home/osg</workdirectory>
>   </pool>
> 
> </config>
> 
> The swift program is very simple; running a word count on a file:
> [erin at communicado swift1]$ cat wc1a.swift
> 
> type file; 
>  
> app (file o) count(file i) { 
>   wc @i stdout=@o; 
> } 
>  
> file i <"input.txt">; 
> 
> foreach j in [1:15]
>         {
> file o <single_file_mapper;file=@strcat("results/",j,".out")>;
>         (o)=count(i);
>         }
> 
> 
> But when I run the swift job, I get a "parameter not supported".  Note: 
> this job works fine on localhost, teraport, and on several OSG sites 
> when set up using gram.
> 
> 
> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file 
> sites1i.xml wc1a.swift
> Swift svn swift-r2950 cog-r2406
> 
> RunID: 20090702-2009-sk2d7m89
> Progress:
> Progress:  Selecting site:13  Initializing site shared directory:1  
> Stage in:1
> Progress:  Selecting site:13  Submitting:1  Submitted:1
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/8 
> on Clemson-ciTeam
> Progress:  Selecting site:12  Stage in:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 
> on Clemson-ciTeam
> Progress:  Selecting site:12  Submitted:1 Failed but can retry:2
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b 
> on Clemson-ciTeam
> Progress:  Selecting site:11  Stage in:1 Failed but can retry:3
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/e 
> on Clemson-ciTeam
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:11 Failed but can retry:4
> Progress:  Selecting site:10  Stage in:1 Failed but can retry:4
> Progress:  Selecting site:10  Submitted:1 Failed but can retry:4
> Progress:  Selecting site:10 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/g 
> on Clemson-ciTeam
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:10 Failed but can retry:5
> Progress:  Selecting site:9  Stage in:1 Failed but can retry:5
> Progress:  Selecting site:9  Submitted:1 Failed but can retry:5
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/i 
> on Clemson-ciTeam
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:9 Failed but can retry:6
> Progress:  Selecting site:8  Stage in:1 Failed but can retry:6
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/k 
> on Clemson-ciTeam
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:8 Failed but can retry:7
> Progress:  Selecting site:7  Stage in:1 Failed but can retry:7
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/m 
> on Clemson-ciTeam
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:7 Failed but can retry:8
> Progress:  Selecting site:6  Stage in:1 Failed but can retry:8
> Progress:  Selecting site:6  Submitted:1 Failed but can retry:8
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/o 
> on Clemson-ciTeam
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:6 Failed but can retry:9
> Progress:  Selecting site:5  Stage in:1 Failed but can retry:9
> Progress:  Selecting site:5  Submitted:1 Failed but can retry:9
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/q 
> on Clemson-ciTeam
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:5 Failed but can retry:10
> Progress:  Selecting site:4  Stage in:1 Failed but can retry:10
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/s 
> on Clemson-ciTeam
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:4 Failed but can retry:11
> Progress:  Selecting site:3  Stage in:1 Failed but can retry:11
> Progress:  Selecting site:3  Submitted:1 Failed but can retry:11
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/u 
> on Clemson-ciTeam
> Progress:  Selecting site:3 Failed but can retry:12
> 
> 
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:3 Failed but can retry:12
> Progress:  Selecting site:2  Stage in:1 Failed but can retry:12
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/x 
> on Clemson-ciTeam
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:2 Failed but can retry:13
> Progress:  Selecting site:1  Stage in:1 Failed but can retry:13
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/z 
> on Clemson-ciTeam
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Selecting site:1 Failed but can retry:14
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/1 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/3 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/5 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/7 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> 
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/9 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/b 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/d 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/f 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/h 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/j 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/l 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/n 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/p 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/r 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/t 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Progress:  Submitted:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/v 
> on Clemson-ciTeam
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress: Failed but can retry:15
> Progress:  Stage in:1 Failed but can retry:14
> Failed to transfer wrapper log from wc1a-20090702-2009-sk2d7m89/info/y 
> on Clemson-ciTeam
> Progress:  Failed:1 Failed but can retry:14
> Execution failed:
>         Exception in wc:
> Arguments: [input.txt]
> Host: Clemson-ciTeam
> Directory: wc1a-20090702-2009-sk2d7m89/jobs/y/wc-y2zfj4dj
> stderr.txt:
> 
> stdout.txt:
> 
> ----
> 
> Caused by:
>         Cannot submit job
> Caused by:
>         Parameter not supported
> [erin at communicado swift1]$
> 
> Thank you for your kind attention.
> 
> Erin
> 
> 
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090703/a3e8a6d6/attachment.html>

From benc at hawaga.org.uk  Fri Jul  3 12:39:34 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Fri, 3 Jul 2009 17:39:34 +0000 (GMT)
Subject: [Swift-user] Problem with running swift and condor
In-Reply-To: <Pine.LNX.4.64.0907030931170.32243@dildano.hawaga.org.uk>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C3717E@BALI.uhd.campus>
	<Pine.LNX.4.64.0907030931170.32243@dildano.hawaga.org.uk>
Message-ID: <Pine.LNX.4.64.0907031739020.32243@dildano.hawaga.org.uk>


Since Swift r2984, the osg sites file generator has had a -condorg option 
which will generate n OSG sites file to use condor-g instead of direct 
gram submission.

You might find that useful.

-- 




From HodgessE at uhd.edu  Mon Jul  6 10:36:42 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Mon, 6 Jul 2009 10:36:42 -0500
Subject: [Swift-user] problem with swift-osg-ress-catalog function
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C3718B@BALI.uhd.campus>

Hi Swift People:

I just tried the swift-osg-ress-site-catalog command with the condor-g option, but I get the following errors:


[erin at communicado ~]$ swift-osg-ress-site-catalog --condor-g
Unknown option: c
Usage:
    swift-osg-ress-site-catalog [options]

Options:
    --help  Show this help message

    --vo=[name]
            Set what VO to query ReSS for

    --engage-verified
            Only retrieve sites verified by the Engagement VO site
            verification tests This can not be used together with --vo, as
            the query will only work for sites advertising support for the
            Engagement VO.

            This option means information will be retrieved from the
            Engagement collector instead of the top-level ReSS collector.

    --out=[filename]
            Write to [filename] instead of stdout

The Swift user guide indicates that the condor-g is indeed an option, but actual use here indicates that it is not an option.

Any help much appreciated.

Sincerely,
Erin



Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090706/c4c4d019/attachment.html>

From benc at hawaga.org.uk  Mon Jul  6 12:50:31 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Mon, 6 Jul 2009 17:50:31 +0000 (GMT)
Subject: [Swift-user] problem with swift-osg-ress-catalog function
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C3718B@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C3718B@BALI.uhd.campus>
Message-ID: <Pine.LNX.4.64.0907061747160.32243@dildano.hawaga.org.uk>


Most likely you do not have a recent enough version of 
swift-osg-ress-site-catalog - you will need a source build of swift of at 
least r2984, as that option has not been in any Swift release yet.

On Mon, 6 Jul 2009, Hodgess, Erin wrote:

> Hi Swift People:
> 
> I just tried the swift-osg-ress-site-catalog command with the condor-g option, but I get the following errors:
> 
> 
> [erin at communicado ~]$ swift-osg-ress-site-catalog --condor-g
> Unknown option: c
> Usage:
>     swift-osg-ress-site-catalog [options]
> 
> Options:
>     --help  Show this help message
> 
>     --vo=[name]
>             Set what VO to query ReSS for
> 
>     --engage-verified
>             Only retrieve sites verified by the Engagement VO site
>             verification tests This can not be used together with --vo, as
>             the query will only work for sites advertising support for the
>             Engagement VO.
> 
>             This option means information will be retrieved from the
>             Engagement collector instead of the top-level ReSS collector.
> 
>     --out=[filename]
>             Write to [filename] instead of stdout
> 
> The Swift user guide indicates that the condor-g is indeed an option, but actual use here indicates that it is not an option.
> 
> Any help much appreciated.
> 
> Sincerely,
> Erin
> 
> 
> 
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
> 
> 


From HodgessE at uhd.edu  Mon Jul  6 14:07:13 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Mon, 6 Jul 2009 14:07:13 -0500
Subject: [Swift-user] condor on communicado problem
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C37193@BALI.uhd.campus>

Hi again, Swift People:

I'm on communicado, trying to run swift via condor (instead of gram).

I generated my sites file and selected one location.  

Here is my output:

[erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file sites1k.xml wc1a.swift
Swift svn swift-r2998 (swift modified locally) cog-r2427

RunID: 20090706-1334-iq5phkm6
Progress:
Progress:  Selecting site:13  Initializing site shared directory:2
Progress:  Selecting site:10  Initializing site shared directory:5
Progress:  Selecting site:9  Initializing site shared directory:6
Progress:  Selecting site:8  Initializing site shared directory:7
Progress:  Selecting site:7  Initializing site shared directory:8
Progress:  Selecting site:6  Initializing site shared directory:9
Progress:  Selecting site:5  Initializing site shared directory:10
Progress:  Selecting site:4  Initializing site shared directory:11
Progress:  Selecting site:3  Initializing site shared directory:12
Progress:  Selecting site:2  Initializing site shared directory:13
Progress:  Selecting site:2  Initializing site shared directory:13
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Progress:  Initializing site shared directory:15
Execution failed:
	Could not initialize shared directory on Clemson-ciTeam_osgce.cs.clemson.edu
Caused by:
	org.globus.cog.abstraction.impl.file.FileResourceException: Cannot create directory /export/osg/data/engage/tmp
Caused by:
	Server refused performing the request. Custom message: Server refused creating directory (error code 1) [Nested exception message:  Custom message: Unexpected reply: 500-Command failed : globus_gridftp_server_file.c:globus_l_gfs_file_mkdir:554:
500-System error in mkdir: Permission denied
500-A system call failed: Permission denied
500 End.]
[erin at communicado swift1]$ 

This was using all of the values from the sites file.

The directory that the sites file has is not present.

Does anyone have any suggestions, please?

Thanks in advance,
Erin

Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090706/de7d8adf/attachment.html>

From aespinosa at cs.uchicago.edu  Mon Jul  6 14:16:37 2009
From: aespinosa at cs.uchicago.edu (Allan Espinosa)
Date: Mon, 6 Jul 2009 14:16:37 -0500
Subject: [Swift-user] condor on communicado problem
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C37193@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37193@BALI.uhd.campus>
Message-ID: <50b07b4b0907061216q21fd655btc87f4de3721b148c@mail.gmail.com>

Hi Erin,

under what VO is your proxy when you accessed the resource?  each VO
runs under a different userspace so you can't access Engage user
directories if the vo with you current proxy is not using engage.

to make sure you get the right VO, you can use voms-proxy-init

-Allan

2009/7/6 Hodgess, Erin <HodgessE at uhd.edu>:
> Hi again, Swift People:
>
> I'm on communicado, trying to run swift via condor (instead of gram).
>
> I generated my sites file and selected one location.
>
> Here is my output:
>
> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file
> sites1k.xml wc1a.swift
> Swift svn swift-r2998 (swift modified locally) cog-r2427
>
> RunID: 20090706-1334-iq5phkm6
> Progress:
> Progress:? Selecting site:13? Initializing site shared directory:2
> Progress:? Selecting site:10? Initializing site shared directory:5
> Progress:? Selecting site:9? Initializing site shared directory:6
> Progress:? Selecting site:8? Initializing site shared directory:7
> Progress:? Selecting site:7? Initializing site shared directory:8
> Progress:? Selecting site:6? Initializing site shared directory:9
> Progress:? Selecting site:5? Initializing site shared directory:10
> Progress:? Selecting site:4? Initializing site shared directory:11
> Progress:? Selecting site:3? Initializing site shared directory:12
> Progress:? Selecting site:2? Initializing site shared directory:13
> Progress:? Selecting site:2? Initializing site shared directory:13
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Execution failed:
> ??????? Could not initialize shared directory on
> Clemson-ciTeam_osgce.cs.clemson.edu
> Caused by:
> ??????? org.globus.cog.abstraction.impl.file.FileResourceException: Cannot
> create directory /export/osg/data/engage/tmp
> Caused by:
> ??????? Server refused performing the request. Custom message: Server
> refused creating directory (error code 1) [Nested exception message:? Custom
> message: Unexpected reply: 500-Command failed :
> globus_gridftp_server_file.c:globus_l_gfs_file_mkdir:554:
> 500-System error in mkdir: Permission denied
> 500-A system call failed: Permission denied
> 500 End.]
> [erin at communicado swift1]$
>
> This was using all of the values from the sites file.
>
> The directory that the sites file has is not present.
>
> Does anyone have any suggestions, please?
>
> Thanks in advance,
> Erin
>
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu


From HodgessE at uhd.edu  Mon Jul  6 14:18:13 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Mon, 6 Jul 2009 14:18:13 -0500
Subject: [Swift-user] condor on communicado problem
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37193@BALI.uhd.campus>
	<50b07b4b0907061216q21fd655btc87f4de3721b148c@mail.gmail.com>
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C37196@BALI.uhd.campus>

I tried the following:

[erin at communicado ~]$  voms-proxy-init
-bash: voms-proxy-init: command not found
[erin at communicado ~]$ 

But this is not working either.

Thanks,
Erin


Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu



-----Original Message-----
From: yecartes at gmail.com on behalf of Allan Espinosa
Sent: Mon 7/6/2009 2:16 PM
To: Hodgess, Erin
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] condor on communicado problem
 
Hi Erin,

under what VO is your proxy when you accessed the resource?  each VO
runs under a different userspace so you can't access Engage user
directories if the vo with you current proxy is not using engage.

to make sure you get the right VO, you can use voms-proxy-init

-Allan

2009/7/6 Hodgess, Erin <HodgessE at uhd.edu>:
> Hi again, Swift People:
>
> I'm on communicado, trying to run swift via condor (instead of gram).
>
> I generated my sites file and selected one location.
>
> Here is my output:
>
> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file
> sites1k.xml wc1a.swift
> Swift svn swift-r2998 (swift modified locally) cog-r2427
>
> RunID: 20090706-1334-iq5phkm6
> Progress:
> Progress:? Selecting site:13? Initializing site shared directory:2
> Progress:? Selecting site:10? Initializing site shared directory:5
> Progress:? Selecting site:9? Initializing site shared directory:6
> Progress:? Selecting site:8? Initializing site shared directory:7
> Progress:? Selecting site:7? Initializing site shared directory:8
> Progress:? Selecting site:6? Initializing site shared directory:9
> Progress:? Selecting site:5? Initializing site shared directory:10
> Progress:? Selecting site:4? Initializing site shared directory:11
> Progress:? Selecting site:3? Initializing site shared directory:12
> Progress:? Selecting site:2? Initializing site shared directory:13
> Progress:? Selecting site:2? Initializing site shared directory:13
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Progress:? Initializing site shared directory:15
> Execution failed:
> ??????? Could not initialize shared directory on
> Clemson-ciTeam_osgce.cs.clemson.edu
> Caused by:
> ??????? org.globus.cog.abstraction.impl.file.FileResourceException: Cannot
> create directory /export/osg/data/engage/tmp
> Caused by:
> ??????? Server refused performing the request. Custom message: Server
> refused creating directory (error code 1) [Nested exception message:? Custom
> message: Unexpected reply: 500-Command failed :
> globus_gridftp_server_file.c:globus_l_gfs_file_mkdir:554:
> 500-System error in mkdir: Permission denied
> 500-A system call failed: Permission denied
> 500 End.]
> [erin at communicado swift1]$
>
> This was using all of the values from the sites file.
>
> The directory that the sites file has is not present.
>
> Does anyone have any suggestions, please?
>
> Thanks in advance,
> Erin
>
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090706/d7c8ffec/attachment.html>

From aespinosa at cs.uchicago.edu  Mon Jul  6 14:38:47 2009
From: aespinosa at cs.uchicago.edu (Allan Espinosa)
Date: Mon, 6 Jul 2009 14:38:47 -0500
Subject: [Swift-user] condor on communicado problem
In-Reply-To: <70A5AC06FDB5E54482D19E1C04CDFCF307C37196@BALI.uhd.campus>
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37193@BALI.uhd.campus>
	<50b07b4b0907061216q21fd655btc87f4de3721b148c@mail.gmail.com>
	<70A5AC06FDB5E54482D19E1C04CDFCF307C37196@BALI.uhd.campus>
Message-ID: <50b07b4b0907061238r207d7312pcd31fe7e920d1383@mail.gmail.com>

do you have +osg-client or @osg in your ~/.soft file?

2009/7/6 Hodgess, Erin <HodgessE at uhd.edu>:
> I tried the following:
>
> [erin at communicado ~]$? voms-proxy-init
> -bash: voms-proxy-init: command not found
> [erin at communicado ~]$
>
> But this is not working either.
>
> Thanks,
> Erin
>
>
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
>
>
>
> -----Original Message-----
> From: yecartes at gmail.com on behalf of Allan Espinosa
> Sent: Mon 7/6/2009 2:16 PM
> To: Hodgess, Erin
> Cc: swift-user at ci.uchicago.edu
> Subject: Re: [Swift-user] condor on communicado problem
>
> Hi Erin,
>
> under what VO is your proxy when you accessed the resource?? each VO
> runs under a different userspace so you can't access Engage user
> directories if the vo with you current proxy is not using engage.
>
> to make sure you get the right VO, you can use voms-proxy-init
>
> -Allan
>
> 2009/7/6 Hodgess, Erin <HodgessE at uhd.edu>:
>> Hi again, Swift People:
>>
>> I'm on communicado, trying to run swift via condor (instead of gram).
>>
>> I generated my sites file and selected one location.
>>
>> Here is my output:
>>
>> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file
>> sites1k.xml wc1a.swift
>> Swift svn swift-r2998 (swift modified locally) cog-r2427
>>
>> RunID: 20090706-1334-iq5phkm6
>> Progress:
>> Progress:? Selecting site:13? Initializing site shared directory:2
>> Progress:? Selecting site:10? Initializing site shared directory:5
>> Progress:? Selecting site:9? Initializing site shared directory:6
>> Progress:? Selecting site:8? Initializing site shared directory:7
>> Progress:? Selecting site:7? Initializing site shared directory:8
>> Progress:? Selecting site:6? Initializing site shared directory:9
>> Progress:? Selecting site:5? Initializing site shared directory:10
>> Progress:? Selecting site:4? Initializing site shared directory:11
>> Progress:? Selecting site:3? Initializing site shared directory:12
>> Progress:? Selecting site:2? Initializing site shared directory:13
>> Progress:? Selecting site:2? Initializing site shared directory:13
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Execution failed:
>> ??????? Could not initialize shared directory on
>> Clemson-ciTeam_osgce.cs.clemson.edu
>> Caused by:
>> ??????? org.globus.cog.abstraction.impl.file.FileResourceException: Cannot
>> create directory /export/osg/data/engage/tmp
>> Caused by:
>> ??????? Server refused performing the request. Custom message: Server
>> refused creating directory (error code 1) [Nested exception message:
>> Custom
>> message: Unexpected reply: 500-Command failed :
>> globus_gridftp_server_file.c:globus_l_gfs_file_mkdir:554:
>> 500-System error in mkdir: Permission denied
>> 500-A system call failed: Permission denied
>> 500 End.]
>> [erin at communicado swift1]$
>>
>> This was using all of the values from the sites file.
>>
>> The directory that the sites file has is not present.
>>
>> Does anyone have any suggestions, please?
>>
>> Thanks in advance,
>> Erin
>>
>> Erin M. Hodgess, PhD
>> Associate Professor
>> Department of Computer and Mathematical Sciences
>> University of Houston - Downtown
>> mailto: hodgesse at uhd.edu
>
>



-- 
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>


From HodgessE at uhd.edu  Mon Jul  6 14:47:31 2009
From: HodgessE at uhd.edu (Hodgess, Erin)
Date: Mon, 6 Jul 2009 14:47:31 -0500
Subject: [Swift-user] condor on communicado problem
References: <70A5AC06FDB5E54482D19E1C04CDFCF307C37193@BALI.uhd.campus><50b07b4b0907061216q21fd655btc87f4de3721b148c@mail.gmail.com><70A5AC06FDB5E54482D19E1C04CDFCF307C37196@BALI.uhd.campus>
	<50b07b4b0907061238r207d7312pcd31fe7e920d1383@mail.gmail.com>
Message-ID: <70A5AC06FDB5E54482D19E1C04CDFCF307C37198@BALI.uhd.campus>

They are both commented out.



Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodgesse at uhd.edu



-----Original Message-----
From: yecartes at gmail.com on behalf of Allan Espinosa
Sent: Mon 7/6/2009 2:38 PM
To: Hodgess, Erin
Cc: swift-user at ci.uchicago.edu
Subject: Re: [Swift-user] condor on communicado problem
 
do you have +osg-client or @osg in your ~/.soft file?

2009/7/6 Hodgess, Erin <HodgessE at uhd.edu>:
> I tried the following:
>
> [erin at communicado ~]$? voms-proxy-init
> -bash: voms-proxy-init: command not found
> [erin at communicado ~]$
>
> But this is not working either.
>
> Thanks,
> Erin
>
>
> Erin M. Hodgess, PhD
> Associate Professor
> Department of Computer and Mathematical Sciences
> University of Houston - Downtown
> mailto: hodgesse at uhd.edu
>
>
>
> -----Original Message-----
> From: yecartes at gmail.com on behalf of Allan Espinosa
> Sent: Mon 7/6/2009 2:16 PM
> To: Hodgess, Erin
> Cc: swift-user at ci.uchicago.edu
> Subject: Re: [Swift-user] condor on communicado problem
>
> Hi Erin,
>
> under what VO is your proxy when you accessed the resource?? each VO
> runs under a different userspace so you can't access Engage user
> directories if the vo with you current proxy is not using engage.
>
> to make sure you get the right VO, you can use voms-proxy-init
>
> -Allan
>
> 2009/7/6 Hodgess, Erin <HodgessE at uhd.edu>:
>> Hi again, Swift People:
>>
>> I'm on communicado, trying to run swift via condor (instead of gram).
>>
>> I generated my sites file and selected one location.
>>
>> Here is my output:
>>
>> [erin at communicado swift1]$ swift -tc.file tc.test.data -sites.file
>> sites1k.xml wc1a.swift
>> Swift svn swift-r2998 (swift modified locally) cog-r2427
>>
>> RunID: 20090706-1334-iq5phkm6
>> Progress:
>> Progress:? Selecting site:13? Initializing site shared directory:2
>> Progress:? Selecting site:10? Initializing site shared directory:5
>> Progress:? Selecting site:9? Initializing site shared directory:6
>> Progress:? Selecting site:8? Initializing site shared directory:7
>> Progress:? Selecting site:7? Initializing site shared directory:8
>> Progress:? Selecting site:6? Initializing site shared directory:9
>> Progress:? Selecting site:5? Initializing site shared directory:10
>> Progress:? Selecting site:4? Initializing site shared directory:11
>> Progress:? Selecting site:3? Initializing site shared directory:12
>> Progress:? Selecting site:2? Initializing site shared directory:13
>> Progress:? Selecting site:2? Initializing site shared directory:13
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Progress:? Initializing site shared directory:15
>> Execution failed:
>> ??????? Could not initialize shared directory on
>> Clemson-ciTeam_osgce.cs.clemson.edu
>> Caused by:
>> ??????? org.globus.cog.abstraction.impl.file.FileResourceException: Cannot
>> create directory /export/osg/data/engage/tmp
>> Caused by:
>> ??????? Server refused performing the request. Custom message: Server
>> refused creating directory (error code 1) [Nested exception message:
>> Custom
>> message: Unexpected reply: 500-Command failed :
>> globus_gridftp_server_file.c:globus_l_gfs_file_mkdir:554:
>> 500-System error in mkdir: Permission denied
>> 500-A system call failed: Permission denied
>> 500 End.]
>> [erin at communicado swift1]$
>>
>> This was using all of the values from the sites file.
>>
>> The directory that the sites file has is not present.
>>
>> Does anyone have any suggestions, please?
>>
>> Thanks in advance,
>> Erin
>>
>> Erin M. Hodgess, PhD
>> Associate Professor
>> Department of Computer and Mathematical Sciences
>> University of Houston - Downtown
>> mailto: hodgesse at uhd.edu
>
>



-- 
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090706/5ef29fba/attachment.html>

From aespinosa at cs.uchicago.edu  Fri Jul 10 14:29:11 2009
From: aespinosa at cs.uchicago.edu (Allan Espinosa)
Date: Fri, 10 Jul 2009 14:29:11 -0500
Subject: [Swift-user] single structs for ext mapper.
Message-ID: <50b07b4b0907101229s47e28772nc3b28c26b6ea74af@mail.gmail.com>

Hi

How do you use ext mapper for single structs? I used a $.xxx but swift
does not recognize that.

I have the workflow below based on 0755-ext-mapper.swift:

type messagefile;

type cols {
  messagefile l;
  messagefile r;
}

(messagefile t) write(string s) {
    app {
        echo s stdout=@filename(t);
    }
}

cols outfile <ext; exec="./0755-ext-mapper.sh">;

outfile.l = write("slot 0");
outfile.r = write("slot 1");

mapper file 0755-ext-mapper.sh:
echo "$.l 0755-ext-mapper.AAA.out"
echo "$.r 0755-ext-mapper.0.3.2.1.out"

session:
[aespinosa at communicado struct_ext]$ swift 0755-ext-mapper.swift
Swift svn swift-r2998 cog-r2410

RunID: 20090710-1424-49r93eqg
Progress:
Execution failed:
        java.lang.IllegalStateException: mapper.existing() returned a
path .$.l that it cannot subsequently map


thanks,
-Allan



-- 
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>


From benc at hawaga.org.uk  Sat Jul 11 16:46:57 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Sat, 11 Jul 2009 21:46:57 +0000 (GMT)
Subject: [Swift-user] single structs for ext mapper.
In-Reply-To: <50b07b4b0907101229s47e28772nc3b28c26b6ea74af@mail.gmail.com>
References: <50b07b4b0907101229s47e28772nc3b28c26b6ea74af@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0907112146380.32243@dildano.hawaga.org.uk>

> How do you use ext mapper for single structs? I used a $.xxx but swift
> does not recognize that.

Try specifying xxx without the $. on the front.

-- 



From benc at hawaga.org.uk  Mon Jul 13 02:27:15 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Mon, 13 Jul 2009 07:27:15 +0000 (GMT)
Subject: [Swift-user] single structs for ext mapper.
In-Reply-To: <Pine.LNX.4.64.0907112146380.32243@dildano.hawaga.org.uk>
References: <50b07b4b0907101229s47e28772nc3b28c26b6ea74af@mail.gmail.com>
	<Pine.LNX.4.64.0907112146380.32243@dildano.hawaga.org.uk>
Message-ID: <Pine.LNX.4.64.0907130726570.29145@dildano.hawaga.org.uk>



On Sat, 11 Jul 2009, Ben Clifford wrote:

> > How do you use ext mapper for single structs? I used a $.xxx but swift
> > does not recognize that.
> 
> Try specifying xxx without the $. on the front.

r3005 adds a test tests/language-behaviour/07554-ext-mapper-struct which 
demonstrates this.

-- 


From me.melly at gmail.com  Mon Jul 13 14:52:44 2009
From: me.melly at gmail.com (Melinda Chin)
Date: Mon, 13 Jul 2009 14:52:44 -0500
Subject: [Swift-user] [swift-plot-log]
Message-ID: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com>

I have some of the log files of the run we made today and I was trying out
swift-plot-log on them, however, I'm running to some slight problems.  It
seems like it runs just fine but at the end there is a make error.  I
thought it might be that I was suppose to run swift-plot-log from the bin
since it expects the libexec to to be in a specific place I tried that, but
it still has the same error.  Also I believe that problem of where to run
swift-plot-log was solved when I added in my .bashrc a PATH = ... and export
path of where my swift was located.

This is last error I see is: (only one too as far as I can tell)
=======================================================================================================================================
make: *** [kickstart.html] Error 1
rm karatasks.FILE_OPERATION.sorted-start.event
karatasks.JOB_SUBMISSION.Queue.sorted-start.event karatasks.last
dost.JOB_SUBMISSION.seenstates karatasks.FILE_TRANSFER.sorted-start.event
execute.last
karatasks.JOB_SUBMISSION.Queue.ei_SUBMISSION.Active.sorted-start.event
execute.seenstates karatasks.FILE_OPERATION.seenstates
karatasks.JOB_SUBMISSIOs.FILE_OPERATION.eip createdirset.event
karatasks.JOB_SUBMISSION.event dostageout.sorted-by-duration
karatasks.FILE_OPERATION.event execute2.seenstates
karatasks.FILE_TRANSFER.eip dostagein.sorted-by-duration
karatasks.JOB_SUBMISSIOks.JOB_SUBMISSION.Active.eip
karatasks.FILE_TRANSFER.event execute2.last execute.sorted-start.event
karatasks.JOB_SUks.JOB_SUBMISSION.Active.event
=======================================================================================================================================

Thank you,
Melinda Chin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090713/55e22719/attachment.html>

From benc at hawaga.org.uk  Mon Jul 13 15:03:10 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Mon, 13 Jul 2009 20:03:10 +0000 (GMT)
Subject: [Swift-user] [swift-plot-log]
In-Reply-To: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com>
References: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0907132002290.32243@dildano.hawaga.org.uk>


Can you paste about 10 lines or so before this message:



> make: *** [kickstart.html] Error 1


What is the commandline that you typed to generate this?

-- 



From me.melly at gmail.com  Mon Jul 13 15:08:22 2009
From: me.melly at gmail.com (Melinda Chin)
Date: Mon, 13 Jul 2009 15:08:22 -0500
Subject: [Swift-user] [swift-plot-log]
In-Reply-To: <Pine.LNX.4.64.0907132002290.32243@dildano.hawaga.org.uk>
References: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com>
	<Pine.LNX.4.64.0907132002290.32243@dildano.hawaga.org.uk>
Message-ID: <63cc32bc0907131308h2880e807h76412ae0c2aa1230@mail.gmail.com>

On Mon, Jul 13, 2009 at 3:03 PM, Ben Clifford <benc at hawaga.org.uk> wrote:

>
> Can you paste about 10 lines or so before this message:
>

cat execute2.transitions | sed 's/[^ ]* *[^ ]* \([^ ]*\).*/\1/' | sort |
uniq > execute2.seenstates
trail execute2
info-and-karajan-actives
grep: /*-info: No such file or directory
grep: /*-info: No such file or directory
(standard_in) 1: syntax error
(standard_in) 1: syntax error
Warning: empty y range [1:1], adjusting to [0.99:1.01]
kickstarts-to-event > kickstart.event
execution-summaries > execution-counts.txt
table-jobs-sites > jobs-sites.html
per-site-execute2-durations > site-duration.txt
cat execute2.transitions | swap-and-sort |last-transition-line  >
execute2.last
cat execute2.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort
| uniq -c > execute2.lastsummary
cat execute.transitions | swap-and-sort |last-transition-line  >
execute.last
cat execute.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort |
uniq -c > execute.lastsummary
cat execute2.event  | cut -f 5 -d ' ' | sort | uniq -c | sort | sed 's/^
*\(.*\) .*$/\1/' | uniq -c > jobs.retrycount.summary
cat execute.event | cut -f 5 -d ' ' | sort  | uniq -c > trname-summary
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//index.html.template >
index.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//execute2.html.template >
execute2.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//execute.html.template >
execute.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//karajan.html.template >
karajan.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//info.html.template >
info.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//assorted.html.template >
assorted.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//kickstart.html.template >
kickstart.html
m4:/opt/swift/r3003/bin/../libexec/log-processing//kickstart.html.template:23:
cannot open `kickstart.stats': No such file or directory

> make: *** [kickstart.html] Error 1
>
>
> What is the commandline that you typed to generate this?
>

* swift-plot-log radial_norefine-20090713-1120-mt712ud7.log*


>
> --
>
> Melinda Chin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090713/a0446b10/attachment.html>

From me.melly at gmail.com  Tue Jul 14 10:25:13 2009
From: me.melly at gmail.com (Melinda Chin)
Date: Tue, 14 Jul 2009 10:25:13 -0500
Subject: [Swift-user] [swift-plot-log]
In-Reply-To: <63cc32bc0907131308h2880e807h76412ae0c2aa1230@mail.gmail.com>
References: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com>
	<Pine.LNX.4.64.0907132002290.32243@dildano.hawaga.org.uk>
	<63cc32bc0907131308h2880e807h76412ae0c2aa1230@mail.gmail.com>
Message-ID: <63cc32bc0907140825j4f9cdc4tb271bebcd3135dce@mail.gmail.com>

I'm not too sure if this got through:

On Mon, Jul 13, 2009 at 3:03 PM, Ben Clifford <benc at hawaga.org.uk> wrote:

>
> Can you paste about 10 lines or so before this message:
>

*Here is about 25 lines:*
-----------------------------------------------------------------------------------------------------------------
cat execute2.transitions | sed 's/[^ ]* *[^ ]* \([^ ]*\).*/\1/' | sort |
uniq > execute2.seenstates
trail execute2
info-and-karajan-actives
grep: /*-info: No such file or directory
grep: /*-info: No such file or directory
(standard_in) 1: syntax error
(standard_in) 1: syntax error
Warning: empty y range [1:1], adjusting to [0.99:1.01]
kickstarts-to-event > kickstart.event
execution-summaries > execution-counts.txt
table-jobs-sites > jobs-sites.html
per-site-execute2-durations > site-duration.txt
cat execute2.transitions | swap-and-sort |last-transition-line  >
execute2.last
cat execute2.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort
| uniq -c > execute2.lastsummary
cat execute.transitions | swap-and-sort |last-transition-line  >
execute.last
cat execute.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort |
uniq -c > execute.lastsummary
cat execute2.event  | cut -f 5 -d ' ' | sort | uniq -c | sort | sed 's/^
*\(.*\) .*$/\1/' | uniq -c > jobs.retrycount.summary
cat execute.event | cut -f 5 -d ' ' | sort  | uniq -c > trname-summary
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//index.html.template >
index.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//execute2.html.template >
execute2.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//execute.html.template >
execute.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//karajan.html.template >
karajan.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//info.html.template >
info.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//assorted.html.template >
assorted.html
m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
/opt/swift/r3003/bin/../libexec/log-processing//kickstart.html.template >
kickstart.html
m4:/opt/swift/r3003/bin/../libexec/log-processing//kickstart.html.template:23:
cannot open `kickstart.stats': No such file or directory
-----------------------------------------------------------------------------------------------------------------

What is the commandline that you typed to generate this?
>

I used* swift-plot-log radial_norefine-20090713-1120-mt712ud7.log*

Thanks,
Melinda Chin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090714/0b3de1ab/attachment.html>

From aespinosa at cs.uchicago.edu  Tue Jul 14 12:45:08 2009
From: aespinosa at cs.uchicago.edu (Allan Espinosa)
Date: Tue, 14 Jul 2009 12:45:08 -0500
Subject: struct of structs via ext mapper(was Re: [Swift-user] single structs 
	for ext mapper.)
Message-ID: <50b07b4b0907141045x72be74edvbda14a3ecef1cce0@mail.gmail.com>

got it. thanks ben!

now based on this, i tried out a struct of structs.  my swift session
(latest on cog svn and swift svn) reports as follows:

RunID: testing
Progress:
Progress:  Initializing site shared directory:1  Failed:1
Execution failed:
        Mapper failed to map org.griphyn.vdl.mapping.DataNode
identifier tag:benc at ci.uchicago.edu,2008:swift:dataset:20090714-1343-6lzjg014:720000000039
type AmplFilter with no value at dataset=res path=.out (not closed)

my instance_mapper.sh:
#!/bin/bash

while getopts ":i:" options; do
  case $options in
    i) export instance=$OPTARG ;;
    *) exit 1;;
  esac
done

echo "expend result/$instance/expend.dat";
echo "limits result/$instance/limits.dat";
echo "price result/$instance/price.dat";
echo "ratio result/$instance/ratio.dat";
echo "solve result/$instance/solve.dat";

echo "ofile result/$instance/stdout";

echo "out.expend_out result/$instance/expend.out";
echo "out.price_out result/$instance/price.out";
echo "out.ratio_out result/$instance/ratio.out";



here is the workflow i was working on:

type Template;
type AmplIn;
type StdOut;

type AmplCmd {
  Template temp;
  AmplIn mod;
  AmplIn process;
  AmplIn output;
  AmplIn so;
  AmplIn tree;
}

type ExpendDat;
type LimitsDat;
type PriceDat;
type RatioDat;
type SolveDat;

type ExpendOut;
type PriceOut;
type RatioOut;

type AmplFilter {
  ExpendOut expend_out;
  PriceOut price_out;
  RatioOut ratio_out;
}

type AmplResult {
  ExpendDat expend;
  LimitsDat limits;
  PriceDat price;
  RatioDat ratio;
  SolveDat solve;
  StdOut ofile;
  AmplFilter out;
}

app (AmplResult result) run_ampl (string instanceID, AmplCmd cmd)
{
  run_ampl instanceID @filename(cmd.temp)
      @filename(cmd.mod) @filename(cmd.process)
      @filename(cmd.output) @filename(cmd.so) @filename(cmd.tree)
      stdout=@filename(result.ofile);
}
AmplCmd const_cmd <ext;exec="./cmd_mapper.sh">;

int runs[]=[2001:2002];
foreach i in runs {

  string instanceID = @strcat("run", i);

  AmplResult res <ext;exec="./instance_mapper.sh", i=instanceID>;
  res = run_ampl(instanceID, const_cmd);
}




2009/7/11 Ben Clifford <benc at hawaga.org.uk>:
>> How do you use ext mapper for single structs? I used a $.xxx but swift
>> does not recognize that.
>
> Try specifying xxx without the $. on the front.
>
> --
>
>
>



-- 
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>


From benc at hawaga.org.uk  Tue Jul 14 17:05:57 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Tue, 14 Jul 2009 22:05:57 +0000 (GMT)
Subject: struct of structs via ext mapper(was Re: [Swift-user] single
	structs  for ext mapper.)
In-Reply-To: <50b07b4b0907141045x72be74edvbda14a3ecef1cce0@mail.gmail.com>
References: <50b07b4b0907141045x72be74edvbda14a3ecef1cce0@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0907142204540.29145@dildano.hawaga.org.uk>

looks like its trying to map a path for the out structure itself, which 
probably shouldn't be happening.

as a workaround you might try adding a made-up filename for out itself, 
in addition to the members of out. no idea if that will work or not but 
worth a try.

-- 


From aespinosa at cs.uchicago.edu  Tue Jul 14 17:13:03 2009
From: aespinosa at cs.uchicago.edu (Allan Espinosa)
Date: Tue, 14 Jul 2009 17:13:03 -0500
Subject: struct of structs via ext mapper(was Re: [Swift-user] single 
	structs for ext mapper.)
In-Reply-To: <Pine.LNX.4.64.0907142204540.29145@dildano.hawaga.org.uk>
References: <50b07b4b0907141045x72be74edvbda14a3ecef1cce0@mail.gmail.com>
	<Pine.LNX.4.64.0907142204540.29145@dildano.hawaga.org.uk>
Message-ID: <50b07b4b0907141513g1a98a52wbe0ff62520642727@mail.gmail.com>

i see. so this mean that in my will need to generate this made-up file
too right?  the swift -dryrun works.  will test it out with the real
run now.

thanks,
-Allan

2009/7/14 Ben Clifford <benc at hawaga.org.uk>:
> looks like its trying to map a path for the out structure itself, which
> probably shouldn't be happening.
>
> as a workaround you might try adding a made-up filename for out itself,
> in addition to the members of out. no idea if that will work or not but
> worth a try.
>


-- 
Allan M. Espinosa <http://allan.88-mph.net/blog>
PhD student, Computer Science
University of Chicago <http://people.cs.uchicago.edu/~aespinosa>


From benc at hawaga.org.uk  Tue Jul 14 17:18:43 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Tue, 14 Jul 2009 22:18:43 +0000 (GMT)
Subject: struct of structs via ext mapper(was Re: [Swift-user] single 
	structs for ext mapper.)
In-Reply-To: <50b07b4b0907141513g1a98a52wbe0ff62520642727@mail.gmail.com>
References: <50b07b4b0907141045x72be74edvbda14a3ecef1cce0@mail.gmail.com> 
	<Pine.LNX.4.64.0907142204540.29145@dildano.hawaga.org.uk>
	<50b07b4b0907141513g1a98a52wbe0ff62520642727@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0907142218030.29145@dildano.hawaga.org.uk>


On Tue, 14 Jul 2009, Allan Espinosa wrote:

> i see. so this mean that in my will need to generate this made-up file
> too right? 

no idea. probably not.

if you look at the full stack trace in the logs you might get more of a 
clue of which piece of code is trying to map this filename when it 
shouldn't be doing so.

-- 



From benc at hawaga.org.uk  Wed Jul 15 09:03:36 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Wed, 15 Jul 2009 14:03:36 +0000 (GMT)
Subject: struct of structs via ext mapper(was Re: [Swift-user] single
	structs  for ext mapper.)
In-Reply-To: <50b07b4b0907141045x72be74edvbda14a3ecef1cce0@mail.gmail.com>
References: <50b07b4b0907141045x72be74edvbda14a3ecef1cce0@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0907151403090.19679@dildano.hawaga.org.uk>


On Tue, 14 Jul 2009, Allan Espinosa wrote:

> now based on this, i tried out a struct of structs.  my swift session
> (latest on cog svn and swift svn) reports as follows:

try r3011 without the workaround that I suggested.

-- 



From benc at hawaga.org.uk  Wed Jul 15 09:14:44 2009
From: benc at hawaga.org.uk (Ben Clifford)
Date: Wed, 15 Jul 2009 14:14:44 +0000 (GMT)
Subject: [Swift-user] [swift-plot-log]
In-Reply-To: <63cc32bc0907131307x11766d58t9717a778b81d0a75@mail.gmail.com>
References: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com> 
	<Pine.LNX.4.64.0907132002290.32243@dildano.hawaga.org.uk>
	<63cc32bc0907131307x11766d58t9717a778b81d0a75@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0907151414110.29145@dildano.hawaga.org.uk>


looks like the generation dependencies are not correctly defined in 
swift-plot-log but that usually this does not cause a problem. I will have 
a poke around and see if it is easy to fix.

On Mon, 13 Jul 2009, Melinda Chin wrote:

> On Mon, Jul 13, 2009 at 3:03 PM, Ben Clifford <benc at hawaga.org.uk> wrote:
> 
> >
> > Can you paste about 10 lines or so before this message:
> >
> 
> cat execute2.transitions | sed 's/[^ ]* *[^ ]* \([^ ]*\).*/\1/' | sort |
> uniq > execute2.seenstates
> trail execute2
> info-and-karajan-actives
> grep: /*-info: No such file or directory
> grep: /*-info: No such file or directory
> (standard_in) 1: syntax error
> (standard_in) 1: syntax error
> Warning: empty y range [1:1], adjusting to [0.99:1.01]
> kickstarts-to-event > kickstart.event
> execution-summaries > execution-counts.txt
> table-jobs-sites > jobs-sites.html
> per-site-execute2-durations > site-duration.txt
> cat execute2.transitions | swap-and-sort |last-transition-line  >
> execute2.last
> cat execute2.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort
> | uniq -c > execute2.lastsummary
> cat execute.transitions | swap-and-sort |last-transition-line  >
> execute.last
> cat execute.last | sed 's/^\([^ ]*\) \([^ ]*\) \([^ ]*\)\(.*\)/\3/' | sort |
> uniq -c > execute.lastsummary
> cat execute2.event  | cut -f 5 -d ' ' | sort | uniq -c | sort | sed 's/^
> *\(.*\) .*$/\1/' | uniq -c > jobs.retrycount.summary
> cat execute.event | cut -f 5 -d ' ' | sort  | uniq -c > trname-summary
> m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
> /opt/swift/r3003/bin/../libexec/log-processing//index.html.template >
> index.html
> m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
> /opt/swift/r3003/bin/../libexec/log-processing//execute2.html.template >
> execute2.html
> m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
> /opt/swift/r3003/bin/../libexec/log-processing//execute.html.template >
> execute.html
> m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
> /opt/swift/r3003/bin/../libexec/log-processing//karajan.html.template >
> karajan.html
> m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
> /opt/swift/r3003/bin/../libexec/log-processing//info.html.template >
> info.html
> m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
> /opt/swift/r3003/bin/../libexec/log-processing//assorted.html.template >
> assorted.html
> m4 -I/opt/swift/r3003/bin/../libexec/log-processing/
> /opt/swift/r3003/bin/../libexec/log-processing//kickstart.html.template >
> kickstart.html
> m4:/opt/swift/r3003/bin/../libexec/log-processing//kickstart.html.template:23:
> cannot open `kickstart.stats': No such file or directory
> 
> > make: *** [kickstart.html] Error 1
> >
> >
> > What is the commandline that you typed to generate this?
> >
> 
> * swift-plot-log radial_norefine-20090713-1120-mt712ud7.log*
> 
> 
> >
> > --
> >
> > Melinda Chin
> 


From iraicu at cs.uchicago.edu  Wed Jul 15 14:04:49 2009
From: iraicu at cs.uchicago.edu (Ioan Raicu)
Date: Wed, 15 Jul 2009 14:04:49 -0500
Subject: [Swift-user] CFP: 2nd ACM Workshop on Many-Task Computing on Grids
 and Supercomputers (MTAGS09) at Supercomputing 2009
Message-ID: <4A5E2851.1000500@cs.uchicago.edu>

Call for Papers

--------------------------------------------------------------------------------------- 

The 2nd ACM Workshop on Many-Task Computing on Grids and Supercomputers 
(MTAGS) 2009
http://dsl.cs.uchicago.edu/MTAGS09/
--------------------------------------------------------------------------------------- 

November 16th, 2009
Portland, Oregon, USA

Co-located with with IEEE/ACM International Conference for
High Performance Computing, Networking, Storage and Analysis (SC09)

======================================================================================= 

The 2nd workshop on Many-Task Computing on Grids and Supercomputers 
(MTAGS) will
provide the scientific community a dedicated forum for presenting new 
research,
development, and deployment efforts of loosely coupled large scale 
applications on
large scale clusters, Grids, Supercomputers, and Cloud Computing 
infrastructure.
Many-task computing (MTC), the theme of the workshop encompasses loosely 
coupled
applications, which are generally composed of many tasks (both 
independent and
dependent tasks) to achieve some larger application goal.  This workshop 
will cover
challenges that can hamper efficiency and utilization in running 
applications on
large-scale systems, such as local resource manager scalability and 
granularity,
efficient utilization of the raw hardware, parallel file system 
contention and
scalability, reliability at scale, and application scalability. We 
welcome paper
submissions on all topics related to MTC on large scale systems.  Papers 
will be
peer-reviewed, and accepted papers will be published in the workshop 
proceedings as
part of the ACM digital library.  The workshop will be co-located with the
IEEE/ACM Supercomputing 2009 Conference in Portland Oregon on November 
16th, 2009.
For more information, please visithttp://dsl.cs.uchicago.edu/MTAGS09/.

Scope
--------------------------------------------------------------------------------------- 

This workshop will focus on the ability to manage and execute large 
scale applications
on today's largest clusters, Grids, and Supercomputers. Clusters with 
50K+ processor
cores are beginning to come online (i.e. TACC Sun Constellation System - 
Ranger), Grids
(i.e. TeraGrid) with a dozen sites and 100K+ processors, and 
supercomputers with 160K
processors (i.e. IBM BlueGene/P). Large clusters and supercomputers have 
traditionally
been high performance computing (HPC) systems, as they are efficient at 
executing
tightly coupled parallel jobs within a particular machine with low-latency
interconnects; the applications typically use message passing interface 
(MPI) to
achieve the needed inter-process communication. On the other hand, Grids 
have been the
preferred platform for more loosely coupled applications that tend to be 
managed and
executed through workflow systems. In contrast to HPC (tightly coupled 
applications),
these loosely coupled applications make up a new class of applications 
as what we call
Many-Task Computing (MTC). MTC systems generally involve the execution 
of independent,
sequential jobs that can be individually scheduled on many different 
computing
resources across multiple administrative boundaries. MTC systems 
typically achieve this
using various grid computing technologies and techniques, and often 
times use files to
achieve the inter-process communication as alternative communication 
mechanisms than
MPI. MTC is reminiscent to High Throughput Computing (HTC); however, MTC 
differs from
HTC in the emphasis of using many computing resources over short periods 
of time to
accomplish many computational tasks, where the primary metrics are 
measured in seconds
(e.g. FLOPS, tasks/sec, MB/s I/O rates). HTC on the other hand requires 
large amounts
of computing for longer times (months and years, rather than hours and 
days, and are
generally measured in operations per month).

Today's existing HPC systems are a viable platform to host MTC 
applications. However,
some challenges arise in large scale applications when run on large 
scale systems,
which can hamper the efficiency and utilization of these large scale 
systems.  These
challenges vary from local resource manager scalability and granularity, 
efficient
utilization of the raw hardware, shared file system contention and 
scalability,
reliability at scale, application scalability, and understanding the 
limitations of the
HPC systems in order to identify good candidate MTC applications. 
Furthermore, the MTC
paradigm can be naturally applied to the emerging Cloud Computing 
paradigm due to its
loosely coupled nature, which is being adopted by industry as the next 
wave of
technological advancement to reduce operational costs while improving 
efficiencies in
large scale infrastructures.

For an interesting discussion in a blog by Ian Foster on the difference 
between MTC and
HTC, please see his blog 
athttp://ianfoster.typepad.com/blog/2008/07/many-tasks-comp.html.
We also published two papers that are highly relevant to this workshop. 
One paper is
titled "Toward Loosely Coupled Programming on Petascale Systems", and 
was published in
SC08; the second paper is titled "Many-Task Computing for Grids and 
Supercomputers",
which was published in MTAGS08. Furthermore, to see last year's workshop 
program agenda,
and accepted papers and presentations, please 
seehttp://dsl.cs.uchicago.edu/MTAGS08/.
For more information, please visithttp://dsl.cs.uchicago.edu/MTAGS09/.

Topics
--------------------------------------------------------------------------------------- 

MTAGS 2008 topics of interest include, but are not limited to:
*    Compute Resource Management in large scale clusters, large Grids, 
Supercomputers,
        or Cloud Computing infrastructure
    o    Scheduling
    o    Job execution frameworks
    o    Local resource manager extensions
    o    Performance evaluation of resource managers in use on large 
scale systems
    o    Challenges and opportunities in running many-task workloads on 
HPC systems
    o    Challenges and opportunities in running many-task workloads
                on Cloud Computing infrastructure
*    Data Management in large scale Grid and Supercomputer environments:
    o    Data-Aware Scheduling
    o    Parallel File System performance and scalability in large 
deployments
    o    Distributed file systems
    o    Data caching frameworks and techniques
*    Large-Scale Workflow Systems
    o    Workflow system performance and scalability analysis
    o    Scalability of workflow systems
    o    Workflow infrastructure and e-Science middleware
    o    Programming Paradigms and Models
*    Large-Scale Many-Task Applications
    o    Large-scale many-task applications
    o    Large-scale many-task data-intensive applications
    o    Large-scale high throughput computing (HTC) applications
    o    Quasi-supercomputing applications, deployments, and experiences

Paper Submission and Publication
--------------------------------------------------------------------------------------- 

Authors are invited to submit papers with unpublished, original work of 
not more than
10 pages of double column text using single spaced 10 point size on 8.5 
x 11 inch pages,
as per ACM 8.5 x 11 manuscript guidelines
(http://www.acm.org/publications/instructions_for_proceedings_volumes); 
document
templates can be found 
athttp://www.acm.org/sigs/publications/proceedings-templates.
A 250 word abstract (PDF format) must be submitted online at
https://cmt.research.microsoft.com/MTAGS2009/  before the deadline of 
August 1st, 2009
at 11:59PM PST; the final 10 page papers in PDF format will be due on 
September 1st,
2009 at 11:59PM PST. Papers will be peer-reviewed, and accepted papers 
will be
published in the workshop proceedings as part of the ACM digital 
library. Notifications
of the paper decisions will be sent out by October 1st, 2009. Selected 
excellent work
will be invited to submit extended versions of the workshop paper to the
IEEE Transactions on Parallel and Distributed Systems (TPDS) Journal, 
Special Issue on
Many-Task Computing (due December 21st, 2009); for more information 
about this journal
special issue, please visithttp://dsl.cs.uchicago.edu/TPDS_MTC/.  
Submission implies
the willingness of at least one of the authors to register and present 
the paper. For
more information, please visithttp://dsl.cs.uchicago.edu/MTAGS09/.

Important Dates
--------------------------------------------------------------------------------------- 

*    Abstract Due:               August 1st, 2009
*    Papers Due:                 September 1st, 2009
*    Notification of Acceptance: October 1st, 2009
*    Camera Ready Papers Due:    November 1st, 2009
*    Workshop Date:              November 16th, 2009



Committee Members
--------------------------------------------------------------------------------------- 

Workshop Chairs
*    Ioan Raicu, University of Chicago
*    Ian Foster, University of Chicago&  Argonne National Laboratory
*    Yong Zhao, Microsoft

Technical Committee (confirmed)
*       David Abramson, Monash University, Australia
*       Pete Beckman, Argonne National Laboratory, USA
*       Peter Dinda, Northwestern University, USA
*       Ian Foster, University of Chicago&  Argonne National Laboratory, 
USA
*       Bob Grossman, University of Illinois at Chicago, USA
*       Indranil Gupta, University of Illinois at Urbana Champaign, USA
*       Alexandru Iosup, Delft University of Technology, Netherlands
*       Kamil Iskra, Argonne National Laboratory, USA
*       Chuang Liu, Ask.com, USA
*       Zhou Lei, Shanghai University, China
*       Shiyong Lu, Wayne State University, USA
*       Reagan Moore, University of North Carolina at Chapel Hill, USA
*       Marlon Pierce, Indiana University, USA
*       Ioan Raicu, University of Chicago, USA
*       Matei Ripeanu, University of British Columbia, Canada
*       David Swanson, University of Nebraska, USA
*       Greg Thain, Univeristy of Wisconsin, USA
*       Matthew Woitaszek, The University Corporation for Atmospheric 
Research, USA
*       Mike Wilde, University of Chicago&  Argonne National Laboratory, 
USA
*       Sherali Zeadally, University of the District of Columbia, USA
*       Yong Zhao, Microsoft, USA



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090715/b665fc5a/attachment.html>

From wwj at ci.uchicago.edu  Thu Jul 16 15:37:57 2009
From: wwj at ci.uchicago.edu (Wenjun Wu)
Date: Thu, 16 Jul 2009 15:37:57 -0500
Subject: [Swift-user] swift RuntimeStats
In-Reply-To: <Pine.LNX.4.64.0907151414110.29145@dildano.hawaga.org.uk>
References: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com>
	<Pine.LNX.4.64.0907132002290.32243@dildano.hawaga.org.uk>	<63cc32bc0907131307x11766d58t9717a778b81d0a75@mail.gmail.com>
	<Pine.LNX.4.64.0907151414110.29145@dildano.hawaga.org.uk>
Message-ID: <4A5F8FA5.1000705@ci.uchicago.edu>

Hello,
    I noticed that RuntimeStats class ( 
org.griphyn.vdl.karajan.lib.RuntimeStats ) has been there for a while.
    Is it possible for swift users to get the information from this 
class and check the progress of their workflows
    in real time?

   Thanks,

Wenjun


From hategan at mcs.anl.gov  Thu Jul 16 21:27:24 2009
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Thu, 16 Jul 2009 21:27:24 -0500
Subject: [Swift-user] swift RuntimeStats
In-Reply-To: <4A5F8FA5.1000705@ci.uchicago.edu>
References: <63cc32bc0907131252oe4c4952ra08246c5c7ee6bb6@mail.gmail.com>
	<Pine.LNX.4.64.0907132002290.32243@dildano.hawaga.org.uk>
	<63cc32bc0907131307x11766d58t9717a778b81d0a75@mail.gmail.com>
	<Pine.LNX.4.64.0907151414110.29145@dildano.hawaga.org.uk>
	<4A5F8FA5.1000705@ci.uchicago.edu>
Message-ID: <1247797644.10075.23.camel@localhost>

On Thu, 2009-07-16 at 15:37 -0500, Wenjun Wu wrote:
> Hello,
>     I noticed that RuntimeStats class ( 
> org.griphyn.vdl.karajan.lib.RuntimeStats ) has been there for a while.
>     Is it possible for swift users to get the information from this 
> class and check the progress of their workflows
>     in real time?

When swift prints information on the console (things like "Initializing
site: 1, Submitting: 12, Finished successfully: Millions!"), that's what
RuntimeStats is used for.

I don't think it's straightforward to extract information from that
class programmatically, but you could try.

There is an unfinished monitoring package. Right now there is a text
interface for it (start swift with the "-tui" argument). Since it's
intended to be able to have multiple types of interfaces, there is an
abstract part to it that collects relevant information from various
places in swift. It's mostly based on intercepting log data. 

If you're interested in using that, some pointers are:
?MonitorAppender.java and Loader.java line 438 (where it's being set
up).

Mihael



From hategan at mcs.anl.gov  Thu Jul 23 13:20:16 2009
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Thu, 23 Jul 2009 13:20:16 -0500
Subject: [Swift-user] Re: [Swift-devel] errors from HNL machines/swift
In-Reply-To: <b445e57f0907231113m1d7963a9l7b6c2c629aeae42a@mail.gmail.com>
References: <b445e57f0907231113m1d7963a9l7b6c2c629aeae42a@mail.gmail.com>
Message-ID: <1248373216.28628.1.camel@localhost>

There's a JVM dump, namely ?hs_err_pid32310.log. I'd like to see that.

Otherwise it seems related to this:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6390352

You could try a newer JVM and see if the problem persists.

On Thu, 2009-07-23 at 13:13 -0500, Michael Andric wrote:
> HI Support, Swift dev, anyone else reading,
> 
> I keep getting this crash on swift jobs submitted from HNL machines
> (both andrew.bsd.uchicago.edu and gwynn.bsd.uchicago.edu).  These
> happen for different workflows, involving different processes.  I am
> totally in the dark as to what this error is referring to as well as
> to what may be causing it.  This crash has occurred on workflows that
> have just gone 'Active' as well as on workflows that were running for
> hours before crashing. 
> 
> 
> Below is the error message.  The log file is too big to attach but can
> be found here:
> /gpfs/pads/fmri/cnari/swift/projects/andric/peakfit_pilots/PK2/turnpointAnalysis/tpChiSqTests-20090723-1113-na2cuboc.log
> from one of the HNL machines (e.g., gwynn.bsd.uchicago.edu)
> 
> 
> Any insight is hugely appreciated - like i said, i don't even know
> what to debug b/c i don't know what the error is referring to.  
> Michael
> 
> 
> 
> 
> 
> 
> 
> Progress:  Submitted:11 Active:1
> Progress:  Active:10 Stage out:2
> #
> # An unexpected error has been detected by HotSpot Virtual Machine:
> #
> #  SIGBUS (0x7) at pc=0xb75b9a62, pid=32310, tid=2949090208
> #
> # Java VM: Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode,
> sharing)
> # Problematic frame:
> # C  [libzip.so+0xfa62]
> #
> # An error report file with more information is saved as
> hs_err_pid32310.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp
> #
> /gpfs/pads/fmri/apps/swift/bin/swift: line 100: 32310 Aborted
> java -Xmx2048M
> -Djava.endorsed.dirs=/gpfs/pads/fmri/apps/swift/bin/../lib/endorsed
> -DUID=1309 -DGLOBUS_TCP_PORT_RANGE=50000,51000
> -DGLOBUS_HOSTNAME=andrew.bsd.uchicago.edu -DCOG_INSTALL_PATH=/gpfs/pads/fmri/apps/swift/bin/.. -Dvds.home=/gpfs/pads/fmri/apps/swift/bin/.. -Dswift.home=/gpfs/pads/fmri/apps/swift/bin/.. -Djava.security.egd=file:///dev/urandom -Xmx1024m -classpath /gpfs/pads/fmri/apps/swift/bin/../etc:/gpfs/pads/fmri/apps/swift/bin/../libexec:/gpfs/pads/fmri/apps/swift/bin/../lib/addressing-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/ant.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/antlr-2.7.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/backport-util-concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/castor-0.9.6.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/coaster-bootstrap.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-abstraction-common-2.3.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-grapheditor-0.47.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-jglobus-dev-080222.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-karajan-0.36-dev.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-clref-gt4_0_0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-coaster-0.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-dcache-0.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt2-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt4_0_0-2.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-local-2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-localscheduler-0.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-ssh-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-webdav-2.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-resources-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-swift-svn.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-trap-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-util-0.92.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commonj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-beanutils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-collections-3.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-digester.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-discovery.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-httpclient.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-logging-1.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix32.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix-asn1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_mds_aggregator_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rft_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-client.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-utils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gvds.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-common-0.2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-core-0.2.2-patched.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-regexp-1.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-slide-webdavlib-2.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jaxrpc.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jce-jdk13-131.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jgss.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jsr173_1.0_api.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jug-lgpl-2.0.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/junit.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/log4j-1.2.8.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-common.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-factory.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-java.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-resources.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/opensaml.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/puretls.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/resolver.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/saaj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/stringtemplate.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/vdldefinitions.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsdl4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_index_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_usefulrp_schema_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_provider_jce.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_tools.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wss4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xalan.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean_xpath.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xercesImpl.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xml-apis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xmlsec.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xpp3-1.1.3.4d_b4_min.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xstream-1.1.1-patched.jar: org.griphyn.vdl.karajan.Loader 'tpChiSqTests.swift' '-sites.file' '/gpfs/pads/fmri/cnari_svn/config/coaster_ranger.xml' '-user=andric'
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel



From hategan at mcs.anl.gov  Thu Jul 23 15:33:23 2009
From: hategan at mcs.anl.gov (Mihael Hategan)
Date: Thu, 23 Jul 2009 15:33:23 -0500
Subject: [Swift-user] Re: [Swift-devel] errors from HNL machines/swift
In-Reply-To: <b445e57f0907231323p3c91bc14hfa7c1174ce076cef@mail.gmail.com>
References: <b445e57f0907231113m1d7963a9l7b6c2c629aeae42a@mail.gmail.com>
	<1248373216.28628.1.camel@localhost>
	<b445e57f0907231323p3c91bc14hfa7c1174ce076cef@mail.gmail.com>
Message-ID: <1248381203.32020.0.camel@localhost>

Can't help you much there. It seems to be a bug in the JVM. Again, I'd
try other versions of java.

On Thu, 2009-07-23 at 15:23 -0500, Michael Andric wrote:
> there are a couple here:  andrew.bsd.uchicago.edu:/tmp/hs*.log
> 
> On Thu, Jul 23, 2009 at 1:20 PM, Mihael Hategan <hategan at mcs.anl.gov>
> wrote:
>         There's a JVM dump, namely ?hs_err_pid32310.log. I'd like to
>         see that.
>         
>         Otherwise it seems related to this:
>         http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6390352
>         
>         You could try a newer JVM and see if the problem persists.
>         
>         
>         On Thu, 2009-07-23 at 13:13 -0500, Michael Andric wrote:
>         > HI Support, Swift dev, anyone else reading,
>         >
>         > I keep getting this crash on swift jobs submitted from HNL
>         machines
>         > (both andrew.bsd.uchicago.edu and gwynn.bsd.uchicago.edu).
>          These
>         > happen for different workflows, involving different
>         processes.  I am
>         > totally in the dark as to what this error is referring to as
>         well as
>         > to what may be causing it.  This crash has occurred on
>         workflows that
>         > have just gone 'Active' as well as on workflows that were
>         running for
>         > hours before crashing.
>         >
>         >
>         > Below is the error message.  The log file is too big to
>         attach but can
>         > be found here:
>         > /gpfs/pads/fmri/cnari/swift/projects/andric/peakfit_pilots/PK2/turnpointAnalysis/tpChiSqTests-20090723-1113-na2cuboc.log
>         > from one of the HNL machines (e.g., gwynn.bsd.uchicago.edu)
>         >
>         >
>         > Any insight is hugely appreciated - like i said, i don't
>         even know
>         > what to debug b/c i don't know what the error is referring
>         to.
>         > Michael
>         >
>         >
>         >
>         >
>         >
>         >
>         >
>         > Progress:  Submitted:11 Active:1
>         > Progress:  Active:10 Stage out:2
>         > #
>         > # An unexpected error has been detected by HotSpot Virtual
>         Machine:
>         > #
>         > #  SIGBUS (0x7) at pc=0xb75b9a62, pid=32310, tid=2949090208
>         > #
>         > # Java VM: Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed
>         mode,
>         > sharing)
>         > # Problematic frame:
>         > # C  [libzip.so+0xfa62]
>         > #
>         > # An error report file with more information is saved as
>         > hs_err_pid32310.log
>         > #
>         > # If you would like to submit a bug report, please visit:
>         > #   http://java.sun.com/webapps/bugreport/crash.jsp
>         > #
>         > /gpfs/pads/fmri/apps/swift/bin/swift: line 100: 32310
>         Aborted
>         > java -Xmx2048M
>         >
>         -Djava.endorsed.dirs=/gpfs/pads/fmri/apps/swift/bin/../lib/endorsed
>         > -DUID=1309 -DGLOBUS_TCP_PORT_RANGE=50000,51000
>         > -DGLOBUS_HOSTNAME=andrew.bsd.uchicago.edu
>         -DCOG_INSTALL_PATH=/gpfs/pads/fmri/apps/swift/bin/..
>         -Dvds.home=/gpfs/pads/fmri/apps/swift/bin/..
>         -Dswift.home=/gpfs/pads/fmri/apps/swift/bin/..
>         -Djava.security.egd=file:///dev/urandom -Xmx1024m
>         -classpath /gpfs/pads/fmri/apps/swift/bin/../etc:/gpfs/pads/fmri/apps/swift/bin/../libexec:/gpfs/pads/fmri/apps/swift/bin/../lib/addressing-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/ant.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/antlr-2.7.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/backport-util-concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/castor-0.9.6.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/coaster-bootstrap.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-abstraction-common-2.3.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-grapheditor-0.47.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-jglobus-dev-080222.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-karajan-0.36-dev.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-clref-gt4_0_0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-coaster-0.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-dcache-0.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt2-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt4_0_0-2.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-local-2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-localscheduler-0.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-ssh-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-webdav-2.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-resources-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-swift-svn.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-trap-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-util-0.92.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commonj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-beanutils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-collections-3.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-digester.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-discovery.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-httpclient.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-logging-1.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix32.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix-asn1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_mds_aggregator_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rft_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-client.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-utils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gvds.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-common-0.2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-core-0.2.2-patched.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-regexp-1.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-slide-webdavlib-2.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jaxrpc.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jce-jdk13-131.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jgss.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jsr173_1.0_api.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jug-lgpl-2.0.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/junit.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/log4j-1.2.8.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-common.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-factory.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-java.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-resources.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/opensaml.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/puretls.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/resolver.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/saaj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/stringtemplate.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/vdldefinitions.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsdl4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_index_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_usefulrp_schema_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_provider_jce.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_tools.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wss4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xalan.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean_xpath.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xercesImpl.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xml-apis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xmlsec.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xpp3-1.1.3.4d_b4_min.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xstream-1.1.1-patched.jar: org.griphyn.vdl.karajan.Loader 'tpChiSqTests.swift' '-sites.file' '/gpfs/pads/fmri/cnari_svn/config/coaster_ranger.xml' '-user=andric'
>         
>         > _______________________________________________
>         > Swift-devel mailing list
>         > Swift-devel at ci.uchicago.edu
>         > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>         
> 



From jamalphd at gmail.com  Sun Jul 26 14:50:02 2009
From: jamalphd at gmail.com (J A)
Date: Sun, 26 Jul 2009 15:50:02 -0400
Subject: [Swift-user] XDTM
Message-ID: <b19f113c0907261250q3270e8c5j469850b8401b4fa0@mail.gmail.com>

Hi All:

Can any one direct me to a source with more examples/explanation on how XDTM
is working/implemented?

Thanks,
Jamal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090726/9e94d62b/attachment.html>

From wilde at mcs.anl.gov  Sun Jul 26 18:09:59 2009
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Sun, 26 Jul 2009 18:09:59 -0500
Subject: [Swift-user] XDTM
In-Reply-To: <b19f113c0907261250q3270e8c5j469850b8401b4fa0@mail.gmail.com>
References: <b19f113c0907261250q3270e8c5j469850b8401b4fa0@mail.gmail.com>
Message-ID: <4A6CE247.4010105@mcs.anl.gov>

Jamal,

As Swift evolved from its early prototypes to a more mature system, the 
notion of XDTM evolved to one of mapping between filesystem-based 
structures and Swift in-memory data structures (ie, scalars, arrays, and 
structures, which can be nested and typed).

This is best seen by looking at the "external" mapper, which allows a 
user to map a dataset using any external program (typically a script) 
that returns the members of the dataset as a two-column list: the Swift 
variable reference, and the external file or URI.

See the user guide section on the external mapper:

  http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.ext_mapper
(but the example in the user guide doesn't show the power of mapping to 
nested structures).

In other words, it still has the flavor of XDTM, but without any XML 
being visible to the user. It meets the same need but is easier to use 
and explain.

- Mike

On 7/26/09 2:50 PM, J A wrote:
> Hi All:
>  
> Can any one direct me to a source with more examples/explanation on 
> how XDTM is working/implemented?
>  
> Thanks,
> Jamal
>  
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user


From wilde at mcs.anl.gov  Sun Jul 26 20:53:44 2009
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Sun, 26 Jul 2009 20:53:44 -0500
Subject: [Swift-user] XDTM
In-Reply-To: <b19f113c0907261727r19c00cf5qa99c0742136c714@mail.gmail.com>
References: <b19f113c0907261250q3270e8c5j469850b8401b4fa0@mail.gmail.com>	
	<4A6CE247.4010105@mcs.anl.gov>
	<b19f113c0907261727r19c00cf5qa99c0742136c714@mail.gmail.com>
Message-ID: <4A6D08A8.9010400@mcs.anl.gov>

Hi Jamal,

A lot of this is covered in the Swift user guide and tutorial. Have you 
read through those yet?

All the docs are at: http://www.ci.uchicago.edu/swift/docs/index.php

If so, and the clarifications below don't help, please ask again on the 
list, OK?

- Mike


On 7/26/09 7:27 PM, J A wrote:
> Hi Michael:
>  
> First, thank you for your reply and information provided.
>  
> I am trying to understand more how it handles the input/output 
> parameters and make them available for other functions.

All functions in Swift are either atomic interfaces to application 
programs (ie, how o exec the program) or composite higher level functions.
>  
> To illustrate, I will give this example for the sake of discussion:
>  
> I have a C program called test.c that contains 4 functions ( main(), F1, 
> F2, and F3).  each function takes some parameters such as int, string, 
> name of a file that is in the same directory, and each one produced some 
> output (string, int, and a file).  Of course i am using global 
> variables.  Now, main calls F1, F1 passes its output to F2, and F2 
> passes its output to F3.

Swift doesnt look at the functions inside an application. It invokes the 
application as a program (think fork/exec) just like a shell would, but 
distributed and in parallel if so specified.
>  
> Overall, the test.c takes an int, string, and file, and output several 
> files.  the output files contains output produced by the internal 
> functions (tasks).

Swift functions can take accept files, int, string, float and boolean 
values as arguments. They return files, or scalar values inside files. 
(Again, think shell scripts).  Composite structures - structs and arrays 
- of the above can be passed.
>  
> I would like to understand more when i transfer my code to Swift how it 
> handles the input/output data, where it stores them, etc.  I read couple 
> of papers about XDTM and still have some confusion about the terms:  
> dataset, typed, how/where its physical representation is located at, and 
> how the input/output is used within the internal functions.

Files are by default named ("mapped") relative to the directory in which 
you run the Swift command. Many flexible extensions to that model are 
provided for (eg, URIs).  Swift sends the data to the site chosen for 
execution (thats yet another topic) and returns results back to the same 
submission host.

Mapping declarations in the Swift script specify how files and directory 
structures are mapped to Swift variables (scalars, arrays, structures). 
These are used in the specification of the Swift code. When Swift runs 
programs, it takes files that were mapped and knows how to send them to 
grid sites or clusters and get data back.
>  
>  
> I am new to this area and trying to understand how the DTM works.
>  
> Any help from your side on this area is really appreciated.
>  
> Thanks,
> Jamal
>  
> 
>  
> On Sun, Jul 26, 2009 at 7:09 PM, Michael Wilde <wilde at mcs.anl.gov 
> <mailto:wilde at mcs.anl.gov>> wrote:
> 
>     Jamal,
> 
>     As Swift evolved from its early prototypes to a more mature system,
>     the notion of XDTM evolved to one of mapping between
>     filesystem-based structures and Swift in-memory data structures (ie,
>     scalars, arrays, and structures, which can be nested and typed).
> 
>     This is best seen by looking at the "external" mapper, which allows
>     a user to map a dataset using any external program (typically a
>     script) that returns the members of the dataset as a two-column
>     list: the Swift variable reference, and the external file or URI.
> 
>     See the user guide section on the external mapper:
> 
>      http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.ext_mapper
>     (but the example in the user guide doesn't show the power of mapping
>     to nested structures).
> 
>     In other words, it still has the flavor of XDTM, but without any XML
>     being visible to the user. It meets the same need but is easier to
>     use and explain.
> 
>     - Mike
> 
> 
>     On 7/26/09 2:50 PM, J A wrote:
> 
>         Hi All:
>          Can any one direct me to a source with more
>         examples/explanation on how XDTM is working/implemented?
>          Thanks,
>         Jamal
>          
> 
>         ------------------------------------------------------------------------
> 
>         _______________________________________________
>         Swift-user mailing list
>         Swift-user at ci.uchicago.edu <mailto:Swift-user at ci.uchicago.edu>
>         http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
> 
> 


From iraicu at cs.uchicago.edu  Thu Jul 30 08:15:50 2009
From: iraicu at cs.uchicago.edu (Ioan Raicu)
Date: Thu, 30 Jul 2009 08:15:50 -0500
Subject: [Swift-user] [Fwd: Swift and DTM]
Message-ID: <4A719D06.4000200@cs.uchicago.edu>

Hi,
Are there any pointers anyone has about DTM in Swift?

Thanks,
Ioan

-------- Original Message --------
Subject: 	Swift and DTM
Date: 	Thu, 23 Jul 2009 11:39:07 -0400 (EDT)
From: 	Shiyong Lu <shiyong at wayne.edu>
Reply-To: 	Shiyong Lu <shiyong at wayne.edu>
To: 	Yong Zhao <yozha at microsoft.com>, iraicu at cs.uchicago.edu
CC: 	Jamal Alhiyafi <alhiyafi at gmail.com>



Dear Yong and Ioan,

I asked my PhD student Jamal (whose focus is bioinformatics workflows) 

to look at Swift, in particular the DTM part. Are there more examples

besides those that come with the installation package? Is it linked to 
myexperiment already? Thanks alot!

 

Best regards

 

-Shiyong


-- 
===================================================
Ioan Raicu, Ph.D.
===================================================
Distributed Systems Laboratory
Computer Science Department
University of Chicago
1100 E. 58th Street, Ryerson Hall
Chicago, IL 60637
===================================================
Email: iraicu at cs.uchicago.edu
Web:   http://www.cs.uchicago.edu/~iraicu
http://dev.globus.org/wiki/Incubator/Falkon
http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
===================================================
===================================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090730/376abd2a/attachment.html>

From wilde at mcs.anl.gov  Thu Jul 30 08:29:11 2009
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Thu, 30 Jul 2009 08:29:11 -0500
Subject: [Swift-user] [Fwd: Swift and DTM]
In-Reply-To: <4A719D06.4000200@cs.uchicago.edu>
References: <4A719D06.4000200@cs.uchicago.edu>
Message-ID: <4A71A027.7010901@mcs.anl.gov>

Ioan, Shiyong,

The concepts of DTM are embodied in the mapping constructs of Swift.
I sent Jamal a response on swift-user about mapping a few days ago. Was 
that of use to him?

What I can refer you and Jamal to is:

- the original XTDM paper in 2005:
http://www.ci.uchicago.edu/swift/papers/sigmod-swf-vdl.pdf

- first tests of XDTM in an early prototype of what became Swift:
http://people.cs.uchicago.edu/~yongzh/pub/sigmod-swf-vdl.pdf

- the current Swift support for dataset mapping:
http://www.ci.uchicago.edu/swift/guides/userguide.php#mappers

Regards,

Mike

On 7/30/09 8:15 AM, Ioan Raicu wrote:
> Hi,
> Are there any pointers anyone has about DTM in Swift?
> 
> Thanks,
> Ioan
> 
> -------- Original Message --------
> Subject: 	Swift and DTM
> Date: 	Thu, 23 Jul 2009 11:39:07 -0400 (EDT)
> From: 	Shiyong Lu <shiyong at wayne.edu>
> Reply-To: 	Shiyong Lu <shiyong at wayne.edu>
> To: 	Yong Zhao <yozha at microsoft.com>, iraicu at cs.uchicago.edu
> CC: 	Jamal Alhiyafi <alhiyafi at gmail.com>
> 
> 
> 
> Dear Yong and Ioan,
> 
> I asked my PhD student Jamal (whose focus is bioinformatics workflows) 
> 
> to look at Swift, in particular the DTM part. Are there more examples
> 
> besides those that come with the installation package? Is it linked to 
> myexperiment already? Thanks alot!
> 
>  
> 
> Best regards
> 
>  
> 
> -Shiyong
> 
> 
> -- 
> ===================================================
> Ioan Raicu, Ph.D.
> ===================================================
> Distributed Systems Laboratory
> Computer Science Department
> University of Chicago
> 1100 E. 58th Street, Ryerson Hall
> Chicago, IL 60637
> ===================================================
> Email: iraicu at cs.uchicago.edu
> Web:   http://www.cs.uchicago.edu/~iraicu
> http://dev.globus.org/wiki/Incubator/Falkon
> http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
> ===================================================
> ===================================================
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
-- 
Michael Wilde
Computation Institute
University of Chicago and Argonne National Laboratory
5640 S. Ellis Av, Suite 405
Chicago, IL 60637 USA
708-203-9548


From jamalphd at gmail.com  Thu Jul 30 15:00:04 2009
From: jamalphd at gmail.com (J A)
Date: Thu, 30 Jul 2009 16:00:04 -0400
Subject: [Swift-user] [Fwd: Swift and DTM]
In-Reply-To: <4A71A027.7010901@mcs.anl.gov>
References: <4A719D06.4000200@cs.uchicago.edu> <4A71A027.7010901@mcs.anl.gov>
Message-ID: <b19f113c0907301300x19d5527j43ac671374f197b4@mail.gmail.com>

Hi Michael:

You sent you email while i was going over your XTDM paper in 2005.  I may
have some questions that i will post later.

By the way, the first two links in your email below point to the same
paper.  Is that what you meant?

Thanks for your cooperation and help.

Jamal



On Thu, Jul 30, 2009 at 9:29 AM, Michael Wilde <wilde at mcs.anl.gov> wrote:

> Ioan, Shiyong,
>
> The concepts of DTM are embodied in the mapping constructs of Swift.
> I sent Jamal a response on swift-user about mapping a few days ago. Was
> that of use to him?
>
> What I can refer you and Jamal to is:
>
> - the original XTDM paper in 2005:
> http://www.ci.uchicago.edu/swift/papers/sigmod-swf-vdl.pdf
>
> - first tests of XDTM in an early prototype of what became Swift:
> http://people.cs.uchicago.edu/~yongzh/pub/sigmod-swf-vdl.pdf
>
> - the current Swift support for dataset mapping:
> http://www.ci.uchicago.edu/swift/guides/userguide.php#mappers
>
> Regards,
>
> Mike
>
>
> On 7/30/09 8:15 AM, Ioan Raicu wrote:
>
>>  Hi,
>> Are there any pointers anyone has about DTM in Swift?
>>
>> Thanks,
>> Ioan
>>
>> -------- Original Message --------
>> Subject:        Swift and DTM
>> Date:   Thu, 23 Jul 2009 11:39:07 -0400 (EDT)
>> From:   Shiyong Lu <shiyong at wayne.edu>
>> Reply-To:       Shiyong Lu <shiyong at wayne.edu>
>> To:     Yong Zhao <yozha at microsoft.com>, iraicu at cs.uchicago.edu
>> CC:     Jamal Alhiyafi <alhiyafi at gmail.com>
>>
>>
>>
>> Dear Yong and Ioan,
>>
>> I asked my PhD student Jamal (whose focus is bioinformatics workflows)
>> to look at Swift, in particular the DTM part. Are there more examples
>>
>> besides those that come with the installation package? Is it linked to
>> myexperiment already? Thanks alot!
>>
>>
>> Best regards
>>
>>
>> -Shiyong
>>
>>
>> --
>> ===================================================
>> Ioan Raicu, Ph.D.
>> ===================================================
>> Distributed Systems Laboratory
>> Computer Science Department
>> University of Chicago
>> 1100 E. 58th Street, Ryerson Hall
>> Chicago, IL 60637
>> ===================================================
>> Email: iraicu at cs.uchicago.edu
>> Web:   http://www.cs.uchicago.edu/~iraicu
>> http://dev.globus.org/wiki/Incubator/Falkon
>> http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
>> ===================================================
>> ===================================================
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Swift-user mailing list
>> Swift-user at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
>>
> --
> Michael Wilde
> Computation Institute
> University of Chicago and Argonne National Laboratory
> 5640 S. Ellis Av, Suite 405
> Chicago, IL 60637 USA
> 708-203-9548
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090730/e6dd53e9/attachment.html>

From wilde at mcs.anl.gov  Thu Jul 30 15:26:28 2009
From: wilde at mcs.anl.gov (Michael Wilde)
Date: Thu, 30 Jul 2009 15:26:28 -0500
Subject: [Swift-user] [Fwd: Swift and DTM]
In-Reply-To: <b19f113c0907301300x19d5527j43ac671374f197b4@mail.gmail.com>
References: <4A719D06.4000200@cs.uchicago.edu> <4A71A027.7010901@mcs.anl.gov>
	<b19f113c0907301300x19d5527j43ac671374f197b4@mail.gmail.com>
Message-ID: <4A7201F4.1040703@mcs.anl.gov>



On 7/30/09 3:00 PM, J A wrote:
> Hi Michael:
>  
> You sent you email while i was going over your XTDM paper in 2005.  I 
> may have some questions that i will post later.
>  
> By the way, the first two links in your email below point to the same 
> paper.  Is that what you meant?

No, sorry. I meant the first link to be the 2005 EGC paper:

http://www.ci.uchicago.edu/swift/papers/XDTM_egc05.pdf

which is what I think you are reading now. That first link is bad on the 
Swift web, we need to fix it.

Thanks,

Mike

>  
> Thanks for your cooperation and help.
>  
> Jamal
> 
> 
>  
> On Thu, Jul 30, 2009 at 9:29 AM, Michael Wilde <wilde at mcs.anl.gov 
> <mailto:wilde at mcs.anl.gov>> wrote:
> 
>     Ioan, Shiyong,
> 
>     The concepts of DTM are embodied in the mapping constructs of Swift.
>     I sent Jamal a response on swift-user about mapping a few days ago.
>     Was that of use to him?
> 
>     What I can refer you and Jamal to is:
> 
>     - the original XTDM paper in 2005:
>     http://www.ci.uchicago.edu/swift/papers/sigmod-swf-vdl.pdf
> 
>     - first tests of XDTM in an early prototype of what became Swift:
>     http://people.cs.uchicago.edu/~yongzh/pub/sigmod-swf-vdl.pdf
> 
>     - the current Swift support for dataset mapping:
>     http://www.ci.uchicago.edu/swift/guides/userguide.php#mappers
> 
>     Regards,
> 
>     Mike
> 
> 
>     On 7/30/09 8:15 AM, Ioan Raicu wrote:
> 
>         Hi,
>         Are there any pointers anyone has about DTM in Swift?
> 
>         Thanks,
>         Ioan
> 
>         -------- Original Message --------
>         Subject:        Swift and DTM
>         Date:   Thu, 23 Jul 2009 11:39:07 -0400 (EDT)
>         From:   Shiyong Lu <shiyong at wayne.edu <mailto:shiyong at wayne.edu>>
>         Reply-To:       Shiyong Lu <shiyong at wayne.edu
>         <mailto:shiyong at wayne.edu>>
>         To:     Yong Zhao <yozha at microsoft.com
>         <mailto:yozha at microsoft.com>>, iraicu at cs.uchicago.edu
>         <mailto:iraicu at cs.uchicago.edu>
>         CC:     Jamal Alhiyafi <alhiyafi at gmail.com
>         <mailto:alhiyafi at gmail.com>>
> 
> 
> 
>         Dear Yong and Ioan,
> 
>         I asked my PhD student Jamal (whose focus is bioinformatics
>         workflows)
>         to look at Swift, in particular the DTM part. Are there more
>         examples
> 
>         besides those that come with the installation package? Is it
>         linked to myexperiment already? Thanks alot!
> 
>          
>         Best regards
> 
>          
>         -Shiyong
> 
> 
>         -- 
>         ===================================================
>         Ioan Raicu, Ph.D.
>         ===================================================
>         Distributed Systems Laboratory
>         Computer Science Department
>         University of Chicago
>         1100 E. 58th Street, Ryerson Hall
>         Chicago, IL 60637
>         ===================================================
>         Email: iraicu at cs.uchicago.edu <mailto:iraicu at cs.uchicago.edu>
>         Web:   http://www.cs.uchicago.edu/~iraicu
>         http://dev.globus.org/wiki/Incubator/Falkon
>         http://dsl-wiki.cs.uchicago.edu/index.php/Main_Page
>         ===================================================
>         ===================================================
> 
> 
>         ------------------------------------------------------------------------
> 
>         _______________________________________________
>         Swift-user mailing list
>         Swift-user at ci.uchicago.edu <mailto:Swift-user at ci.uchicago.edu>
>         http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
> 
>     -- 
>     Michael Wilde
>     Computation Institute
>     University of Chicago and Argonne National Laboratory
>     5640 S. Ellis Av, Suite 405
>     Chicago, IL 60637 USA
>     708-203-9548
>     _______________________________________________
>     Swift-user mailing list
>     Swift-user at ci.uchicago.edu <mailto:Swift-user at ci.uchicago.edu>
>     http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
> 
> 


From andric at uchicago.edu  Thu Jul 23 13:14:07 2009
From: andric at uchicago.edu (Michael Andric)
Date: Thu, 23 Jul 2009 18:14:07 -0000
Subject: [Swift-user] errors from HNL machines/swift
Message-ID: <b445e57f0907231113m1d7963a9l7b6c2c629aeae42a@mail.gmail.com>

HI Support, Swift dev, anyone else reading,
I keep getting this crash on swift jobs submitted from HNL machines (both
andrew.bsd.uchicago.edu and gwynn.bsd.uchicago.edu).  These happen for
different workflows, involving different processes.  I am totally in the
dark as to what this error is referring to as well as to what may be causing
it.  This crash has occurred on workflows that have just gone 'Active' as
well as on workflows that were running for hours before crashing.

Below is the error message.  The log file is too big to attach but can be
found here:
/gpfs/pads/fmri/cnari/swift/projects/andric/peakfit_pilots/PK2/turnpointAnalysis/tpChiSqTests-20090723-1113-na2cuboc.log
from one of the HNL machines (e.g., gwynn.bsd.uchicago.edu)

Any insight is hugely appreciated - like i said, i don't even know what to
debug b/c i don't know what the error is referring to.
Michael




Progress:  Submitted:11 Active:1
Progress:  Active:10 Stage out:2
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGBUS (0x7) at pc=0xb75b9a62, pid=32310, tid=2949090208
#
# Java VM: Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode, sharing)
# Problematic frame:
# C  [libzip.so+0xfa62]
#
# An error report file with more information is saved as hs_err_pid32310.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
/gpfs/pads/fmri/apps/swift/bin/swift: line 100: 32310 Aborted
  java -Xmx2048M
-Djava.endorsed.dirs=/gpfs/pads/fmri/apps/swift/bin/../lib/endorsed
-DUID=1309 -DGLOBUS_TCP_PORT_RANGE=50000,51000 -DGLOBUS_HOSTNAME=
andrew.bsd.uchicago.edu -DCOG_INSTALL_PATH=/gpfs/pads/fmri/apps/swift/bin/..
-Dvds.home=/gpfs/pads/fmri/apps/swift/bin/..
-Dswift.home=/gpfs/pads/fmri/apps/swift/bin/..
-Djava.security.egd=file:///dev/urandom -Xmx1024m -classpath
/gpfs/pads/fmri/apps/swift/bin/../etc:/gpfs/pads/fmri/apps/swift/bin/../libexec:/gpfs/pads/fmri/apps/swift/bin/../lib/addressing-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/ant.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/antlr-2.7.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/backport-util-concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/castor-0.9.6.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/coaster-bootstrap.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-abstraction-common-2.3.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-grapheditor-0.47.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-jglobus-dev-080222.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-karajan-0.36-dev.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-clref-gt4_0_0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-coaster-0.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-dcache-0.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt2-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt4_0_0-2.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-local-2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-localscheduler-0.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-ssh-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-webdav-2.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-resources-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-swift-svn.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-trap-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-util-0.92.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commonj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-beanutils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-collections-3.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-digester.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-discovery.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-httpclient.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-logging-1.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix32.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix-asn1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_mds_aggregator_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rft_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-client.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-utils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gvds.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-common-0.2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-core-0.2.2-patched.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-regexp-1.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-slide-webdavlib-2.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jaxrpc.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jce-jdk13-131.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jgss.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jsr173_1.0_api.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jug-lgpl-2.0.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/junit.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/log4j-1.2.8.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-common.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-factory.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-java.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-resources.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/opensaml.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/puretls.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/resolver.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/saaj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/stringtemplate.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/vdldefinitions.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsdl4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_index_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_usefulrp_schema_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_provider_jce.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_tools.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wss4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xalan.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean_xpath.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xercesImpl.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xml-apis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xmlsec.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xpp3-1.1.3.4d_b4_min.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xstream-1.1.1-patched.jar:
org.griphyn.vdl.karajan.Loader 'tpChiSqTests.swift' '-sites.file'
'/gpfs/pads/fmri/cnari_svn/config/coaster_ranger.xml' '-user=andric'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090723/90fc42f5/attachment.html>

From andric at uchicago.edu  Thu Jul 23 15:23:37 2009
From: andric at uchicago.edu (Michael Andric)
Date: Thu, 23 Jul 2009 20:23:37 -0000
Subject: [Swift-user] Re: [Swift-devel] errors from HNL machines/swift
In-Reply-To: <1248373216.28628.1.camel@localhost>
References: <b445e57f0907231113m1d7963a9l7b6c2c629aeae42a@mail.gmail.com> 
	<1248373216.28628.1.camel@localhost>
Message-ID: <b445e57f0907231323p3c91bc14hfa7c1174ce076cef@mail.gmail.com>

there are a couple here:  andrew.bsd.uchicago.edu:/tmp/hs*.log

On Thu, Jul 23, 2009 at 1:20 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:

> There's a JVM dump, namely ?hs_err_pid32310.log. I'd like to see that.
>
> Otherwise it seems related to this:
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6390352
>
> You could try a newer JVM and see if the problem persists.
>
> On Thu, 2009-07-23 at 13:13 -0500, Michael Andric wrote:
> > HI Support, Swift dev, anyone else reading,
> >
> > I keep getting this crash on swift jobs submitted from HNL machines
> > (both andrew.bsd.uchicago.edu and gwynn.bsd.uchicago.edu).  These
> > happen for different workflows, involving different processes.  I am
> > totally in the dark as to what this error is referring to as well as
> > to what may be causing it.  This crash has occurred on workflows that
> > have just gone 'Active' as well as on workflows that were running for
> > hours before crashing.
> >
> >
> > Below is the error message.  The log file is too big to attach but can
> > be found here:
> >
> /gpfs/pads/fmri/cnari/swift/projects/andric/peakfit_pilots/PK2/turnpointAnalysis/tpChiSqTests-20090723-1113-na2cuboc.log
> > from one of the HNL machines (e.g., gwynn.bsd.uchicago.edu)
> >
> >
> > Any insight is hugely appreciated - like i said, i don't even know
> > what to debug b/c i don't know what the error is referring to.
> > Michael
> >
> >
> >
> >
> >
> >
> >
> > Progress:  Submitted:11 Active:1
> > Progress:  Active:10 Stage out:2
> > #
> > # An unexpected error has been detected by HotSpot Virtual Machine:
> > #
> > #  SIGBUS (0x7) at pc=0xb75b9a62, pid=32310, tid=2949090208
> > #
> > # Java VM: Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode,
> > sharing)
> > # Problematic frame:
> > # C  [libzip.so+0xfa62]
> > #
> > # An error report file with more information is saved as
> > hs_err_pid32310.log
> > #
> > # If you would like to submit a bug report, please visit:
> > #   http://java.sun.com/webapps/bugreport/crash.jsp
> > #
> > /gpfs/pads/fmri/apps/swift/bin/swift: line 100: 32310 Aborted
> > java -Xmx2048M
> > -Djava.endorsed.dirs=/gpfs/pads/fmri/apps/swift/bin/../lib/endorsed
> > -DUID=1309 -DGLOBUS_TCP_PORT_RANGE=50000,51000
> > -DGLOBUS_HOSTNAME=andrew.bsd.uchicago.edu-DCOG_INSTALL_PATH=/gpfs/pads/fmri/apps/swift/bin/..
> -Dvds.home=/gpfs/pads/fmri/apps/swift/bin/..
> -Dswift.home=/gpfs/pads/fmri/apps/swift/bin/..
> -Djava.security.egd=file:///dev/urandom -Xmx1024m -classpath
> /gpfs/pads/fmri/apps/swift/bin/../etc:/gpfs/pads/fmri/apps/swift/bin/../libexec:/gpfs/pads/fmri/apps/swift/bin/../lib/addressing-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/ant.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/antlr-2.7.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/backport-util-concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/castor-0.9.6.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/coaster-bootstrap.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-abstraction-common-2.3.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-grapheditor-0.47.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-jglobus-dev-080222.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-karajan-0.36-dev.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-clref-gt4_0_0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-coaster-0.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-dcache-0.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt2-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-gt4_0_0-2.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-local-2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-localscheduler-0.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-ssh-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-webdav-2.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-resources-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-swift-svn.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-trap-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-util-0.92.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commonj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-beanutils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-collections-3.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-digester.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-discovery.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-httpclient.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-logging-1.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix32.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix-asn1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_mds_aggregator_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rft_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-client.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-utils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gvds.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-common-0.2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-core-0.2.2-patched.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-regexp-1.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-slide-webdavlib-2.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jaxrpc.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jce-jdk13-131.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jgss.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jsr173_1.0_api.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jug-lgpl-2.0.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/junit.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/log4j-1.2.8.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-common.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-factory.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-java.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-resources.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/opensaml.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/puretls.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/resolver.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/saaj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/stringtemplate.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/vdldefinitions.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsdl4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_index_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_usefulrp_schema_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_provider_jce.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_tools.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wss4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xalan.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean_xpath.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xercesImpl.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xml-apis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xmlsec.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xpp3-1.1.3.4d_b4_min.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xstream-1.1.1-patched.jar:
> org.griphyn.vdl.karajan.Loader 'tpChiSqTests.swift' '-sites.file'
> '/gpfs/pads/fmri/cnari_svn/config/coaster_ranger.xml' '-user=andric'
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090723/21ad1f15/attachment.html>

From support at ci.uchicago.edu  Fri Jul 24 08:50:21 2009
From: support at ci.uchicago.edu (Ti Leggett)
Date: Fri, 24 Jul 2009 13:50:21 -0000
Subject: [Swift-user] [CI Ticketing System #1372] errors from HNL
	machines/swift 
In-Reply-To: <1248381203.32020.0.camel@localhost>
References: <RT-Ticket-1372@ci.uchicago.edu>
	<b445e57f0907231113m1d7963a9l7b6c2c629aeae42a@mail.gmail.com>
	<1248373216.28628.1.camel@localhost>
	<b445e57f0907231323p3c91bc14hfa7c1174ce076cef@mail.gmail.com>
	<1248381203.32020.0.camel@localhost>
Message-ID: <rt-3.8.2-5145-1248443415-105.1372-6-0@ci.uchicago.edu>

Try adding +java-1.6.0_03-sun-r1 above any other lines in your ~/.soft
and run resoft. See if that helps your issues.

On Thu Jul 23 15:33:34 2009, hategan at mcs.anl.gov wrote:
> Can't help you much there. It seems to be a bug in the JVM. Again, I'd
> try other versions of java.
> 
> On Thu, 2009-07-23 at 15:23 -0500, Michael Andric wrote:
> > there are a couple here:  andrew.bsd.uchicago.edu:/tmp/hs*.log
> >
> > On Thu, Jul 23, 2009 at 1:20 PM, Mihael Hategan
>    <hategan at mcs.anl.gov>
> > wrote:
> >         There's a JVM dump, namely ?hs_err_pid32310.log. I'd like to
> >         see that.
> >
> >         Otherwise it seems related to this:
> >         http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6390352
> >
> >         You could try a newer JVM and see if the problem persists.
> >
> >
> >         On Thu, 2009-07-23 at 13:13 -0500, Michael Andric wrote:
> >         > HI Support, Swift dev, anyone else reading,
> >         >
> >         > I keep getting this crash on swift jobs submitted from HNL
> >         machines
> >         > (both andrew.bsd.uchicago.edu and gwynn.bsd.uchicago.edu).
> >          These
> >         > happen for different workflows, involving different
> >         processes.  I am
> >         > totally in the dark as to what this error is referring to
>    as
> >         well as
> >         > to what may be causing it.  This crash has occurred on
> >         workflows that
> >         > have just gone 'Active' as well as on workflows that were
> >         running for
> >         > hours before crashing.
> >         >
> >         >
> >         > Below is the error message.  The log file is too big to
> >         attach but can
> >         > be found here:
> >         >
>   
/gpfs/pads/fmri/cnari/swift/projects/andric/peakfit_pilots/PK2/turnpointAnalysis/tpChiSqTests-
>    20090723-1113-na2cuboc.log
> >         > from one of the HNL machines (e.g.,
>    gwynn.bsd.uchicago.edu)
> >         >
> >         >
> >         > Any insight is hugely appreciated - like i said, i don't
> >         even know
> >         > what to debug b/c i don't know what the error is referring
> >         to.
> >         > Michael
> >         >
> >         >
> >         >
> >         >
> >         >
> >         >
> >         >
> >         > Progress:  Submitted:11 Active:1
> >         > Progress:  Active:10 Stage out:2
> >         > #
> >         > # An unexpected error has been detected by HotSpot Virtual
> >         Machine:
> >         > #
> >         > #  SIGBUS (0x7) at pc=0xb75b9a62, pid=32310,
>    tid=2949090208
> >         > #
> >         > # Java VM: Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed
> >         mode,
> >         > sharing)
> >         > # Problematic frame:
> >         > # C  [libzip.so+0xfa62]
> >         > #
> >         > # An error report file with more information is saved as
> >         > hs_err_pid32310.log
> >         > #
> >         > # If you would like to submit a bug report, please visit:
> >         > #   http://java.sun.com/webapps/bugreport/crash.jsp
> >         > #
> >         > /gpfs/pads/fmri/apps/swift/bin/swift: line 100: 32310
> >         Aborted
> >         > java -Xmx2048M
> >         >
> >
>    -Djava.endorsed.dirs=/gpfs/pads/fmri/apps/swift/bin/../lib/endorsed
> >         > -DUID=1309 -DGLOBUS_TCP_PORT_RANGE=50000,51000
> >         > -DGLOBUS_HOSTNAME=andrew.bsd.uchicago.edu
> >         -DCOG_INSTALL_PATH=/gpfs/pads/fmri/apps/swift/bin/..
> >         -Dvds.home=/gpfs/pads/fmri/apps/swift/bin/..
> >         -Dswift.home=/gpfs/pads/fmri/apps/swift/bin/..
> >         -Djava.security.egd=file:///dev/urandom -Xmx1024m
> >         -classpath
>   
/gpfs/pads/fmri/apps/swift/bin/../etc:/gpfs/pads/fmri/apps/swift/bin/../libexec:/gpfs/pads/fmri/apps/swift/bin/../lib/addressing-
>   
1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/ant.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/antlr-
>   
2.7.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/axis-
>    url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/backport-util-
>    concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/castor-
>    0.9.6.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/coaster-
>    bootstrap.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-
>    abstraction-common-
>    2.3.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-
>    axis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-grapheditor-
>    0.47.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-jglobus-dev-
>    080222.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-karajan-0.36-
>    dev.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-clref-
>    gt4_0_0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-
>    coaster-0.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-
>    dcache-0.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-
>    gt2-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-
>    gt4_0_0-2.5.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-
>    local-2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-provider-
>    localscheduler-0.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-
>    provider-ssh-2.4.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-
>    provider-webdav-2.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-
>    resources-1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-swift-
>    svn.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-trap-
>    1.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-
>    url.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cog-util-
>   
0.92.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commonj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-
>    beanutils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-
>    collections-3.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-
>    digester.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-
>    discovery.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-
>    httpclient.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/commons-
>    logging-
>   
1.1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/concurrent.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix32.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix-
>   
asn1.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/cryptix.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_delegation_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_mds_aggregator_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_service.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rendezvous_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/globus_wsrf_rft_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-
>    client.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-
>    stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gram-
>   
utils.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/gvds.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-
>    common-0.2.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/j2ssh-core-
>    0.2.2-patched.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-
>    regexp-1.2.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jakarta-slide-
>    webdavlib-
>   
2.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jaxrpc.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jce-
>    jdk13-
>   
131.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jgss.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jsr173_1.0_api.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/jug-
>    lgpl-
>   
2.0.0.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/junit.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/log4j-
>    1.2.8.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-
>    common.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-
>    factory.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-
>    java.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/naming-
>   
resources.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/opensaml.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/puretls.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/resolver.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/saaj.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/stringtemplate.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/vdldefinitions.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsdl4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_core_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_index_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_mds_usefulrp_schema_stubs.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_provider_jce.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wsrf_tools.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/wss4j.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xalan.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xbean_xpath.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xercesImpl.jar:/gpf
 s/pads/fmri/apps/swift/bin/../lib/xml-
>   
apis.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xmlsec.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xpp3-
>    1.1.3.4d_b4_min.jar:/gpfs/pads/fmri/apps/swift/bin/../lib/xstream-
>    1.1.1-patched.jar: org.griphyn.vdl.karajan.Loader
>    'tpChiSqTests.swift' '-sites.file'
>    '/gpfs/pads/fmri/cnari_svn/config/coaster_ranger.xml' '-
>    user=andric'
> >
> >         > _______________________________________________
> >         > Swift-devel mailing list
> >         > Swift-devel at ci.uchicago.edu
> >         > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> >
> >
>