[Swift-devel] Swift unresponsive while using local provider.

Jonathan Monette jonmon at utexas.edu
Fri Jun 17 19:47:56 CDT 2011 

I meant what was the swift command line that is executed but if it is run within the test suite I guess you don't know that upfront.  

The reason I am asking is I wanted to know what sites file the test suite reverted to when you moved the other on out.  Since you set the workdirectory to /tmp I thought maybe /tmp was being filled up which was causing the timeouts and hangs since there was no more room in the workdirectory.

Not sure if this is the case but it was a thought when I saw that /tmp was the workdirectory.
On Jun 17, 2011, at 7:30 PM, Alberto Chavez wrote:

> >Have you been running this within the test suite or by itself?
> Within the test suite, 
> >What is the command line that was used to execute this script?
> bash suite.sh -t -o /tmp/chavez documentation/
> 
> On Jun 17, 2011, at 7:13 PM, Alberto Chavez wrote:
> 
> No, is the same one.
> But I had a sites.template.xml file in that directory, which contained that information; as soon as I removed sites.template.xml from my directory, the script worked just fine.
> 
> type messagefile;
> app (messagefile t) greeting (string s[]) {
> echo s[0] s[1] s[2] stdout=@filename(t);
> }
> messagefile outfile <"q5out.txt">;
> string words[] = ["how","are","you"];
> outfile = greeting(words);
> 
> 
> 
> > Subject: RE: [Swift-devel] Swift unresponsive while using local provider.
> > From: hategan at mcs.anl.gov
> > To: alberto_chavez at live.com
> > CC: davidkelly999 at gmail.com; swift-devel at ci.uchicago.edu
> > Date: Fri, 17 Jun 2011 17:09:52 -0700
> > 
> > I'm sorry, but I don't follow. Is there a new error?
> > 
> > 
> > On Fri, 2011-06-17 at 18:55 -0500, Alberto Chavez wrote:
> > > sites.template.xml is producing this error, as soon as I remove the
> > > file from the directory, the error goes away as well.
> > > These are the contents of such file:
> > > 
> > > <config>
> > > <pool handle="localhost">
> > > <filesystem provider="local" />
> > > <execution provider="coaster" jobmanager="local:local"/>
> > > <profile namespace="globus"
> > > key="internalHostname">127.0.0.1</profile>
> > > <profile namespace="karajan" key="jobthrottle">1000</profile>
> > > <profile namespace="karajan" key="initialScore">10000</profile>
> > > <profile namespace="globus" key="jobsPerNode">4</profile>
> > > <profile namespace="globus" key="slots">8</profile>
> > > <profile namespace="globus" key="maxTime">1000</profile>
> > > <profile namespace="globus" key="nodeGranularity">1</profile>
> > > <profile namespace="globus" key="maxNodes">4</profile>
> > > <workdirectory>/tmp</workdirectory>
> > > </pool>
> > > </config>
> > > 
> > > 
> > > -Alberto
> > > 
> > > 
> > > > Subject: Re: [Swift-devel] Swift unresponsive while using local
> > > provider.
> > > > From: hategan at mcs.anl.gov
> > > > To: davidkelly999 at gmail.com
> > > > Date: Fri, 17 Jun 2011 12:56:25 -0700
> > > > CC: swift-devel at ci.uchicago.edu
> > > > 
> > > > do "jstack -l <pid_of_swift_java_process>" whenever it happens and
> > > send
> > > > the output.
> > > > 
> > > > 
> > > > 
> > > > On Fri, 2011-06-17 at 14:48 -0500, David Kelly wrote:
> > > > > I saw similar things on my laptop (4 gb ram) this weekend when I
> > > was
> > > > > testing the galaxy demo scripts using the local provider. I was
> > > using
> > > > > trunk. In the output I would see things like "no activity for 10s"
> > > and
> > > > > it just would sit there and do nothing until I manually killed it.
> > > But
> > > > > most of the time it would work fine. I wrote a little shell script
> > > > > that would repeatedly run it until it hung. Then I was talking to
> > > Jon
> > > > > about this and he saw something similar with his montage work. He
> > > > > thought it might be related to a configuration issue - that either
> > > > > wrapper.parameter.mode=files or status.mode=provider should be
> > > set.
> > > > > 
> > > > > I can send my scripts as well if you need some help in tracking
> > > this
> > > > > down.
> > > > > 
> > > > > David
> > > > > 
> > > > > On Fri, Jun 17, 2011 at 2:38 PM, Michael Wilde <wilde at mcs.anl.gov>
> > > > > wrote:
> > > > > Alberto, how long are you letting it run for, and under what
> > > > > environment? if you are running on your laptop, how much RAM
> > > > > do you have? Its possible that you are seeing paging delays
> > > > > if you are running the Swift Java app with too little memory.
> > > > > 
> > > > > 
> > > > > Also, are you running trunk or 0.92.1? You should compare the
> > > > > two.
> > > > > 
> > > > > 
> > > > > Its *possible* that this simple test is hanging under recent
> > > > > trunk mods, but its more likely that this is some kind of
> > > > > resource shortage.
> > > > > 
> > > > > 
> > > > > Can you run this on one of the Swift lab machines bridled or
> > > > > communcado, or better yet on the MCS compute servers, or a
> > > > > PADS worker node (which you can get with qsub -I on pads)?
> > > > > 
> > > > > 
> > > > > Look at Swift under the "top" command to see if Swift is
> > > > > running and slow, or is hung.
> > > > > 
> > > > > 
> > > > > Stop by and we can discuss in more detail.
> > > > > 
> > > > > 
> > > > > - Mike
> > > > > 
> > > > > 
> > > > > 
> > > > > ______________________________________________________________
> > > > > 
> > > > > When I run the following SwiftScript using suite.sh,
> > > > > the report shows an odd behavior, most of the time it
> > > > > times out, but once in a while it passes, however this
> > > > > outcome is completely random, since sometimes that
> > > > > test has passed 3 times in a row, and all of the
> > > > > sudden it fails.
> > > > > This is my script:
> > > > > 
> > > > > 
> > > > > type messagefile;
> > > > > 
> > > > > 
> > > > > app (messagefile t) greeting (string s[]) {
> > > > > echo s[0] s[1] s[2] stdout=@filename(t);
> > > > > }
> > > > > 
> > > > > 
> > > > > messagefile outfile <"q5out.txt">;
> > > > > 
> > > > > 
> > > > > string words[] = ["how","are","you"];
> > > > > 
> > > > > 
> > > > > outfile = greeting(words);
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > Swift.properties contents:
> > > > > 
> > > > > 
> > > > > $ cat swift.properties 
> > > > > wrapperlog.always.transfer=true
> > > > > sitedir.keep=true
> > > > > execution.retries=0
> > > > > lazy.errors=false
> > > > > status.mode=provider
> > > > > use.provider.staging=false
> > > > > provider.staging.pin.swiftfiles=false
> > > > > 
> > > > > 
> > > > > Sites.template.xml contents:
> > > > > 
> > > > > 
> > > > > $ cat sites.template.xml 
> > > > > <config>
> > > > > <pool handle="localhost">
> > > > > <filesystem provider="local" />
> > > > > <execution provider="coaster"
> > > > > jobmanager="local:local"/>
> > > > > <profile namespace="globus"
> > > > > key="internalHostname">127.0.0.1</profile>
> > > > > <profile namespace="karajan"
> > > > > key="jobthrottle">1000</profile>
> > > > > <profile namespace="karajan"
> > > > > key="initialScore">10000</profile>
> > > > > <profile namespace="globus"
> > > > > key="jobsPerNode">4</profile>
> > > > > <profile namespace="globus"
> > > > > key="slots">8</profile>
> > > > > <profile namespace="globus"
> > > > > key="maxTime">1000</profile>
> > > > > <profile namespace="globus"
> > > > > key="nodeGranularity">1</profile>
> > > > > <profile namespace="globus"
> > > > > key="maxNodes">4</profile>
> > > > > <workdirectory>/tmp</workdirectory>
> > > > > </pool>
> > > > > </config>
> > > > > 
> > > > > 
> > > > > -Alberto
> > > > > 
> > > > > 
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > > 
> > > > > 
> > > > > 
> > > > > -- 
> > > > > Michael Wilde
> > > > > Computation Institute, University of Chicago
> > > > > Mathematics and Computer Science Division
> > > > > Argonne National Laboratory
> > > > > 
> > > > > 
> > > > > 
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > > 
> > > > > 
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > 
> > > > 
> > > > _______________________________________________
> > > > Swift-devel mailing list
> > > > Swift-devel at ci.uchicago.edu
> > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > 
> > 
> > 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110617/cc0e0059/attachment.html>

More information about the Swift-devel mailing list