[Swift-devel] Swift unresponsive while using local provider.
Alberto Chavez
alberto_chavez at live.com
Fri Jun 17 19:29:16 CDT 2011
I already did that in the revision I have, and the tests passed every iteration; I am going to update my working copies to the latest svn code, and I'll run it again.By the way, I put sites.template.xml back, and ran the command jstack
This is the output:
$ jstack -l 27912791: Unable to open socket file: target process not responding or HotSpot VM not loadedThe -F option can be used when the target process is not responding$ jstack -F 2791Attaching to process ID 2791, please wait...sun.jvm.hotspot.debugger.NoSuchSymbolException: Could not find symbol "gHotSpotVMTypes" in any of the known library names (libjvm.so, libjvm_g.so, gamma_g) at sun.jvm.hotspot.HotSpotTypeDataBase.lookupInProcess(HotSpotTypeDataBase.java:389) at sun.jvm.hotspot.HotSpotTypeDataBase.readVMTypes(HotSpotTypeDataBase.java:104) at sun.jvm.hotspot.HotSpotTypeDataBase.<init>(HotSpotTypeDataBase.java:85) at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:568) at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494) at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:332) at sun.jvm.hotspot.tools.Tool.start(Tool.java:163) at sun.jvm.hotspot.tools.JStack.main(JStack.java:86) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.tools.jstack.JStack.runJStackTool(JStack.java:118) at sun.tools.jstack.JStack.main(JStack.java:84)Debugger attached successfully.jstack requires a java VM process/core!
> Subject: RE: [Swift-devel] Swift unresponsive while using local provider.
> From: hategan at mcs.anl.gov
> To: alberto_chavez at live.com
> CC: davidkelly999 at gmail.com; swift-devel at ci.uchicago.edu
> Date: Fri, 17 Jun 2011 17:19:46 -0700
>
> Right. Now run it 100 more times (make a loop in a shell script) and see
> if none of those deadlock.
>
> Then update to the latest svn code, re-compile and run the script 100
> more times. See if it deadlocks then.
>
> On Fri, 2011-06-17 at 19:13 -0500, Alberto Chavez wrote:
> > No, is the same one.
> > But I had a sites.template.xml file in that directory, which contained
> > that information; as soon as I removed sites.template.xml from my
> > directory, the script worked just fine.
> >
> >
> > type messagefile;
> > app (messagefile t) greeting (string s[]) {
> > echo s[0] s[1] s[2] stdout=@filename(t);
> > }
> > messagefile outfile <"q5out.txt">;
> > string words[] = ["how","are","you"];
> > outfile = greeting(words);
> >
> >
> >
> >
> >
> > > Subject: RE: [Swift-devel] Swift unresponsive while using local
> > provider.
> > > From: hategan at mcs.anl.gov
> > > To: alberto_chavez at live.com
> > > CC: davidkelly999 at gmail.com; swift-devel at ci.uchicago.edu
> > > Date: Fri, 17 Jun 2011 17:09:52 -0700
> > >
> > > I'm sorry, but I don't follow. Is there a new error?
> > >
> > >
> > > On Fri, 2011-06-17 at 18:55 -0500, Alberto Chavez wrote:
> > > > sites.template.xml is producing this error, as soon as I remove
> > the
> > > > file from the directory, the error goes away as well.
> > > > These are the contents of such file:
> > > >
> > > > <config>
> > > > <pool handle="localhost">
> > > > <filesystem provider="local" />
> > > > <execution provider="coaster" jobmanager="local:local"/>
> > > > <profile namespace="globus"
> > > > key="internalHostname">127.0.0.1</profile>
> > > > <profile namespace="karajan" key="jobthrottle">1000</profile>
> > > > <profile namespace="karajan" key="initialScore">10000</profile>
> > > > <profile namespace="globus" key="jobsPerNode">4</profile>
> > > > <profile namespace="globus" key="slots">8</profile>
> > > > <profile namespace="globus" key="maxTime">1000</profile>
> > > > <profile namespace="globus" key="nodeGranularity">1</profile>
> > > > <profile namespace="globus" key="maxNodes">4</profile>
> > > > <workdirectory>/tmp</workdirectory>
> > > > </pool>
> > > > </config>
> > > >
> > > >
> > > > -Alberto
> > > >
> > > >
> > > > > Subject: Re: [Swift-devel] Swift unresponsive while using local
> > > > provider.
> > > > > From: hategan at mcs.anl.gov
> > > > > To: davidkelly999 at gmail.com
> > > > > Date: Fri, 17 Jun 2011 12:56:25 -0700
> > > > > CC: swift-devel at ci.uchicago.edu
> > > > >
> > > > > do "jstack -l <pid_of_swift_java_process>" whenever it happens
> > and
> > > > send
> > > > > the output.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, 2011-06-17 at 14:48 -0500, David Kelly wrote:
> > > > > > I saw similar things on my laptop (4 gb ram) this weekend when
> > I
> > > > was
> > > > > > testing the galaxy demo scripts using the local provider. I
> > was
> > > > using
> > > > > > trunk. In the output I would see things like "no activity for
> > 10s"
> > > > and
> > > > > > it just would sit there and do nothing until I manually killed
> > it.
> > > > But
> > > > > > most of the time it would work fine. I wrote a little shell
> > script
> > > > > > that would repeatedly run it until it hung. Then I was talking
> > to
> > > > Jon
> > > > > > about this and he saw something similar with his montage work.
> > He
> > > > > > thought it might be related to a configuration issue - that
> > either
> > > > > > wrapper.parameter.mode=files or status.mode=provider should be
> > > > set.
> > > > > >
> > > > > > I can send my scripts as well if you need some help in
> > tracking
> > > > this
> > > > > > down.
> > > > > >
> > > > > > David
> > > > > >
> > > > > > On Fri, Jun 17, 2011 at 2:38 PM, Michael Wilde
> > <wilde at mcs.anl.gov>
> > > > > > wrote:
> > > > > > Alberto, how long are you letting it run for, and under what
> > > > > > environment? if you are running on your laptop, how much RAM
> > > > > > do you have? Its possible that you are seeing paging delays
> > > > > > if you are running the Swift Java app with too little memory.
> > > > > >
> > > > > >
> > > > > > Also, are you running trunk or 0.92.1? You should compare the
> > > > > > two.
> > > > > >
> > > > > >
> > > > > > Its *possible* that this simple test is hanging under recent
> > > > > > trunk mods, but its more likely that this is some kind of
> > > > > > resource shortage.
> > > > > >
> > > > > >
> > > > > > Can you run this on one of the Swift lab machines bridled or
> > > > > > communcado, or better yet on the MCS compute servers, or a
> > > > > > PADS worker node (which you can get with qsub -I on pads)?
> > > > > >
> > > > > >
> > > > > > Look at Swift under the "top" command to see if Swift is
> > > > > > running and slow, or is hung.
> > > > > >
> > > > > >
> > > > > > Stop by and we can discuss in more detail.
> > > > > >
> > > > > >
> > > > > > - Mike
> > > > > >
> > > > > >
> > > > > >
> > > > > > ______________________________________________________________
> > > > > >
> > > > > > When I run the following SwiftScript using suite.sh,
> > > > > > the report shows an odd behavior, most of the time it
> > > > > > times out, but once in a while it passes, however this
> > > > > > outcome is completely random, since sometimes that
> > > > > > test has passed 3 times in a row, and all of the
> > > > > > sudden it fails.
> > > > > > This is my script:
> > > > > >
> > > > > >
> > > > > > type messagefile;
> > > > > >
> > > > > >
> > > > > > app (messagefile t) greeting (string s[]) {
> > > > > > echo s[0] s[1] s[2] stdout=@filename(t);
> > > > > > }
> > > > > >
> > > > > >
> > > > > > messagefile outfile <"q5out.txt">;
> > > > > >
> > > > > >
> > > > > > string words[] = ["how","are","you"];
> > > > > >
> > > > > >
> > > > > > outfile = greeting(words);
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > Swift.properties contents:
> > > > > >
> > > > > >
> > > > > > $ cat swift.properties
> > > > > > wrapperlog.always.transfer=true
> > > > > > sitedir.keep=true
> > > > > > execution.retries=0
> > > > > > lazy.errors=false
> > > > > > status.mode=provider
> > > > > > use.provider.staging=false
> > > > > > provider.staging.pin.swiftfiles=false
> > > > > >
> > > > > >
> > > > > > Sites.template.xml contents:
> > > > > >
> > > > > >
> > > > > > $ cat sites.template.xml
> > > > > > <config>
> > > > > > <pool handle="localhost">
> > > > > > <filesystem provider="local" />
> > > > > > <execution provider="coaster"
> > > > > > jobmanager="local:local"/>
> > > > > > <profile namespace="globus"
> > > > > > key="internalHostname">127.0.0.1</profile>
> > > > > > <profile namespace="karajan"
> > > > > > key="jobthrottle">1000</profile>
> > > > > > <profile namespace="karajan"
> > > > > > key="initialScore">10000</profile>
> > > > > > <profile namespace="globus"
> > > > > > key="jobsPerNode">4</profile>
> > > > > > <profile namespace="globus"
> > > > > > key="slots">8</profile>
> > > > > > <profile namespace="globus"
> > > > > > key="maxTime">1000</profile>
> > > > > > <profile namespace="globus"
> > > > > > key="nodeGranularity">1</profile>
> > > > > > <profile namespace="globus"
> > > > > > key="maxNodes">4</profile>
> > > > > > <workdirectory>/tmp</workdirectory>
> > > > > > </pool>
> > > > > > </config>
> > > > > >
> > > > > >
> > > > > > -Alberto
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Swift-devel mailing list
> > > > > > Swift-devel at ci.uchicago.edu
> > > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Michael Wilde
> > > > > > Computation Institute, University of Chicago
> > > > > > Mathematics and Computer Science Division
> > > > > > Argonne National Laboratory
> > > > > >
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Swift-devel mailing list
> > > > > > Swift-devel at ci.uchicago.edu
> > > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Swift-devel mailing list
> > > > > > Swift-devel at ci.uchicago.edu
> > > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > >
> > >
> > >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110617/546a3d1e/attachment.html>
More information about the Swift-devel
mailing list