[Swift-devel] Fwd: NumberFormatException
Mihael Hategan
hategan at mcs.anl.gov
Wed Jul 13 19:19:58 CDT 2011
Sadly there are many other things that will fail if the swift client is
unreachable from the outside world. Reachability (at least on some
number of ports - TCP_PORT_RANGE) is a prerequisite for swift.
On Wed, 2011-07-13 at 19:09 -0500, Michael Wilde wrote:
> If this is the same error I think it is, its likely due to the fact that Swift was trying to use 3rd party transfer with the Swift client on an EC2 VM which was unreachable from the PADS GridFTP server.
>
> It raised some interesting issues on how Swift tried to do transfers from a file mapped to a gsiftp:// URI and a local execution site which I think are still unresolved but were the likely cause of this error.
>
> I think Jon is reworking this test case to use only remote GridFTp-enabled sites outside of EC2 as the execution sites, and the correct DNS or IP name for inbound connections back to Swift.
>
> - Mike
>
>
> ----- Original Message -----
> > The full log will help here, but the "Reply wait timeout" exception is
> > not necessarily an indication of a bad path (though I'm not saying
> > it's
> > not).
> >
> > In the meantime, can you check that you have GLOBUS_HOSTNAME set
> > properly and that your client machine is not behind a firewall?
> >
> > On Wed, 2011-07-13 at 13:00 -0500, Jonathan Monette wrote:
> > > Mike mentioned that this path is correct and makes sense. I was able
> > > to use the globus-url-copy from the swift bin directory and make a
> > > copy from that GSIURI. The error that I am receiving is
> > >
> > >
> > > Caused by: Exception in getFile
> > > Caused by:
> > > org.globus.cog.abstraction.impl.file.IrrecoverableResourceException:
> > > Exception in getFile
> > > Caused by: org.globus.ftp.exception.ServerException: Reply wait
> > > timeout. (error code 4)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.functions.KException.function(KException.java:29)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:27)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:32)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:340)
> > > at
> > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:181)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:309)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:50)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:26)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:238)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:289)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:402)
> > > at
> > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:343)
> > > at
> > > org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:230)
> > > at
> > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > at
> > > org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:44)
> > > at edu.emory.mathcs.backport.java.util.concurrent.Executors
> > > $RunnableAdapter.call(Executors.java:431)
> > > at
> > > edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor
> > > $Worker.runTask(ThreadPoolExecutor.java:643)
> > > at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor
> > > $Worker.run(ThreadPoolExecutor.java:668)
> > > at java.lang.Thread.run(Thread.java:636)
> > >
> > >
> > > I don't know if I mentioned this but this is with the 0.92.1
> > > release.
> > >
> > >
> > > On Jul 13, 2011, at 11:29 AM, Jonathan Monette wrote:
> > >
> > > > I am starting to believe that this maybe a bug. I have this line
> > > > in
> > > > the log.
> > > >
> > > >
> > > > 2011-07-13 16:17:50,835+0000 DEBUG vdl:dostageinfile
> > > > FILE_STAGE_IN_START file=data.txt
> > > > srchost=gridftp.pads.ci.uchicago.edu:2811
> > > > srcdir=/gpfs/pads/projects/CI-CCR000013/jonmon/Swift/tests/cat_test
> > > > srcname=data.txt desthost=localhost
> > > > destdir=script-20110713-1617-lzk6sxj3/shared/gpfs/pads/projects/CI-CCR000013/jonmon/Swift/tests/cat_test
> > > > provider=gsiftp policy=DEFAULT
> > > >
> > > >
> > > > The desthost is localhost. That is correct. But why does the
> > > > destdir have /gpfs in it? My localhost is VM. It does not have
> > > > access to gpfs.
> > > >
> > > >
> > > > The files for this run are in ~jonmon/run.0002 on the ci machines.
> > > >
> > > >
> > > > On Jul 13, 2011, at 9:13 AM, Michael Wilde wrote:
> > > >
> > > > > I was going to suggest that same as Sarah pointed out,
> > > > > suspecting
> > > > > that the short-form of single_file_mapper is not correctly
> > > > > parsing
> > > > > the GSIFTP URI. If thats the case, please file it as a bug, Jon.
> > > > >
> > > > >
> > > > > Can you try both forms of the mapper syntax with a known-good
> > > > > URI?
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > >
> > > > > - Mike
> > > > >
> > > > >
> > > > >
> > > > > __________________________________________________________________
> > > > > Thanks. I just verified that something maybe wrong with
> > > > > the path. I could not globus-url-copy that URI so I will
> > > > > continue to investigate
> > > > >
> > > > > On Jul 12, 2011, at 4:59 PM, Sarah Kenny wrote:
> > > > >
> > > > > i've used this successfully in the past:
> > > > >
> > > > > file mybrain<single_file_mapper;
> > > > > file="gsiftp://calero.bsd.uchicago.edu/gpfs/pads/projects/stroke_recovery/dude.mgz")>;
> > > > >
> > > > > it's been a while though...haven't tested with
> > > > > the
> > > > > latest swift...
> > > > >
> > > > > On Tue, Jul 12, 2011 at 1:59 PM, Jonathan
> > > > > Monette <jonmon at utexas.edu> wrote:
> > > > > How does this syntax look when mapping
> > > > > with the GSIURI? How does it work? I am
> > > > > still getting errors when trying to map
> > > > > a
> > > > > file with a GSIURI. It says file not
> > > > > found. Maybe I have a different
> > > > > understanding on how it works.
> > > > >
> > > > > The files and the logs are in
> > > > > ~jonmon/run.0001 on the ci machines.
> > > > >
> > > > >
> > > > > On Jul 12, 2011, at 2:16 AM, Mihael
> > > > > Hategan wrote:
> > > > >
> > > > > > You have a colon after the host name
> > > > > > but
> > > > > no port. Either remove the
> > > > > > colon or put a number after it.
> > > > > >
> > > > > > On Mon, 2011-07-11 at 22:45 -0500,
> > > > > Jonathan Monette wrote:
> > > > > >> I actually meant to send this to
> > > > > swift-devel.
> > > > > >>
> > > > > >>
> > > > > >> Here is the background to the
> > > > > >> problem.
> > > > > I have data on PADS, I am
> > > > > >> executing Swift on a VM, and I want
> > > > > >> to
> > > > > use OSG to compute with the
> > > > > >> data. Before Mike left for vacation
> > > > > >> he
> > > > > said that you can map data in
> > > > > >> Swift using the GSIURI scheme but he
> > > > > did not tell me how. What I did
> > > > > >> below is
> > > > > >>
> > > > > >>
> > > > > >> file
> > > > > >>
> > > > > data<"gsiftp://stor01.pads.ci.uchicago.edu:/gpfs/pads/projects/CI-CCR000013/jonmon/Swift/tests/cat_test/data.txt">;
> > > > > >>
> > > > > >>
> > > > > >> This does not seem to work in release
> > > > > 0.92.1 as the error below
> > > > > >> shows. How do you map data in Swift
> > > > > using the GSIURI scheme?
> > > > > >>
> > > > > >> Begin forwarded message:
> > > > > >>
> > > > > >>> From: Jonathan Monette
> > > > > <jonmon at utexas.edu>
> > > > > >>>
> > > > > >>> Date: July 11, 2011 2:30:34 PM CDT
> > > > > >>>
> > > > > >>> To: swift-devel Devel
> > > > > <swift-devel at ci.uchicago.edu>
> > > > > >>>
> > > > > >>> Cc: Mihael Hategan Hategan
> > > > > <hategan at mcs.anl.gov>
> > > > > >>>
> > > > > >>> Subject: NumberFormatException
> > > > > >>>
> > > > > >>>
> > > > > >>> Mihael,
> > > > > >>> I am getting this error using
> > > > > >>> release
> > > > > 0.92.1.
> > > > > >>>
> > > > > >>> 2011-07-11 19:24:44,395+0000 INFO
> > > > > unknown RUNID
> > > > > >>> id=run:20110711-1924-c944yl9c
> > > > > >>> 2011-07-11 19:24:44,508+0000 DEBUG
> > > > > VDL2ExecutionContext vdl:new @
> > > > > >>> script.kml, line: 69:
> > > > > java.lang.RuntimeException:
> > > > > >>> java.lang.NumberFormatException: For
> > > > > input string: ""
> > > > > >>> java.lang.RuntimeException:
> > > > > java.lang.NumberFormatException: For
> > > > > >>> input string: ""
> > > > > >>> Caused by:
> > > > > >>> java.lang.RuntimeException:
> > > > > >>> java.lang.NumberFormatException: For
> > > > > input string: ""
> > > > > >>> at
> > > > > org.griphyn.vdl.mapping.AbsFile.exists(AbsFile.java:109)
> > > > > >>> at
> > > > > >>>
> > > > > org.griphyn.vdl.mapping.file.SingleFileMapper.existing(SingleFileMapper.java:24)
> > > > > >>> at
> > > > > >>>
> > > > > org.griphyn.vdl.mapping.RootDataNode.checkInputs(RootDataNode.java:97)
> > > > > >>> at
> > > > > >>>
> > > > > org.griphyn.vdl.mapping.RootDataNode.checkInputs(RootDataNode.java:75)
> > > > > >>> at
> > > > > >>>
> > > > > org.griphyn.vdl.mapping.RootDataNode.innerInit(RootDataNode.java:61)
> > > > > >>> at
> > > > > org.griphyn.vdl.mapping.RootDataNode.init(RootDataNode.java:37)
> > > > > >>> at
> > > > > org.griphyn.vdl.karajan.lib.New.function(New.java:126)
> > > > > >>> at
> > > > > org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:68)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:32)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:340)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:181)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:309)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.user.UserDefinedElement.childCompleted(UserDefinedElement.java:290)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.user.SequentialImplicitExecutionUDE.childCompleted(SequentialImplicitExecutionUDE.java:85)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:32)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:340)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:181)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:309)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.functions.Argument.post(Argument.java:45)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:32)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:340)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:181)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:309)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.functions.Map_Map.post(Map_Map.java:55)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:50)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.childCompleted(Sequential.java:44)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:32)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:340)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:181)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:309)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > > > > >>> at
> > > > > org.globus.cog.karajan.workflow.nodes.Each.post(Each.java:31)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:32)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:340)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:181)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:309)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:50)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:26)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:238)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:289)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:402)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:343)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:173)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:44)
> > > > > >>> at
> > > > > edu.emory.mathcs.backport.java.util.concurrent.Executors
> > > > > >>>
> > > > > $RunnableAdapter.call(Executors.java:431)
> > > > > >>> at
> > > > > >>>
> > > > > edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:166)
> > > > > >>> at
> > > > > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor
> > > > > >>>
> > > > > $Worker.runTask(ThreadPoolExecutor.java:643)
> > > > > >>> at
> > > > > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor
> > > > > >>>
> > > > > $Worker.run(ThreadPoolExecutor.java:668)
> > > > > >>> at
> > > > > java.lang.Thread.run(Thread.java:636)
> > > > > >>> Caused by:
> > > > > java.lang.NumberFormatException: For
> > > > > input
> > > > > string: ""
> > > > > >>> at
> > > > > >>>
> > > > > java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> > > > > >>> at
> > > > > java.lang.Integer.parseInt(Integer.java:493)
> > > > > >>> at
> > > > > java.lang.Integer.parseInt(Integer.java:514)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.abstraction.impl.common.task.ServiceContactImpl.parse(ServiceContactImpl.java:90)
> > > > > >>> at
> > > > > >>>
> > > > > org.globus.cog.abstraction.impl.common.task.ServiceContactImpl.<init>(ServiceContactImpl.java:27)
> > > > > >>> at
> > > > > org.griphyn.vdl.mapping.AbsFile.getFileResource(AbsFile.java:84)
> > > > > >>> at
> > > > > org.griphyn.vdl.mapping.AbsFile.exists(AbsFile.java:99)
> > > > > >>> ... 63 more
> > > > > >>>
> > > > > >>> The files needed for this run are
> > > > > located in ~jonmon/run.0000 on the
> > > > > >>> ci machines.
> > > > > >>
> > > > > >>
> > > > > >
> > > > > >
> > > > > >
> > > > > _______________________________________________
> > > > > > Swift-devel mailing list
> > > > > > Swift-devel at ci.uchicago.edu
> > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > > > >
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sarah Kenny
> > > > > Programmer
> > > > > University of Chicago, Computation Institute
> > > > > University of California Irvine, Dept. of
> > > > > Neurology
> > > > > 773-818-8300
> > > > >
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Michael Wilde
> > > > > Computation Institute, University of Chicago
> > > > > Mathematics and Computer Science Division
> > > > > Argonne National Laboratory
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > _______________________________________________
> > > Swift-devel mailing list
> > > Swift-devel at ci.uchicago.edu
> > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> >
> >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>
More information about the Swift-devel
mailing list