[Swift-devel] null pointer exception from nested loops

Michael Wilde wilde at mcs.anl.gov
Tue Mar 17 17:05:37 CDT 2009 

Not quite sure what you're asking, Ian.

The latest tests have been on BGP w/ Falkon.
Earlier tests were on other clusters.

The scripts has grown in last week or so, on BGP, and grew some more 
today to explore some new science code algorithm questions.

Its not yet running at full desired scale on the BGP; we are now scaling 
up carefully so as not to impact other users.

This is a test case for the "cio" work as well.

- Mike


On 3/17/09 4:26 PM, Ian Foster wrote:
> Just curious, is the whole thing working with just Falkon?
> 
> 
> On Mar 17, 2009, at 4:25 PM, Michael Wilde wrote:
> 
>> The log contains this just before the NPE, including the suspicious 
>> message: WARN  FlowNode Ex098:
>>
>> Thats giving me a clue as to the offending statements.
>>
>> ---
>>
>> 2009-03-17 16:20:34,723-0500 INFO  AbstractDataNode Adding handle 
>> listener "F/org.griphyn.vdl.mapping.DataNode identifier 
>> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090\
>> 317-1620-e1n1bz3g:720000000071 type SecSeq with no value at 
>> dataset=secseq path=[0] (not closed)" to 
>> "org.griphyn.vdl.mapping.DataNode identifier tag:benc at ci.uchicago.edu,\
>> 2008:swift:dataset:20090317-1620-e1n1bz3g:720000000071 type SecSeq 
>> with no value at dataset=secseq path=[0] (not closed)"
>> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode closed 
>> org.griphyn.vdl.mapping.RootDataNode identifier 
>> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz\
>> 3g:720000000092 type string value=s/@DIT@/10/ dataset=unnamed 
>> SwiftScript value (closed)
>> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode ROOTPATH 
>> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000092 
>> path=$
>> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode VALUE 
>> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000092 
>> VALUE=s/@DIT@/10/
>> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode closed 
>> org.griphyn.vdl.mapping.RootDataNode identifier 
>> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz\
>> 3g:720000000093 type string value=s/@TUI@/1/ dataset=unnamed 
>> SwiftScript value (closed)
>> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode ROOTPATH 
>> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000093 
>> path=$
>> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode VALUE 
>> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000093 
>> VALUE=s/@TUI@/1/
>> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode closed 
>> org.griphyn.vdl.mapping.RootDataNode identifier 
>> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz\
>> 3g:720000000094 type string value=params.tloop dataset=unnamed 
>> SwiftScript value (closed)
>> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode ROOTPATH 
>> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000094 
>> path=$
>> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode VALUE 
>> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000094 
>> VALUE=params.tloop
>> 2009-03-17 16:20:34,727-0500 WARN  FlowNode Ex098
>> java.lang.NullPointerException
>>        at 
>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>
>>        at 
>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>        at 
>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>        at 
>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>
>>        at 
>> org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>
>>
>>
>> On 3/17/09 4:17 PM, Michael Wilde wrote:
>>> It seems not related to scale or Falkon.
>>> It occurs when running on localhost (but on bgp) and when I cut all 
>>> the loops down to a single iteration.
>>> I'm still debugging.
>>> On 3/17/09 4:07 PM, Michael Wilde wrote:
>>>> I just expanded my oops protein folding script to add another level 
>>>> of parameter sweep. This script is getting pretty complex now (at 
>>>> least, for a swift script).
>>>>
>>>> I got the following npe on my first two tries. Im going to start 
>>>> debugging, but any clues as to the cause would be helpful.
>>>>
>>>> The outer loops are:
>>>>
>>>> main()
>>>> {
>>>>  string protein[] = readData(@arg("plist"));
>>>>  string startTemp[] = ["10","20"];
>>>>  string tempUpdate[] = ["1","2","3"];
>>>>
>>>>  foreach p in protein {
>>>>    foreach st in startTemp {
>>>>      foreach tu in tempUpdate {
>>>>        doRoundSet(p,st,tu);
>>>>      }
>>>>    }
>>>>  }
>>>> }
>>>>
>>>> There are two levels of inner loops further down below doRoundSet().
>>>>
>>>> The script, output, command line args and log are in:
>>>> http://ww.ci.uchicago.edu/~wilde/swift3.tar.gz
>>>>
>>>> I suspect it will take a while to narrow the cause to a simpler test 
>>>> case thats easy tp reproduce without a lot of setup.
>>>>
>>>> I'll try on a vanilla swift on local execution; this is on bgp with 
>>>> Falkon.
>>>>
>>>> Thanks.
>>>>
>>>> -- 
>>>>
>>>> ...
>>>> Progress:  uninitialized:1 Selecting site:2
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 7
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 2
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 8
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 0
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 5
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 9
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 1
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 6
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 3
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 4
>>>> Ex098
>>>> java.lang.NullPointerException
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>>>
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>>>
>>>>    at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
>>>>
>>>> Execution failed:
>>>>    java.lang.NullPointerException
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>>>
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>>>
>>>>    at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
>>>>
>>>>
>>>> Ex098
>>>> java.lang.NullPointerException
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>>>
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>>>    at 
>>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>>>
>>>>    at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>
>>>>    at 
>>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
>>>>
>>>> SwiftScript trace: T1af7, Round, 0, Sim, 7
>>>> _______________________________________________
>>>> Swift-devel mailing list
>>>> Swift-devel at ci.uchicago.edu
>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>> _______________________________________________
>>> Swift-devel mailing list
>>> Swift-devel at ci.uchicago.edu
>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>

More information about the Swift-devel mailing list