[Swift-devel] null pointer exception from nested loops

Michael Wilde wilde at mcs.anl.gov
Tue Mar 17 18:03:34 CDT 2009 

I see. No, we've been doing all tests through Swift, but testing the app 
standalone at various points on both local hosts and bgp compute nodes.

On 3/17/09 5:22 PM, Ian Foster wrote:
> I wasn't sure if this ran without Swift--with just Falkon
> 
> 
> On Mar 17, 2009, at 5:05 PM, Michael Wilde wrote:
> 
>> Not quite sure what you're asking, Ian.
>>
>> The latest tests have been on BGP w/ Falkon.
>> Earlier tests were on other clusters.
>>
>> The scripts has grown in last week or so, on BGP, and grew some more 
>> today to explore some new science code algorithm questions.
>>
>> Its not yet running at full desired scale on the BGP; we are now 
>> scaling up carefully so as not to impact other users.
>>
>> This is a test case for the "cio" work as well.
>>
>> - Mike
>>
>>
>> On 3/17/09 4:26 PM, Ian Foster wrote:
>>> Just curious, is the whole thing working with just Falkon?
>>> On Mar 17, 2009, at 4:25 PM, Michael Wilde wrote:
>>>> The log contains this just before the NPE, including the suspicious 
>>>> message: WARN  FlowNode Ex098:
>>>>
>>>> Thats giving me a clue as to the offending statements.
>>>>
>>>> ---
>>>>
>>>> 2009-03-17 16:20:34,723-0500 INFO  AbstractDataNode Adding handle 
>>>> listener "F/org.griphyn.vdl.mapping.DataNode identifier 
>>>> tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090\
>>>> 317-1620-e1n1bz3g:720000000071 type SecSeq with no value at 
>>>> dataset=secseq path=[0] (not closed)" to 
>>>> "org.griphyn.vdl.mapping.DataNode identifier 
>>>> tag:benc at ci.uchicago.edu <mailto:benc at ci.uchicago.edu>,\
>>>> 2008:swift:dataset:20090317-1620-e1n1bz3g:720000000071 type SecSeq 
>>>> with no value at dataset=secseq path=[0] (not closed)"
>>>> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode closed 
>>>> org.griphyn.vdl.mapping.RootDataNode identifier 
>>>> tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz\
>>>> 3g:720000000092 type string value=s/@DIT@/10/ dataset=unnamed 
>>>> SwiftScript value (closed)
>>>> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode ROOTPATH 
>>>> dataset=tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000092 
>>>> path=$
>>>> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode VALUE 
>>>> dataset=tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000092 
>>>> VALUE=s/@DIT@/10/
>>>> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode closed 
>>>> org.griphyn.vdl.mapping.RootDataNode identifier 
>>>> tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz\
>>>> 3g:720000000093 type string value=s/@TUI@/1/ dataset=unnamed 
>>>> SwiftScript value (closed)
>>>> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode ROOTPATH 
>>>> dataset=tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000093 
>>>> path=$
>>>> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode VALUE 
>>>> dataset=tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000093 
>>>> VALUE=s/@TUI@/1/
>>>> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode closed 
>>>> org.griphyn.vdl.mapping.RootDataNode identifier 
>>>> tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz\
>>>> 3g:720000000094 type string value=params.tloop dataset=unnamed 
>>>> SwiftScript value (closed)
>>>> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode ROOTPATH 
>>>> dataset=tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000094 
>>>> path=$
>>>> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode VALUE 
>>>> dataset=tag:benc at ci.uchicago.edu 
>>>> <mailto:benc at ci.uchicago.edu>,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000094 
>>>> VALUE=params.tloop
>>>> 2009-03-17 16:20:34,727-0500 WARN  FlowNode Ex098
>>>> java.lang.NullPointerException
>>>>       at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>>>
>>>>       at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>>>       at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>>>       at 
>>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>>>
>>>>       at 
>>>> org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>>>
>>>>
>>>>
>>>> On 3/17/09 4:17 PM, Michael Wilde wrote:
>>>>> It seems not related to scale or Falkon.
>>>>> It occurs when running on localhost (but on bgp) and when I cut all 
>>>>> the loops down to a single iteration.
>>>>> I'm still debugging.
>>>>> On 3/17/09 4:07 PM, Michael Wilde wrote:
>>>>>> I just expanded my oops protein folding script to add another 
>>>>>> level of parameter sweep. This script is getting pretty complex 
>>>>>> now (at least, for a swift script).
>>>>>>
>>>>>> I got the following npe on my first two tries. Im going to start 
>>>>>> debugging, but any clues as to the cause would be helpful.
>>>>>>
>>>>>> The outer loops are:
>>>>>>
>>>>>> main()
>>>>>> {
>>>>>> string protein[] = readData(@arg("plist"));
>>>>>> string startTemp[] = ["10","20"];
>>>>>> string tempUpdate[] = ["1","2","3"];
>>>>>>
>>>>>> foreach p in protein {
>>>>>>   foreach st in startTemp {
>>>>>>     foreach tu in tempUpdate {
>>>>>>       doRoundSet(p,st,tu);
>>>>>>     }
>>>>>>   }
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> There are two levels of inner loops further down below doRoundSet().
>>>>>>
>>>>>> The script, output, command line args and log are in:
>>>>>> http://ww.ci.uchicago.edu/~wilde/swift3.tar.gz
>>>>>>
>>>>>> I suspect it will take a while to narrow the cause to a simpler 
>>>>>> test case thats easy tp reproduce without a lot of setup.
>>>>>>
>>>>>> I'll try on a vanilla swift on local execution; this is on bgp 
>>>>>> with Falkon.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> -- 
>>>>>>
>>>>>> ...
>>>>>> Progress:  uninitialized:1 Selecting site:2
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 7
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 2
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 8
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 0
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 5
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 9
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 1
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 6
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 3
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 4
>>>>>> Ex098
>>>>>> java.lang.NullPointerException
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>>>>>
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>>>>>
>>>>>>   at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
>>>>>>
>>>>>> Execution failed:
>>>>>>   java.lang.NullPointerException
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>>>>>
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>>>>>
>>>>>>   at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
>>>>>>
>>>>>>
>>>>>> Ex098
>>>>>> java.lang.NullPointerException
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
>>>>>>
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
>>>>>>   at 
>>>>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
>>>>>>
>>>>>>   at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
>>>>>>
>>>>>>   at 
>>>>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
>>>>>>
>>>>>> SwiftScript trace: T1af7, Round, 0, Sim, 7
>>>>>> _______________________________________________
>>>>>> Swift-devel mailing list
>>>>>> Swift-devel at ci.uchicago.edu <mailto:Swift-devel at ci.uchicago.edu>
>>>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>>>> _______________________________________________
>>>>> Swift-devel mailing list
>>>>> Swift-devel at ci.uchicago.edu <mailto:Swift-devel at ci.uchicago.edu>
>>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>>> _______________________________________________
>>>> Swift-devel mailing list
>>>> Swift-devel at ci.uchicago.edu <mailto:Swift-devel at ci.uchicago.edu>
>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>

More information about the Swift-devel mailing list