[Swift-devel] null pointer exception from nested loops

Ben Clifford benc at hawaga.org.uk
Wed Mar 18 07:35:35 CDT 2009 

r2707 fixes this.  It was the nested procedure call interacting badly with 
anything that returns mapped content rather than in-memory values.

On Tue, 17 Mar 2009, Mihael Hategan wrote:

> Looks like it might be the nested procedure call interacting badly with
> readData.
> 
> On Tue, 2009-03-17 at 17:02 -0500, Michael Wilde wrote:
> > OK, I think this is it:
> > 
> > The following script *works* with the commented out code and fails with 
> > the currently enabled alternative statement:
> > 
> > --
> > 
> > type file;
> > 
> > app (file editedParams) setTemps ( file inParams )
> > {
> >    sed "-e" "s/@DTI@/123/" @inParams stdout=@editedParams;
> > }
> > 
> > file inParams<single_file_mapper; file="params.tloop" >;
> > 
> > /* works:
> > file o<"pout">;
> > o = setTemps(inParams);
> > string config [] = readData(o);
> > */
> > 
> > /* Fails: */
> > string config [] = readData( setTemps(inParams ) );
> > 
> > trace(0,config[0]);
> > trace(1,config[1]);
> > trace(2,config[2]);
> > 
> > --
> > 
> > So readData is indeed happy to take a file-value var as an arg but not a 
> > file-valued expression (procedure return in this case).
> > 
> > 
> > 
> > On 3/17/09 4:46 PM, Michael Wilde wrote:
> > > I got the error narrowed down to this example:
> > > 
> > > type file;
> > > 
> > > app (file editedParams) setTemps ( file inParams )
> > > {
> > >   echo @inParams stdout=@editedParams;
> > > }
> > > 
> > > file inParams<single_file_mapper; file="params.tloop" >;
> > > 
> > > string config [] = readData( setTemps(inParams ) );
> > > trace(0,config[0]);
> > > trace(1,config[1]);
> > > trace(2,config[2]);
> > > 
> > > -- 
> > > 
> > > params.tloops contains:
> > > 
> > > DEFAULT_INIT_TEMP_=_ at DTI@
> > > TEMP_UPDATE_INTERVAL_=_ at TUI@
> > > KILL_TIME_=_3
> > > MAX_NUMBER_OF_ANNEALING_STEPS_=_0
> > > 
> > > -- 
> > > 
> > > In other code, readData() seemed happy to take a file *or* a filename 
> > > string as input, but I wonder if it was not as happy as it seemed.
> > > 
> > > I'd been taking advantage of the flexibility with good results (luck?) 
> > > so far, though.
> > > 
> > > On 3/17/09 4:25 PM, Michael Wilde wrote:
> > >> The log contains this just before the NPE, including the suspicious 
> > >> message: WARN  FlowNode Ex098:
> > >>
> > >> Thats giving me a clue as to the offending statements.
> > >>
> > >> ---
> > >>
> > >> 2009-03-17 16:20:34,723-0500 INFO  AbstractDataNode Adding handle 
> > >> listener "F/org.griphyn.vdl.mapping.DataNode identifier 
> > >> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090\
> > >> 317-1620-e1n1bz3g:720000000071 type SecSeq with no value at 
> > >> dataset=secseq path=[0] (not closed)" to 
> > >> "org.griphyn.vdl.mapping.DataNode identifier tag:benc at ci.uchicago.edu,\
> > >> 2008:swift:dataset:20090317-1620-e1n1bz3g:720000000071 type SecSeq 
> > >> with no value at dataset=secseq path=[0] (not closed)"
> > >> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode closed 
> > >> org.griphyn.vdl.mapping.RootDataNode identifier 
> > >> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz\
> > >> 3g:720000000092 type string value=s/@DIT@/10/ dataset=unnamed 
> > >> SwiftScript value (closed)
> > >> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode ROOTPATH 
> > >> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000092 
> > >> path=$
> > >> 2009-03-17 16:20:34,724-0500 INFO  AbstractDataNode VALUE 
> > >> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000092 
> > >> VALUE=s/@DIT@/10/
> > >> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode closed 
> > >> org.griphyn.vdl.mapping.RootDataNode identifier 
> > >> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz\
> > >> 3g:720000000093 type string value=s/@TUI@/1/ dataset=unnamed 
> > >> SwiftScript value (closed)
> > >> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode ROOTPATH 
> > >> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000093 
> > >> path=$
> > >> 2009-03-17 16:20:34,725-0500 INFO  AbstractDataNode VALUE 
> > >> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000093 
> > >> VALUE=s/@TUI@/1/
> > >> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode closed 
> > >> org.griphyn.vdl.mapping.RootDataNode identifier 
> > >> tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz\
> > >> 3g:720000000094 type string value=params.tloop dataset=unnamed 
> > >> SwiftScript value (closed)
> > >> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode ROOTPATH 
> > >> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000094 
> > >> path=$
> > >> 2009-03-17 16:20:34,726-0500 INFO  AbstractDataNode VALUE 
> > >> dataset=tag:benc at ci.uchicago.edu,2008:swift:dataset:20090317-1620-e1n1bz3g:720000000094 
> > >> VALUE=params.tloop
> > >> 2009-03-17 16:20:34,727-0500 WARN  FlowNode Ex098
> > >> java.lang.NullPointerException
> > >>         at 
> > >> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
> > >>
> > >>         at 
> > >> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
> > >>         at 
> > >> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
> > >>         at 
> > >> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
> > >>
> > >>         at 
> > >> org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
> > >>
> > >>
> > >>
> > >> On 3/17/09 4:17 PM, Michael Wilde wrote:
> > >>> It seems not related to scale or Falkon.
> > >>>
> > >>> It occurs when running on localhost (but on bgp) and when I cut all 
> > >>> the loops down to a single iteration.
> > >>>
> > >>> I'm still debugging.
> > >>>
> > >>> On 3/17/09 4:07 PM, Michael Wilde wrote:
> > >>>> I just expanded my oops protein folding script to add another level 
> > >>>> of parameter sweep. This script is getting pretty complex now (at 
> > >>>> least, for a swift script).
> > >>>>
> > >>>> I got the following npe on my first two tries. Im going to start 
> > >>>> debugging, but any clues as to the cause would be helpful.
> > >>>>
> > >>>> The outer loops are:
> > >>>>
> > >>>> main()
> > >>>> {
> > >>>>   string protein[] = readData(@arg("plist"));
> > >>>>   string startTemp[] = ["10","20"];
> > >>>>   string tempUpdate[] = ["1","2","3"];
> > >>>>
> > >>>>   foreach p in protein {
> > >>>>     foreach st in startTemp {
> > >>>>       foreach tu in tempUpdate {
> > >>>>         doRoundSet(p,st,tu);
> > >>>>       }
> > >>>>     }
> > >>>>   }
> > >>>> }
> > >>>>
> > >>>> There are two levels of inner loops further down below doRoundSet().
> > >>>>
> > >>>> The script, output, command line args and log are in:
> > >>>> http://ww.ci.uchicago.edu/~wilde/swift3.tar.gz
> > >>>>
> > >>>> I suspect it will take a while to narrow the cause to a simpler test 
> > >>>> case thats easy tp reproduce without a lot of setup.
> > >>>>
> > >>>> I'll try on a vanilla swift on local execution; this is on bgp with 
> > >>>> Falkon.
> > >>>>
> > >>>> Thanks.
> > >>>>
> > >>>> -- 
> > >>>>
> > >>>> ...
> > >>>> Progress:  uninitialized:1 Selecting site:2
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 7
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 2
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 8
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 0
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 5
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 9
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 1
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 6
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 3
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 4
> > >>>> Ex098
> > >>>> java.lang.NullPointerException
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
> > >>>>
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
> > >>>>
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
> > >>>>
> > >>>> Execution failed:
> > >>>>     java.lang.NullPointerException
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
> > >>>>
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
> > >>>>
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
> > >>>>
> > >>>>
> > >>>> Ex098
> > >>>> java.lang.NullPointerException
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.leafFileName(VDLFunction.java:285) 
> > >>>>
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:201)
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.filename(VDLFunction.java:182)
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.swiftscript.FileName.function(FileName.java:19) 
> > >>>>
> > >>>>     at 
> > >>>> org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:67)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.childCompleted(AbstractSequentialWithArguments.java:192) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.notificationEvent(Sequential.java:33) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:332)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.fireNotificationEvent(FlowNode.java:176) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:296) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:46) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:51) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:27) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.executeChildren(AbstractFunction.java:40) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:233) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:278)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.controlEvent(FlowNode.java:391) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.nodes.FlowNode.event(FlowNode.java:329)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.FlowElementWrapper.event(FlowElementWrapper.java:227) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.send(EventBus.java:125)
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventBus.sendHooked(EventBus.java:99) 
> > >>>>
> > >>>>     at 
> > >>>> org.globus.cog.karajan.workflow.events.EventWorker.run(EventWorker.java:69) 
> > >>>>
> > >>>> SwiftScript trace: T1af7, Round, 0, Sim, 7
> > >>>> _______________________________________________
> > >>>> Swift-devel mailing list
> > >>>> Swift-devel at ci.uchicago.edu
> > >>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > >>> _______________________________________________
> > >>> Swift-devel mailing list
> > >>> Swift-devel at ci.uchicago.edu
> > >>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > >> _______________________________________________
> > >> Swift-devel mailing list
> > >> Swift-devel at ci.uchicago.edu
> > >> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > _______________________________________________
> > > Swift-devel mailing list
> > > Swift-devel at ci.uchicago.edu
> > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
>

More information about the Swift-devel mailing list