From lpesce at uchicago.edu Sun Sep 1 08:47:50 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Sun, 1 Sep 2013 08:47:50 -0500 Subject: [Swift-devel] Submitting jobs on Blue Waters In-Reply-To: <161E1527-B78A-4214-8ED5-A951CCA239D9@anl.gov> References: <161E1527-B78A-4214-8ED5-A951CCA239D9@anl.gov> Message-ID: I hope that next year we will have the ppn=32 directive on Beagle too ;-) On Aug 30, 2013, at 5:13 PM, Ian Foster wrote: > Nice! > > On Aug 30, 2013, at 5:01 PM, Tim Armstrong wrote: > >> Hi All, >> I just had some preliminary success in getting Swift/T running on Blue Waters. Mike suggested I should email swift-devel to summarize what that entailed. >> >> Essentially things are the same as on other Cray systems (Beagle, Raven, etc), except the PBS directives for job size are different. You need to use something of the form: >> >> #PBS -l nodes=1:ppn=32 >> >> The regular mpp* directives don't work (and strange things like jobs stuck in the queue seem to happen if you do specify them). >> >> A complete submit script is pasted below for context. >> >> Cheers, >> Tim >> >> >> >> ------------------------------------------------------------------------------------------------- >> >> #PBS -N TURBINE >> #PBS -q normal >> #PBS -l walltime=00:15:00 >> #PBS -o /u/sciteam/tarmstro/turbine-output/2013/08/30/16/09/40/output.txt >> >> # Set the job size using appropriate directives for this system >> #PBS -l nodes=1:ppn=32 >> >> # Pass all environment variables to the job >> #PBS -V >> >> # Merge stdout/stderr >> #PBS -j oe >> # Disable mail >> #PBS -m n >> >> SCRIPT=/mnt/a/u/sciteam/tarmstro/helloworld.tcl >> ARGS="" >> NODES=1 >> WALLTIME=00:15:00 >> TURBINE_OUTPUT=/u/sciteam/tarmstro/turbine-output/2013/08/30/16/09/40 >> TCLSH=/usr/bin/tclsh8.5 >> >> cd ${TURBINE_OUTPUT} >> OUTPUT_FILE=/u/sciteam/tarmstro/turbine-output/2013/08/30/16/09/40/output.txt >> aprun -n 32 -N 32 -cc none -d 1 ${TCLSH} ${SCRIPT} ${ARGS} \ >> 2>&1 > "${OUTPUT_FILE}.${PBS_JOBID}.out" >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From lgadelha at lncc.br Tue Sep 3 06:33:09 2013 From: lgadelha at lncc.br (Luiz Gadelha) Date: Tue, 03 Sep 2013 08:33:09 -0300 Subject: [Swift-devel] Trunk failing with provenance.log=true Message-ID: <5225C8F5.8000302@lncc.br> Hi, I'm getting the error message below when setting the provenance.log parameter to true in the latest trunk version. Both scripts I tested work with provenance.log disabled. Regards, Luiz $ swift -sites.file sites-vortex.xml -tc.file tc-vortex.data blast.swift -d=genes.pep -i=seq3.seq -n=64 Swift trunk swift-r7023 cog-r3768 RunID: 20130903-0731-gg3aup4d Warning: The @ syntax for function invocation is deprecated [Error] sites-vortex.xml, line 1, col 9: cvc-elt.1: Cannot find the declaration of element 'config'. Execution failed: java.lang.UnsupportedOperationException k:concat @ swift.k, line: 29 Caused by: java.lang.UnsupportedOperationException at k.rt.Sink.getAll(Sink.java:45) at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) at org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) at org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) at org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) at org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) at org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) at k.thr.LWThread.run(LWThread.java:228) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) $ swift foreach.swift Swift trunk swift-r7023 cog-r3768 RunID: 20130903-0755-r4lxbb67 Warning: The @ syntax for function invocation is deprecated [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the declaration of element 'config'. Execution failed: java.lang.UnsupportedOperationException k:concat @ swift.k, line: 29 Caused by: java.lang.UnsupportedOperationException at k.rt.Sink.getAll(Sink.java:45) at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) at org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) at org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) at org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) at org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) at org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) at k.thr.LWThread.run(LWThread.java:228) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) -- Luiz Gadelha http://www.lncc.br/~lgadelha From iraicu at cs.iit.edu Tue Sep 3 07:31:01 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Tue, 03 Sep 2013 07:31:01 -0500 Subject: [Swift-devel] CFP: ACM MTAGS 2013 @ SC13 -- deadline extension to 09/15/13 & Journal SI in IEEE Transaction on Cloud Computing Message-ID: <5225D685.5010001@cs.iit.edu> CALL FOR PAPERS 6th Workshop on Many-Task Computing on Clouds, Grids, and Supercomputers (MTAGS) 2013 http://datasys.cs.iit.edu/events/MTAGS13/ Co-located with IEEE/ACM Supercomputing/SC 2013 Denver Colorado -- November 17th, 2013 NEWS ------------------------------------------------------------------------------ - 2 week paper deadline extension -- papers due on 09/15/2013 - Journal Special Issue in the IEEE Transaction on Cloud Computing - Keynote: Dr. Douglas Thain, Associate Professor at University of Notre Dame Overview ------------------------------------------------------------------------------ The 6th workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) will provide the scientific community a dedicated forum for presenting new research, development, and deployment efforts of large-scale many-task computing (MTC) applications on large scale clusters, Grids, Supercomputers, and Cloud Computing infrastructure. MTC, the theme of the workshop encompasses loosely coupled applications, which are generally composed of many tasks (both independent and dependent tasks) to achieve some larger application goal. This workshop will cover challenges that can hamper efficiency and utilization in running applications on large-scale systems, such as local resource manager scalability and granularity, efficient utilization of raw hardware, parallel file system contention and scalability, data management, I/O management, reliability at scale, and application scalability. We welcome paper submissions on all theoretical, simulations, and systems topics related to MTC, but we give special consideration to papers addressing petascale to exascale challenges. Papers will be peer-reviewed, and accepted papers will be published in the workshop proceedings as part of the ACM digital library (pending approval). The workshop will be co-located with the IEEE/ACM Supercomputing 2013 Conference in Denver Colorado on November 17th, 2013. For more information, please see http://datasys.cs.iit.edu/events/MTAGS13/. For more information on past workshops, please see MTAGS12, MTAGS11, MTAGS10, MTAGS09, and MTAGS08. We also ran a Special Issue on Many-Task Computing in the IEEE Transactions on Parallel and Distributed Systems (TPDS) which has appeared in June 2011; the proceedings can be found online at http://www.computer.org/portal/web/csdl/abs/trans/td/2011/06/ttd201106toc.htm. We, the workshop organizers, also published a highly relevant paper that defines Many-Task Computing which was published in MTAGS08, titled Many-Task Computing for Grids and Supercomputers; we encourage potential authors to read this paper, and to clearly articulate in your paper submissions how your papers are related to Many-Task Computing. Topics ------------------------------------------------------------------------------ We invite the submission of original work that is related to the topics below. The papers should be 6 pages, including all figures and references. We aim to cover topics related to Many-Task Computing on each of the three major distributed systems paradigms, Cloud Computing, Grid Computing and Supercomputing. Topics of interest include: Compute Resource Management Scheduling Job execution frameworks Local resource manager extensions Performance evaluation of resource managers in use on large scale systems Dynamic resource provisioning Techniques to manage many-core resources and/or GPUs Challenges and opportunities in running many-task workloads on HPC systems Challenges and opportunities in running many-task workloads on Cloud infrastructure Storage architectures and implementations Distributed file systems Parallel file systems Distributed metadata management Content distribution systems for large data Data caching frameworks and techniques Data management within and across data centers Data-aware scheduling Data-intensive computing applications Eventual-consistency storage usage and management Programming models and tools MapReduce and its generalizations Many-task computing middleware and applications Parallel programming frameworks Ensemble MPI techniques and frameworks Service-oriented science applications Large-Scale Workflow Systems Workflow system performance and scalability analysis Scalability of workflow systems Workflow infrastructure and e-Science middleware Programming paradigms and models Large-Scale Many-Task Applications High-throughput computing (HTC) applications Data-intensive applications Quasi-supercomputing applications, deployments, and experiences Performance Evaluation Performance evaluation Real systems Simulations Reliability of large systems How MTC Addresses Challenges of Petascale and Exascale Computing Concurrency & Programmability I/O & Memory Energy Resilience Heterogeneity Important Dates ------------------------------------------------------------------------------ Abstract submission: September 8, 2013 Paper submission: September 15, 2013 Acceptance notification: October 13, 2013 Final papers due: November 10th, 2013 Paper Submission ------------------------------------------------------------------------------ Authors are invited to submit papers with unpublished, original work of not more than 6 pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages, as per ACM 8.5 x 11 manuscript guidelines; document templates can be found at http://www.acm.org/sigs/publications/proceedings-templates. The final 6 page papers in PDF format must be submitted online at https://cmt.research.microsoft.com/MTAGS2013/ before the deadline of September 15th, 2013 at 11:59PM PST. Papers will be peer-reviewed, and accepted papers will be published in the workshop proceedings as part of the ACM digital library (pending approval). Notifications of the paper decisions will be sent out by October 13th, 2011. Selected excellent work may be eligible for additional post-conference publication as journal articles; we will be running a journal special issue in the IEEE Transaction on Cloud Computing (see http://datasys.cs.iit.edu/events/ScienceCloud2014-TCC/). Submission implies the willingness of at least one of the authors to register and present the paper. For more information, please see http://datasys.cs.iit.edu/events/MTAGS13. Organization ------------------------------------------------------------------------------ General Chairs Ioan Raicu, Illinois Institute of Technology & Argonne National Laboratory, USA Ian Foster, University of Chicago & Argonne National Laboratory, USA Yong Zhao, University of Electronic Science and Technology of China, China Justin Wozniak, Argonne National Laboratory, USA Steering Committee David Abramson, Monash University, Australia Jack Dongarra, University of Tennessee, USA Geoffrey Fox, Indiana University, USA Manish Parashar, Rutgers University, USA Marc Snir, Argonne National Laboratory & University of Illinois at Urbana Champaign, USA Xian-He Sun, Illinois Institute of Technology, USA Weimin Zheng, Tsinghua University, China Program Committee Samer Al-Kiswany (University of British Columbia) Mihai Budiu (Microsoft Research) Kyle Chard (University of Chicago) Yong Chen (Texas Tech University) Evangelinos Constantinos (Massachusetts Institute of Technology) Catalin Dumitrescu (Fermi National Labs) Alexandru Iosup (Delft University of Technology - Netherlands) Florin Isaila (Universidad Carlos III de Madrid ) Kamil Iskra (Argonne National Laboratory) Hui Jin (Oracle Corporation) Daniel Katz (University of Chicago) Zhiling Lan (Illinois Institute of Technology) Mike Lang (Los Alamos National Laboratory) Christopher Moretti (Princeton University) Bogdan Nicolae (IBM Research) David O'Hallaron (Carnegie Mellon University & Intel Laboratory) Marlon Pierce (Indiana University) Judy Qiu (Indiana University) Wei Tang (Argonne National Laboratory) Edward Walker (Whitworth University) Matthew Woitaszek (Walmart Labs) Ken Yocum (University of California at San Diego) Zhifeng Yun (Louisiana State University) Zhao Zhang (University of Chicago) Ziming Zheng (Illinois Institute of Technology) -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud, IEEE/ACM DataCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= From wilde at mcs.anl.gov Tue Sep 3 07:43:04 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Tue, 3 Sep 2013 07:43:04 -0500 (CDT) Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: <5225C8F5.8000302@lncc.br> Message-ID: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> Luiz, in trunk, stricter XML parsing is used now. You need to start sites.xml with these two lines: We need to start using the README and Download pages to make people aware of changes like this. - Mike ----- Original Message ----- > From: "Luiz Gadelha" > To: swift-devel at ci.uchicago.edu > Sent: Tuesday, September 3, 2013 6:33:09 AM > Subject: [Swift-devel] Trunk failing with provenance.log=true > > Hi, > > I'm getting the error message below when setting the provenance.log > parameter to true in the latest trunk version. Both scripts I tested > work with provenance.log disabled. > > Regards, > > Luiz > > $ swift -sites.file sites-vortex.xml -tc.file tc-vortex.data > blast.swift > -d=genes.pep -i=seq3.seq -n=64 > Swift trunk swift-r7023 cog-r3768 > RunID: 20130903-0731-gg3aup4d > Warning: The @ syntax for function invocation is deprecated > [Error] sites-vortex.xml, line 1, col 9: cvc-elt.1: Cannot find the > declaration of element 'config'. > > Execution failed: > java.lang.UnsupportedOperationException > k:concat @ swift.k, line: 29 > Caused by: java.lang.UnsupportedOperationException > at k.rt.Sink.getAll(Sink.java:45) > at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > at > org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > at > org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > at > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > at > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > at k.thr.LWThread.run(LWThread.java:228) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > > > $ swift foreach.swift > Swift trunk swift-r7023 cog-r3768 > RunID: 20130903-0755-r4lxbb67 > Warning: The @ syntax for function invocation is deprecated > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > declaration > of element 'config'. > > Execution failed: > java.lang.UnsupportedOperationException > k:concat @ swift.k, line: 29 > Caused by: java.lang.UnsupportedOperationException > at k.rt.Sink.getAll(Sink.java:45) > at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > at > org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > at > org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > at > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > at > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > at k.thr.LWThread.run(LWThread.java:228) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > > -- > Luiz Gadelha > http://www.lncc.br/~lgadelha > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From hategan at mcs.anl.gov Tue Sep 3 10:08:05 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 03 Sep 2013 08:08:05 -0700 Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> References: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> Message-ID: <1378220885.30133.1.camel@echo> While that is true, schema checking of the sites file is optional. You'll get the warning seen below and swift should continue. I believe that the other error is due to the run-time. I'll take a look when I can. Mihael On Tue, 2013-09-03 at 07:43 -0500, Michael Wilde wrote: > Luiz, in trunk, stricter XML parsing is used now. You need to start sites.xml with these two lines: > > > > > We need to start using the README and Download pages to make people aware of changes like this. > > - Mike > > > ----- Original Message ----- > > From: "Luiz Gadelha" > > To: swift-devel at ci.uchicago.edu > > Sent: Tuesday, September 3, 2013 6:33:09 AM > > Subject: [Swift-devel] Trunk failing with provenance.log=true > > > > Hi, > > > > I'm getting the error message below when setting the provenance.log > > parameter to true in the latest trunk version. Both scripts I tested > > work with provenance.log disabled. > > > > Regards, > > > > Luiz > > > > $ swift -sites.file sites-vortex.xml -tc.file tc-vortex.data > > blast.swift > > -d=genes.pep -i=seq3.seq -n=64 > > Swift trunk swift-r7023 cog-r3768 > > RunID: 20130903-0731-gg3aup4d > > Warning: The @ syntax for function invocation is deprecated > > [Error] sites-vortex.xml, line 1, col 9: cvc-elt.1: Cannot find the > > declaration of element 'config'. > > > > Execution failed: > > java.lang.UnsupportedOperationException > > k:concat @ swift.k, line: 29 > > Caused by: java.lang.UnsupportedOperationException > > at k.rt.Sink.getAll(Sink.java:45) > > at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > > at > > org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > > at > > org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > at > > org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > > at > > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > > at > > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > > at k.thr.LWThread.run(LWThread.java:228) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:679) > > > > > > $ swift foreach.swift > > Swift trunk swift-r7023 cog-r3768 > > RunID: 20130903-0755-r4lxbb67 > > Warning: The @ syntax for function invocation is deprecated > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > declaration > > of element 'config'. > > > > Execution failed: > > java.lang.UnsupportedOperationException > > k:concat @ swift.k, line: 29 > > Caused by: java.lang.UnsupportedOperationException > > at k.rt.Sink.getAll(Sink.java:45) > > at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > > at > > org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > > at > > org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > at > > org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > > at > > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > > at > > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > > at k.thr.LWThread.run(LWThread.java:228) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:679) > > > > -- > > Luiz Gadelha > > http://www.lncc.br/~lgadelha > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From ketancmaheshwari at gmail.com Tue Sep 3 15:08:29 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Tue, 3 Sep 2013 15:08:29 -0500 Subject: [Swift-devel] ui http error on browser In-Reply-To: References: <1377647516.11804.3.camel@echo> <1377741039.4674.0.camel@echo> Message-ID: Mihael, Which directory does the http serve index.html from? On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari < ketancmaheshwari at gmail.com> wrote: > Please find attached. > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan wrote: > >> Odd. Can you send me the swift jar file from the dist/[...]/lib dir? >> >> Mihael >> >> On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari wrote: >> > Hi Mihael, >> > >> > I tried with the updated version. The log is attached. >> > >> > In the browser, I type: localhost:8080 >> > >> > The exact message is same as earlier: >> > Error: The requested resource is not available (in large fonts) >> > >> > Thanks, >> > Ketan >> > >> >> >> > > > -- > Ketan > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Sep 3 16:27:06 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 03 Sep 2013 14:27:06 -0700 Subject: [Swift-devel] ui http error on browser In-Reply-To: References: <1377647516.11804.3.camel@echo> <1377741039.4674.0.camel@echo> Message-ID: <1378243626.471.0.camel@echo> It's in the jar file: httpmonitor/ In the source tree it would be in resources/httpmonitor/ Mihael On Tue, 2013-09-03 at 15:08 -0500, Ketan Maheshwari wrote: > Mihael, > > Which directory does the http serve index.html from? > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari < > ketancmaheshwari at gmail.com> wrote: > > > Please find attached. > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan wrote: > > > >> Odd. Can you send me the swift jar file from the dist/[...]/lib dir? > >> > >> Mihael > >> > >> On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari wrote: > >> > Hi Mihael, > >> > > >> > I tried with the updated version. The log is attached. > >> > > >> > In the browser, I type: localhost:8080 > >> > > >> > The exact message is same as earlier: > >> > Error: The requested resource is not available (in large fonts) > >> > > >> > Thanks, > >> > Ketan > >> > > >> > >> > >> > > > > > > -- > > Ketan > > > > > > > Mihael, > > > Which directory does the http serve index.html from? > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari > wrote: > Please find attached. > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan > wrote: > Odd. Can you send me the swift jar file from the > dist/[...]/lib dir? > > Mihael > > On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari > wrote: > > Hi Mihael, > > > > I tried with the updated version. The log is > attached. > > > > In the browser, I type: localhost:8080 > > > > The exact message is same as earlier: > > Error: The requested resource is not available (in > large fonts) > > > > Thanks, > > Ketan > > > > > > > > > > -- > Ketan > > > > > > -- > Ketan > > From hategan at mcs.anl.gov Tue Sep 3 17:27:48 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 03 Sep 2013 15:27:48 -0700 Subject: [Swift-devel] ui http error on browser In-Reply-To: <1378243626.471.0.camel@echo> References: <1377647516.11804.3.camel@echo> <1377741039.4674.0.camel@echo> <1378243626.471.0.camel@echo> Message-ID: <1378247268.471.3.camel@echo> There is a fix in SVN for this. The issue was that there was an extra "/" between the path elements. I.e.: httpmonitor//index.html. I was running this from eclipse and 'resources' was a filesystem directory. The lookup went fine. However, when the same path points to something in a jar file (as happens in normal swift invocations), the lookup fails. Mihael On Tue, 2013-09-03 at 14:27 -0700, Mihael Hategan wrote: > It's in the jar file: httpmonitor/ > > In the source tree it would be in resources/httpmonitor/ > > Mihael > > On Tue, 2013-09-03 at 15:08 -0500, Ketan Maheshwari wrote: > > Mihael, > > > > Which directory does the http serve index.html from? > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari < > > ketancmaheshwari at gmail.com> wrote: > > > > > Please find attached. > > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan wrote: > > > > > >> Odd. Can you send me the swift jar file from the dist/[...]/lib dir? > > >> > > >> Mihael > > >> > > >> On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari wrote: > > >> > Hi Mihael, > > >> > > > >> > I tried with the updated version. The log is attached. > > >> > > > >> > In the browser, I type: localhost:8080 > > >> > > > >> > The exact message is same as earlier: > > >> > Error: The requested resource is not available (in large fonts) > > >> > > > >> > Thanks, > > >> > Ketan > > >> > > > >> > > >> > > >> > > > > > > > > > -- > > > Ketan > > > > > > > > > > > > Mihael, > > > > > > Which directory does the http serve index.html from? > > > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari > > wrote: > > Please find attached. > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan > > wrote: > > Odd. Can you send me the swift jar file from the > > dist/[...]/lib dir? > > > > Mihael > > > > On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari > > wrote: > > > Hi Mihael, > > > > > > I tried with the updated version. The log is > > attached. > > > > > > In the browser, I type: localhost:8080 > > > > > > The exact message is same as earlier: > > > Error: The requested resource is not available (in > > large fonts) > > > > > > Thanks, > > > Ketan > > > > > > > > > > > > > > > > > > > -- > > Ketan > > > > > > > > > > > > -- > > Ketan > > > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From ketancmaheshwari at gmail.com Tue Sep 3 17:57:08 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Tue, 3 Sep 2013 17:57:08 -0500 Subject: [Swift-devel] ui http error on browser In-Reply-To: <1378247268.471.3.camel@echo> References: <1377647516.11804.3.camel@echo> <1377741039.4674.0.camel@echo> <1378243626.471.0.camel@echo> <1378247268.471.3.camel@echo> Message-ID: Works now, Thanks! I see summary tab showing graphical output. The other tabs are showing numbers. Also, the Graph tab shows tabs garbled on my browser. See attached screenshot. Thanks, Ketan On Tue, Sep 3, 2013 at 5:27 PM, Mihael Hategan wrote: > There is a fix in SVN for this. > > The issue was that there was an extra "/" between the path elements. > I.e.: httpmonitor//index.html. > > I was running this from eclipse and 'resources' was a filesystem > directory. The lookup went fine. However, when the same path points to > something in a jar file (as happens in normal swift invocations), the > lookup fails. > > Mihael > > On Tue, 2013-09-03 at 14:27 -0700, Mihael Hategan wrote: > > It's in the jar file: httpmonitor/ > > > > In the source tree it would be in resources/httpmonitor/ > > > > Mihael > > > > On Tue, 2013-09-03 at 15:08 -0500, Ketan Maheshwari wrote: > > > Mihael, > > > > > > Which directory does the http serve index.html from? > > > > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari < > > > ketancmaheshwari at gmail.com> wrote: > > > > > > > Please find attached. > > > > > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan >wrote: > > > > > > > >> Odd. Can you send me the swift jar file from the dist/[...]/lib dir? > > > >> > > > >> Mihael > > > >> > > > >> On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari wrote: > > > >> > Hi Mihael, > > > >> > > > > >> > I tried with the updated version. The log is attached. > > > >> > > > > >> > In the browser, I type: localhost:8080 > > > >> > > > > >> > The exact message is same as earlier: > > > >> > Error: The requested resource is not available (in large fonts) > > > >> > > > > >> > Thanks, > > > >> > Ketan > > > >> > > > > >> > > > >> > > > >> > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > > > > > > Mihael, > > > > > > > > > Which directory does the http serve index.html from? > > > > > > > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari > > > wrote: > > > Please find attached. > > > > > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan > > > wrote: > > > Odd. Can you send me the swift jar file from the > > > dist/[...]/lib dir? > > > > > > Mihael > > > > > > On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari > > > wrote: > > > > Hi Mihael, > > > > > > > > I tried with the updated version. The log is > > > attached. > > > > > > > > In the browser, I type: localhost:8080 > > > > > > > > The exact message is same as earlier: > > > > Error: The requested resource is not available (in > > > large fonts) > > > > > > > > Thanks, > > > > Ketan > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Ketan > > > > > > > > > > > > > > > > > > -- > > > Ketan > > > > > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: swiftuihttp.png Type: image/png Size: 495015 bytes Desc: not available URL: From hategan at mcs.anl.gov Tue Sep 3 18:44:18 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 03 Sep 2013 16:44:18 -0700 Subject: [Swift-devel] ui http error on browser In-Reply-To: References: <1377647516.11804.3.camel@echo> <1377741039.4674.0.camel@echo> <1378243626.471.0.camel@echo> <1378247268.471.3.camel@echo> Message-ID: <1378251858.471.5.camel@echo> I noticed those. I'll remove the text stuff. Was only there for debugging. The graphs tab should be fine if you click on "+" to add a graph. You can then move and resize the graph and add data series to it. Mihael On Tue, 2013-09-03 at 17:57 -0500, Ketan Maheshwari wrote: > Works now, Thanks! > > I see summary tab showing graphical output. The other tabs are showing > numbers. > > Also, the Graph tab shows tabs garbled on my browser. See attached > screenshot. > > Thanks, > Ketan > > > On Tue, Sep 3, 2013 at 5:27 PM, Mihael Hategan wrote: > > > There is a fix in SVN for this. > > > > The issue was that there was an extra "/" between the path elements. > > I.e.: httpmonitor//index.html. > > > > I was running this from eclipse and 'resources' was a filesystem > > directory. The lookup went fine. However, when the same path points to > > something in a jar file (as happens in normal swift invocations), the > > lookup fails. > > > > Mihael > > > > On Tue, 2013-09-03 at 14:27 -0700, Mihael Hategan wrote: > > > It's in the jar file: httpmonitor/ > > > > > > In the source tree it would be in resources/httpmonitor/ > > > > > > Mihael > > > > > > On Tue, 2013-09-03 at 15:08 -0500, Ketan Maheshwari wrote: > > > > Mihael, > > > > > > > > Which directory does the http serve index.html from? > > > > > > > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari < > > > > ketancmaheshwari at gmail.com> wrote: > > > > > > > > > Please find attached. > > > > > > > > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan > >wrote: > > > > > > > > > >> Odd. Can you send me the swift jar file from the dist/[...]/lib dir? > > > > >> > > > > >> Mihael > > > > >> > > > > >> On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari wrote: > > > > >> > Hi Mihael, > > > > >> > > > > > >> > I tried with the updated version. The log is attached. > > > > >> > > > > > >> > In the browser, I type: localhost:8080 > > > > >> > > > > > >> > The exact message is same as earlier: > > > > >> > Error: The requested resource is not available (in large fonts) > > > > >> > > > > > >> > Thanks, > > > > >> > Ketan > > > > >> > > > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > > > -- > > > > > Ketan > > > > > > > > > > > > > > > > > > > > > > Mihael, > > > > > > > > > > > > Which directory does the http serve index.html from? > > > > > > > > > > > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari > > > > wrote: > > > > Please find attached. > > > > > > > > > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan > > > > wrote: > > > > Odd. Can you send me the swift jar file from the > > > > dist/[...]/lib dir? > > > > > > > > Mihael > > > > > > > > On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari > > > > wrote: > > > > > Hi Mihael, > > > > > > > > > > I tried with the updated version. The log is > > > > attached. > > > > > > > > > > In the browser, I type: localhost:8080 > > > > > > > > > > The exact message is same as earlier: > > > > > Error: The requested resource is not available (in > > > > large fonts) > > > > > > > > > > Thanks, > > > > > Ketan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > Works now, Thanks! > > > I see summary tab showing graphical output. The other tabs are showing > numbers. > > > Also, the Graph tab shows tabs garbled on my browser. See attached > screenshot. > > > Thanks, > > Ketan > > > > On Tue, Sep 3, 2013 at 5:27 PM, Mihael Hategan > wrote: > There is a fix in SVN for this. > > The issue was that there was an extra "/" between the path > elements. > I.e.: httpmonitor//index.html. > > I was running this from eclipse and 'resources' was a > filesystem > directory. The lookup went fine. However, when the same path > points to > something in a jar file (as happens in normal swift > invocations), the > lookup fails. > > Mihael > > On Tue, 2013-09-03 at 14:27 -0700, Mihael Hategan wrote: > > It's in the jar file: httpmonitor/ > > > > In the source tree it would be in resources/httpmonitor/ > > > > Mihael > > > > On Tue, 2013-09-03 at 15:08 -0500, Ketan Maheshwari wrote: > > > Mihael, > > > > > > Which directory does the http serve index.html from? > > > > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari < > > > ketancmaheshwari at gmail.com> wrote: > > > > > > > Please find attached. > > > > > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan > wrote: > > > > > > > >> Odd. Can you send me the swift jar file from the > dist/[...]/lib dir? > > > >> > > > >> Mihael > > > >> > > > >> On Wed, 2013-08-28 at 20:35 -0500, Ketan Maheshwari > wrote: > > > >> > Hi Mihael, > > > >> > > > > >> > I tried with the updated version. The log is > attached. > > > >> > > > > >> > In the browser, I type: localhost:8080 > > > >> > > > > >> > The exact message is same as earlier: > > > >> > Error: The requested resource is not available (in > large fonts) > > > >> > > > > >> > Thanks, > > > >> > Ketan > > > >> > > > > >> > > > >> > > > >> > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > > > > > > Mihael, > > > > > > > > > Which directory does the http serve index.html from? > > > > > > > > > > > > On Wed, Aug 28, 2013 at 9:41 PM, Ketan Maheshwari > > > wrote: > > > Please find attached. > > > > > > > > > > > > On Wed, Aug 28, 2013 at 8:50 PM, Mihael Hategan > > > wrote: > > > Odd. Can you send me the swift jar file > from the > > > dist/[...]/lib dir? > > > > > > Mihael > > > > > > On Wed, 2013-08-28 at 20:35 -0500, Ketan > Maheshwari > > > wrote: > > > > Hi Mihael, > > > > > > > > I tried with the updated version. The > log is > > > attached. > > > > > > > > In the browser, I type: localhost:8080 > > > > > > > > The exact message is same as earlier: > > > > Error: The requested resource is not > available (in > > > large fonts) > > > > > > > > Thanks, > > > > Ketan > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Ketan > > > > > > > > > > > > > > > > > > -- > > > Ketan > > > > > > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > -- > Ketan > > From lgadelha at lncc.br Tue Sep 3 21:57:26 2013 From: lgadelha at lncc.br (Luiz Gadelha) Date: Tue, 03 Sep 2013 23:57:26 -0300 Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: <1378220885.30133.1.camel@echo> References: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> <1378220885.30133.1.camel@echo> Message-ID: <5226A196.8040605@lncc.br> That's indeed what happens, I get the warning about the sites file and the scripts execute correctly with provenance.log disabled. Thanks, Luiz On 09/03/2013 12:08 PM, Mihael Hategan wrote: > While that is true, schema checking of the sites file is optional. > You'll get the warning seen below and swift should continue. > > I believe that the other error is due to the run-time. I'll take a look > when I can. > > Mihael > > On Tue, 2013-09-03 at 07:43 -0500, Michael Wilde wrote: >> Luiz, in trunk, stricter XML parsing is used now. You need to start sites.xml with these two lines: >> >> >> >> >> We need to start using the README and Download pages to make people aware of changes like this. >> >> - Mike >> >> >> ----- Original Message ----- >>> From: "Luiz Gadelha" >>> To: swift-devel at ci.uchicago.edu >>> Sent: Tuesday, September 3, 2013 6:33:09 AM >>> Subject: [Swift-devel] Trunk failing with provenance.log=true >>> >>> Hi, >>> >>> I'm getting the error message below when setting the provenance.log >>> parameter to true in the latest trunk version. Both scripts I tested >>> work with provenance.log disabled. >>> >>> Regards, >>> >>> Luiz >>> >>> $ swift -sites.file sites-vortex.xml -tc.file tc-vortex.data >>> blast.swift >>> -d=genes.pep -i=seq3.seq -n=64 >>> Swift trunk swift-r7023 cog-r3768 >>> RunID: 20130903-0731-gg3aup4d >>> Warning: The @ syntax for function invocation is deprecated >>> [Error] sites-vortex.xml, line 1, col 9: cvc-elt.1: Cannot find the >>> declaration of element 'config'. >>> >>> Execution failed: >>> java.lang.UnsupportedOperationException >>> k:concat @ swift.k, line: 29 >>> Caused by: java.lang.UnsupportedOperationException >>> at k.rt.Sink.getAll(Sink.java:45) >>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) >>> at >>> org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) >>> at >>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>> at >>> org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) >>> at >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) >>> at >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>> at k.thr.LWThread.run(LWThread.java:228) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:679) >>> >>> >>> $ swift foreach.swift >>> Swift trunk swift-r7023 cog-r3768 >>> RunID: 20130903-0755-r4lxbb67 >>> Warning: The @ syntax for function invocation is deprecated >>> [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the >>> declaration >>> of element 'config'. >>> >>> Execution failed: >>> java.lang.UnsupportedOperationException >>> k:concat @ swift.k, line: 29 >>> Caused by: java.lang.UnsupportedOperationException >>> at k.rt.Sink.getAll(Sink.java:45) >>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) >>> at >>> org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) >>> at >>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>> at >>> org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) >>> at >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) >>> at >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>> at k.thr.LWThread.run(LWThread.java:228) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:679) >>> >>> -- >>> Luiz Gadelha >>> http://www.lncc.br/~lgadelha >>> _______________________________________________ >>> Swift-devel mailing list >>> Swift-devel at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -- Luiz Gadelha http://www.lncc.br/~lgadelha From tim.g.armstrong at gmail.com Wed Sep 4 12:23:55 2013 From: tim.g.armstrong at gmail.com (Tim Armstrong) Date: Wed, 4 Sep 2013 12:23:55 -0500 Subject: [Swift-devel] Appendable data structures in Swift/K and Swift/T Message-ID: Hi All, I've been working on adding appendable data structures to Swift/T. I thought I would email to clarify a couple of points about the [auto] arrays in Swift/K and also to get feedback on the Swift/T implementation. Ideally we'd do things the same way in both but I'm not confident that the Swift/K semantics will work well for us. First, clarifying questions, since I'm not 100% clear on what happens when you "use auto key values from one array to access another", e.g. the example: ====================== int[auto] a; int[auto] b; append(a, 1); append(a, 2); foreach v, k in a { b[k] = a[k] * 2; } ===================== There are a few corner cases I don't quite understand in the above example. Are there any guarantees about how the keys are assigned, e.g. in order from 0? What is the type of k? Can you treat it as an integer e.g. to trace? Also, is it possible to double-assign an index by doing that? E.g. if I append(b, 1); before the loop, is it possible for that to conflict with one of the b[k] = ... assignments? Anyway, in Swift/T I've been playing around with an unordered set data structure: a "bag": with some similar functionality. There were a couple of reasons I didn't just clone the Swift/K version: - We can't efficiently sequentially assign indices if the data structure is split across multiple nodes (currently not the case, but possible in future). - The append(A, x) operation that mutates it's first argument is a little awkward semantically/implementation-wise, since it looks like a function call but can't be handled as one: the first argument is an LValue whereas a function argument is an RValue. The logic for handling L/RValues in Swift/T is quite different in the grammar/frontend/semantic analyser. I can easily enough make "append" a reserved word and special-case it in the grammar to work around this, but it feels a bit "wrong" and inconsistent to have something that looks like a function that's doesn't obey the semantics of a function. I remember Swift/K had a += operator at one point - what was the argument for moving away from it? Currently this is what it looks like (with the Swift/K equivalent for comparison). I'd be curious to hear people's thoughts are on how to improve it. Swift/T (draft): =================== bag X; X += 1; X += 1; X += 2; foreach x in X { trace(x); } =================== Swift/K equivalent: =================== int X[auto]; append(X, 1); append(X, 1); append(X, 2); foreach x in X { trace(x); } =================== - Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Wed Sep 4 15:01:17 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 04 Sep 2013 13:01:17 -0700 Subject: [Swift-devel] Appendable data structures in Swift/K and Swift/T In-Reply-To: References: Message-ID: <1378324877.17687.18.camel@echo> On Wed, 2013-09-04 at 12:23 -0500, Tim Armstrong wrote: > Hi All, > > I've been working on adding appendable data structures to Swift/T. I > thought I would email to clarify a couple of points about the [auto] arrays > in Swift/K and also to get feedback on the Swift/T implementation. Ideally > we'd do things the same way in both but I'm not confident that the Swift/K > semantics will work well for us. > > First, clarifying questions, since I'm not 100% clear on what happens when > you "use auto key values from one array to access another", e.g. the > example: > > ====================== > int[auto] a; > int[auto] b; > > append(a, 1); > append(a, 2); > > foreach v, k in a { > b[k] = a[k] * 2; > } > ===================== > > There are a few corner cases I don't quite understand in the above example. > Are there any guarantees about how the keys are assigned, e.g. in order > from 0? I don't think foreach guarantees an order at all, regardless of the key types. > What is the type of k? auto :) Some unspecified internal representation that, from the perspective of the type system, is different from every other type (except a universal type if we had one). Internally it is a string at the moment. I think a sequential integer might be an alternative, but I suspect there is a reason why we didn't pick it. > Can you treat it as an integer e.g. to > trace? No. It cannot safely be treated as anything that tracef currently has. But %s would probably work by accident. > > Also, is it possible to double-assign an index by doing that? E.g. if I > append(b, 1); before the loop, is it possible for that to conflict with one > of the b[k] = ... assignments? No. Which is probably why we use source location + thread ID instead of integers as keys. You can't have two different assignment statements in the same source location and running in the same thread. > > Anyway, in Swift/T I've been playing around with an unordered set data > structure: a "bag": with some similar functionality. There were a couple > of reasons I didn't just clone the Swift/K version: > > - We can't efficiently sequentially assign indices if the data structure > is split across multiple nodes (currently not the case, but possible in > future). DHTs? > - The append(A, x) operation that mutates it's first argument is a > little awkward semantically/implementation-wise, since it looks like a > function call but can't be handled as one: the first argument is an LValue > whereas a function argument is an RValue. The logic for handling L/RValues > in Swift/T is quite different in the grammar/frontend/semantic analyser. I > can easily enough make "append" a reserved word and special-case it in the > grammar to work around this, but it feels a bit "wrong" and inconsistent to > have something that looks like a function that's doesn't obey the semantics > of a function. I remember Swift/K had a += operator at one point - what > was the argument for moving away from it? I think that we voted on 'a << value', and that is still supported. Quoting from the swift parser's list of operators: GE : ">="; APPEND : "<<"; ASSIGN : '=' ; I can't find the emails in the online archive, but I do have this in my inbox: Subject: Re: [Swift-devel] New syntax for appending to arrays Date: Tue, 28 Jun 2011 15:48:05 -0500 (06/28/11 13:48:05) > So far we have: > 3, 0, 3, 5, 3, 3, (0, 1, 5) > > Anybody else? > > On Tue, 2011-06-28 at 14:29 -0500, Mihael Hategan wrote: > > On Tue, 2011-06-28 at 09:08 -0500, Ian Foster wrote: > > > $0,02 from me -- given that Swift has a C-like syntax, this syntax will surely confuse people?? > > > > "+=" is a bit confusing, but then I don't think it's more so than using > > "+" as a string concatenation operator. > > > > So please vote or propose an alternative: > > > > 0. a += 1; > > 1. a ++= 1; > > 2. a ++ 1; > > 3. a << 1; > > 4. a[] = 1; > > 5. a .= 1; Mihael From tim.g.armstrong at gmail.com Wed Sep 4 15:23:23 2013 From: tim.g.armstrong at gmail.com (Tim Armstrong) Date: Wed, 4 Sep 2013 15:23:23 -0500 Subject: [Swift-devel] Appendable data structures in Swift/K and Swift/T In-Reply-To: <1378324877.17687.18.camel@echo> References: <1378324877.17687.18.camel@echo> Message-ID: Awesome, thanks, that clears things up. For some reason I was thinking that [auto] was something like an auto-incremented integer. Does that mean that "b[k] = x" and append(b, x) are semantically identical in these two code fragments? ============== foreach v, k in a { b[k] = a[k] * 2; } --------------------------- foreach v in a { append(b, a[k] * 2); } ============== Also, I was wondering how this interacts with mappers? E.g. if a have an [auto] array mapped with simple_mapper, then what filenames would be produced for something like this? file outfile[] ; append(outfile , ...); append(outfile , ...); - Tim On Wed, Sep 4, 2013 at 3:01 PM, Mihael Hategan wrote: > On Wed, 2013-09-04 at 12:23 -0500, Tim Armstrong wrote: > > Hi All, > > > > I've been working on adding appendable data structures to Swift/T. I > > thought I would email to clarify a couple of points about the [auto] > arrays > > in Swift/K and also to get feedback on the Swift/T implementation. > Ideally > > we'd do things the same way in both but I'm not confident that the > Swift/K > > semantics will work well for us. > > > > First, clarifying questions, since I'm not 100% clear on what happens > when > > you "use auto key values from one array to access another", e.g. the > > example: > > > > ====================== > > int[auto] a; > > int[auto] b; > > > > append(a, 1); > > append(a, 2); > > > > foreach v, k in a { > > b[k] = a[k] * 2; > > } > > ===================== > > > > There are a few corner cases I don't quite understand in the above > example. > > Are there any guarantees about how the keys are assigned, e.g. in order > > from 0? > > I don't think foreach guarantees an order at all, regardless of the key > types. > > > What is the type of k? > > auto :) > > Some unspecified internal representation that, from the perspective of > the type system, is different from every other type (except a universal > type if we had one). > > Internally it is a string at the moment. I think a sequential integer > might be an alternative, but I suspect there is a reason why we didn't > pick it. > > > Can you treat it as an integer e.g. to > > trace? > > No. It cannot safely be treated as anything that tracef currently has. > But %s would probably work by accident. > > > > > Also, is it possible to double-assign an index by doing that? E.g. if I > > append(b, 1); before the loop, is it possible for that to conflict with > one > > of the b[k] = ... assignments? > > No. Which is probably why we use source location + thread ID instead of > integers as keys. You can't have two different assignment statements in > the same source location and running in the same thread. > > > > > Anyway, in Swift/T I've been playing around with an unordered set data > > structure: a "bag": with some similar functionality. There were a couple > > of reasons I didn't just clone the Swift/K version: > > > > - We can't efficiently sequentially assign indices if the data > structure > > is split across multiple nodes (currently not the case, but possible > in > > future). > > DHTs? > > > - The append(A, x) operation that mutates it's first argument is a > > little awkward semantically/implementation-wise, since it looks like a > > function call but can't be handled as one: the first argument is an > LValue > > whereas a function argument is an RValue. The logic for handling > L/RValues > > in Swift/T is quite different in the grammar/frontend/semantic > analyser. I > > can easily enough make "append" a reserved word and special-case it > in the > > grammar to work around this, but it feels a bit "wrong" and > inconsistent to > > have something that looks like a function that's doesn't obey the > semantics > > of a function. I remember Swift/K had a += operator at one point - > what > > was the argument for moving away from it? > > I think that we voted on 'a << value', and that is still supported. > Quoting from the swift parser's list of operators: > GE : ">="; > APPEND : "<<"; > ASSIGN : '=' ; > > I can't find the emails in the online archive, but I do have this in my > inbox: > > Subject: Re: [Swift-devel] New syntax for appending to arrays > Date: Tue, 28 Jun 2011 15:48:05 -0500 (06/28/11 13:48:05) > > > So far we have: > > 3, 0, 3, 5, 3, 3, (0, 1, 5) > > > > Anybody else? > > > > On Tue, 2011-06-28 at 14:29 -0500, Mihael Hategan wrote: > > > On Tue, 2011-06-28 at 09:08 -0500, Ian Foster wrote: > > > > $0,02 from me -- given that Swift has a C-like syntax, this syntax > will surely confuse people?? > > > > > > "+=" is a bit confusing, but then I don't think it's more so than > using > > > "+" as a string concatenation operator. > > > > > > So please vote or propose an alternative: > > > > > > 0. a += 1; > > > 1. a ++= 1; > > > 2. a ++ 1; > > > 3. a << 1; > > > 4. a[] = 1; > > > 5. a .= 1; > > Mihael > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Wed Sep 4 16:07:18 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 04 Sep 2013 14:07:18 -0700 Subject: [Swift-devel] Appendable data structures in Swift/K and Swift/T In-Reply-To: References: <1378324877.17687.18.camel@echo> Message-ID: <1378328838.17687.32.camel@echo> On Wed, 2013-09-04 at 15:23 -0500, Tim Armstrong wrote: > Awesome, thanks, that clears things up. For some reason I was thinking > that [auto] was something like an auto-incremented integer. I think we quickly dismissed that because of the problems you alluded to in the previous email. There is also the issue of re-starts, which requires that the index allocation be deterministic (actually identical) between different runs of the same script. > > Does that mean that "b[k] = x" and append(b, x) are semantically identical > in these two code fragments? > > ============== > foreach v, k in a { > b[k] = a[k] * 2; > } > --------------------------- > foreach v in a { > append(b, a[k] * 2); > } > ============== Nope. The first guarantees that both arrays have the same number of elements AND the same keys. The second one only guarantees that the arrays have the same number of elements. The distinction is obvious if you add another loop: foreach v, k in a { int s = a[k] + b[k]; } That subsequent foreach is only guaranteed to work in the first case you have above. > > Also, I was wondering how this interacts with mappers? E.g. if a have an > [auto] array mapped with simple_mapper, then what filenames would be > produced for something like this? > file outfile[] ; > > append(outfile , ...); > > append(outfile , ...); Like a [int] array. The string keys get SHA-1ed. Which might be a reasonable solution to getting rid of the string keys. Thanks! Mihael From tim.g.armstrong at gmail.com Wed Sep 4 16:17:25 2013 From: tim.g.armstrong at gmail.com (Tim Armstrong) Date: Wed, 4 Sep 2013 16:17:25 -0500 Subject: [Swift-devel] Appendable data structures in Swift/K and Swift/T In-Reply-To: <1378328838.17687.32.camel@echo> References: <1378324877.17687.18.camel@echo> <1378328838.17687.32.camel@echo> Message-ID: Ok, I see, thanks, that makes a lot of sense. So essentially each append() call creates a new opaque key value that is guaranteed to be unique and the same across runs, then the rest follows from that. - Tim On Wed, Sep 4, 2013 at 4:07 PM, Mihael Hategan wrote: > On Wed, 2013-09-04 at 15:23 -0500, Tim Armstrong wrote: > > Awesome, thanks, that clears things up. For some reason I was thinking > > that [auto] was something like an auto-incremented integer. > > I think we quickly dismissed that because of the problems you alluded to > in the previous email. > > There is also the issue of re-starts, which requires that the index > allocation be deterministic (actually identical) between different runs > of the same script. > > > > > Does that mean that "b[k] = x" and append(b, x) are semantically > identical > > in these two code fragments? > > > > ============== > > foreach v, k in a { > > b[k] = a[k] * 2; > > } > > --------------------------- > > foreach v in a { > > append(b, a[k] * 2); > > } > > ============== > > Nope. The first guarantees that both arrays have the same number of > elements AND the same keys. > > The second one only guarantees that the arrays have the same number of > elements. > > The distinction is obvious if you add another loop: > > foreach v, k in a { > int s = a[k] + b[k]; > } > > That subsequent foreach is only guaranteed to work in the first case you > have above. > > > > > Also, I was wondering how this interacts with mappers? E.g. if a have an > > [auto] array mapped with simple_mapper, then what filenames would be > > produced for something like this? > > file outfile[] ; > > > > append(outfile , ...); > > > > append(outfile , ...); > > Like a [int] array. The string keys get SHA-1ed. > > Which might be a reasonable solution to getting rid of the string keys. > Thanks! > > Mihael > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Wed Sep 4 17:08:27 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 04 Sep 2013 15:08:27 -0700 Subject: [Swift-devel] Appendable data structures in Swift/K and Swift/T In-Reply-To: References: <1378324877.17687.18.camel@echo> <1378328838.17687.32.camel@echo> Message-ID: <1378332507.20590.0.camel@echo> On Wed, 2013-09-04 at 16:17 -0500, Tim Armstrong wrote: > Ok, I see, thanks, that makes a lot of sense. > > So essentially each append() call creates a new opaque key value that is > guaranteed to be unique and the same across runs, then the rest follows > from that. Yep. Mihael From marialemos72 at gmail.com Fri Sep 6 08:03:47 2013 From: marialemos72 at gmail.com (Maria Lemos) Date: Fri, 6 Sep 2013 14:03:47 +0100 Subject: [Swift-devel] CFP: WorldCIST'14 - World Conference on IST; Best papers published in ISI Journals Message-ID: <20130906130353.D594D7CC53E@mailrelay.anl.gov> Apologies if you are receiving this mail more than once... Please disseminate by friends, colleagues, researchers, students, etc. Thanks a lot! ********************************************************************************** WorldCIST'14 The 2014 World Conference on Information Systems and Technologies April 15 - 18, Madeira Island, Portugal http://www.aisti.eu/worldcist14/ ********************************************************************************** The 2014 World Conference on Information Systems and Technologies (WorldCIST'14: http://www.aisti.eu/worldcist14) is a global forum for researchers and practitioners to present and discuss the most recent innovations, trends, results, experiences and concerns in the several perspectives of Information Systems and Technologies. We are pleased to invite you to submit your papers to WorldCISTI'14. All submissions will be reviewed on the basis of relevance, originality, importance and clarity. THEMES Submitted papers should be related with one or more of the main themes proposed for the Conference: A) Information and Knowledge Management (IKM); B) Organizational Models and Information Systems (OMIS); C) Intelligent and Decision Support Systems (IDSS); D) Software Systems, Architectures, Applications and Tools (SSAAT); E) Computer Networks, Mobility and Pervasive Systems (CNMPS); F) Human-Computer Interaction (HCI); G) Health Informatics (HIS); H) Information Technologies in Education (ITE). TYPES OF SUBMISSIONS AND DECISIONS Four types of papers can be submitted: Full paper: Finished or consolidated R&D works, to be included in one of the Conference themes. These papers are assigned a 10-page limit. Short paper: Ongoing works with relevant preliminary results, open to discussion. These papers are assigned a 7-page limit. Poster paper: Initial work with relevant ideas, open to discussion. These papers are assigned to a 4-page limit. Company paper: Companies' papers that show practical experience, R & D, tools, etc., focused on some topics of the conference. These papers are assigned to a 4-page limit. Submitted papers must comply with the format of Advances in Intelligent Systems and Computing Series (see Instructions for Authors at Springer Website or download a DOC example) be written in English, must not have been published before, not be under review for any other conference or publication and not include any information leading to the authors? identification. Therefore, the authors? names, affiliations and bibliographic references should not be included in the version for evaluation by the Program Committee. This information should only be included in the camera-ready version, saved in Word or Latex format and also in PDF format. These files must be accompanied by the Consent to Publication form filled out, in a ZIP file, and uploaded at the conference management system. All papers will be subjected to a ?double-blind review? by at least two members of the Program Committee. Based on Program Committee evaluation, a paper can be rejected or accepted by the Conference Chairs. In the later case, it can be accepted as the type originally submitted or as another type. Thus, full papers can be accepted as short papers or poster papers only. Similarly, short papers can be accepted as poster papers only. In these cases, the authors will be allowed to maintain the original number of pages in the camera-ready version. The authors of accepted poster papers must also build and print a poster to be exhibited during the Conference. This poster must follow an A1 or A2 vertical format. The Conference includes Work Sessions where these posters are presented and orally discussed, with a 5 minute limit per poster. The authors of accepted full papers will have 15 minutes to present their work in a Conference Work Session; approximately 5 minutes of discussion will follow each presentation. The authors of accepted short papers and company papers will have 11 minutes to present their work in a Conference Work Session; approximately 4 minutes of discussion will follow each presentation. PUBLICATION AND INDEXING To ensure that a full paper, short paper, poster paper or company paper is published in the Proceedings, at least one of the authors must be fully registered by the 24th of January 2014, and the paper must comply with the suggested layout and page-limit. Additionally, all recommended changes must be addressed by the authors before they submit the camera-ready version. No more than one paper per registration will be published in the Conference Proceedings. An extra fee must be paid for publication of additional papers, with a maximum of one additional paper per registration. Full and short papers will be published in Proceedings by Springer, in Advances in Intelligent Systems and Computing Series. Poster and company papers will be published in Proceedings by AISTI. Published full and short papers will be submitted for indexation by ISI, EI-Compendex, SCOPUS and DBLP, among others, and will be available in the SpringerLink Digital Library. Published poster and company papers will be submitted for indexation by EI-Compendex and EBSCO. The authors of the best selected papers will be invited to extend them for publication in international journals indexed by ISI, SCOPUS and DBLP, among others, such as: Journal of Information Technology (JIT) Social Science Computer Review (SSCR) Information Processing & Management (IPM) Information Technology & People (ITP) Computer Science and Information Systems (ComSIS) Information Development (IDV) IEEE IT Professional (ITPro) Computer Methods in Biomechanics and Biomedical Engineering - Imaging & Visualization (CMBBE-IV) Journal of Medical Internet Research (JMIR) International Journal of Health Information Systems & Informatics (IJHISI) International Journal of Web Based Communities (IJWBC) International Journal of Interactive Multimedia and Artificial Intelligence (IJIMAI) INPORTANT DATES Paper Submission: November 15, 2013 Notification of Acceptance: January 10, 2014 Camera-ready Submission: January 19, 2014 Payment of Registration, to ensure the inclusion of an accepted paper in the conference proceedings: January 24, 2014. - Regards, WorldCIST Team http://www.aisti.eu/worldcist14 From hategan at mcs.anl.gov Sun Sep 8 02:51:19 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sun, 08 Sep 2013 00:51:19 -0700 Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: <5226A196.8040605@lncc.br> References: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> <1378220885.30133.1.camel@echo> <5226A196.8040605@lncc.br> Message-ID: <1378626679.30263.0.camel@echo> Should now be fixed in r7042. Mihael On Tue, 2013-09-03 at 23:57 -0300, Luiz Gadelha wrote: > That's indeed what happens, I get the warning about the sites file and > the scripts execute correctly with provenance.log disabled. > > Thanks, > > Luiz > > On 09/03/2013 12:08 PM, Mihael Hategan wrote: > > While that is true, schema checking of the sites file is optional. > > You'll get the warning seen below and swift should continue. > > > > I believe that the other error is due to the run-time. I'll take a look > > when I can. > > > > Mihael > > > > On Tue, 2013-09-03 at 07:43 -0500, Michael Wilde wrote: > >> Luiz, in trunk, stricter XML parsing is used now. You need to start sites.xml with these two lines: > >> > >> > >> > >> > >> We need to start using the README and Download pages to make people aware of changes like this. > >> > >> - Mike > >> > >> > >> ----- Original Message ----- > >>> From: "Luiz Gadelha" > >>> To: swift-devel at ci.uchicago.edu > >>> Sent: Tuesday, September 3, 2013 6:33:09 AM > >>> Subject: [Swift-devel] Trunk failing with provenance.log=true > >>> > >>> Hi, > >>> > >>> I'm getting the error message below when setting the provenance.log > >>> parameter to true in the latest trunk version. Both scripts I tested > >>> work with provenance.log disabled. > >>> > >>> Regards, > >>> > >>> Luiz > >>> > >>> $ swift -sites.file sites-vortex.xml -tc.file tc-vortex.data > >>> blast.swift > >>> -d=genes.pep -i=seq3.seq -n=64 > >>> Swift trunk swift-r7023 cog-r3768 > >>> RunID: 20130903-0731-gg3aup4d > >>> Warning: The @ syntax for function invocation is deprecated > >>> [Error] sites-vortex.xml, line 1, col 9: cvc-elt.1: Cannot find the > >>> declaration of element 'config'. > >>> > >>> Execution failed: > >>> java.lang.UnsupportedOperationException > >>> k:concat @ swift.k, line: 29 > >>> Caused by: java.lang.UnsupportedOperationException > >>> at k.rt.Sink.getAll(Sink.java:45) > >>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > >>> at k.thr.LWThread.run(LWThread.java:228) > >>> at > >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > >>> at > >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >>> at java.lang.Thread.run(Thread.java:679) > >>> > >>> > >>> $ swift foreach.swift > >>> Swift trunk swift-r7023 cog-r3768 > >>> RunID: 20130903-0755-r4lxbb67 > >>> Warning: The @ syntax for function invocation is deprecated > >>> [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > >>> declaration > >>> of element 'config'. > >>> > >>> Execution failed: > >>> java.lang.UnsupportedOperationException > >>> k:concat @ swift.k, line: 29 > >>> Caused by: java.lang.UnsupportedOperationException > >>> at k.rt.Sink.getAll(Sink.java:45) > >>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > >>> at k.thr.LWThread.run(LWThread.java:228) > >>> at > >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > >>> at > >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >>> at java.lang.Thread.run(Thread.java:679) > >>> > >>> -- > >>> Luiz Gadelha > >>> http://www.lncc.br/~lgadelha > >>> _______________________________________________ > >>> Swift-devel mailing list > >>> Swift-devel at ci.uchicago.edu > >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >>> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > From ketancmaheshwari at gmail.com Sun Sep 8 10:49:36 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Sun, 8 Sep 2013 10:49:36 -0500 Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: <1378626679.30263.0.camel@echo> References: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> <1378220885.30133.1.camel@echo> <5226A196.8040605@lncc.br> <1378626679.30263.0.camel@echo> Message-ID: Hi, I add this line to sites.xml suggested by Mike: I still get this message at runtime: [Error] sites.blues.xml, line 2, col 45: cvc-elt.1: Cannot find the declaration of element 'config'. The run however proceeds fine. Swift is trunk: r7042 Thanks, Ketan On Sun, Sep 8, 2013 at 2:51 AM, Mihael Hategan wrote: > Should now be fixed in r7042. > > Mihael > > On Tue, 2013-09-03 at 23:57 -0300, Luiz Gadelha wrote: > > That's indeed what happens, I get the warning about the sites file and > > the scripts execute correctly with provenance.log disabled. > > > > Thanks, > > > > Luiz > > > > On 09/03/2013 12:08 PM, Mihael Hategan wrote: > > > While that is true, schema checking of the sites file is optional. > > > You'll get the warning seen below and swift should continue. > > > > > > I believe that the other error is due to the run-time. I'll take a look > > > when I can. > > > > > > Mihael > > > > > > On Tue, 2013-09-03 at 07:43 -0500, Michael Wilde wrote: > > >> Luiz, in trunk, stricter XML parsing is used now. You need to start > sites.xml with these two lines: > > >> > > >> > > >> > > >> > > >> We need to start using the README and Download pages to make people > aware of changes like this. > > >> > > >> - Mike > > >> > > >> > > >> ----- Original Message ----- > > >>> From: "Luiz Gadelha" > > >>> To: swift-devel at ci.uchicago.edu > > >>> Sent: Tuesday, September 3, 2013 6:33:09 AM > > >>> Subject: [Swift-devel] Trunk failing with provenance.log=true > > >>> > > >>> Hi, > > >>> > > >>> I'm getting the error message below when setting the provenance.log > > >>> parameter to true in the latest trunk version. Both scripts I tested > > >>> work with provenance.log disabled. > > >>> > > >>> Regards, > > >>> > > >>> Luiz > > >>> > > >>> $ swift -sites.file sites-vortex.xml -tc.file tc-vortex.data > > >>> blast.swift > > >>> -d=genes.pep -i=seq3.seq -n=64 > > >>> Swift trunk swift-r7023 cog-r3768 > > >>> RunID: 20130903-0731-gg3aup4d > > >>> Warning: The @ syntax for function invocation is deprecated > > >>> [Error] sites-vortex.xml, line 1, col 9: cvc-elt.1: Cannot find the > > >>> declaration of element 'config'. > > >>> > > >>> Execution failed: > > >>> java.lang.UnsupportedOperationException > > >>> k:concat @ swift.k, line: 29 > > >>> Caused by: java.lang.UnsupportedOperationException > > >>> at k.rt.Sink.getAll(Sink.java:45) > > >>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > > >>> at k.thr.LWThread.run(LWThread.java:228) > > >>> at > > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > > >>> at > > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > >>> at java.lang.Thread.run(Thread.java:679) > > >>> > > >>> > > >>> $ swift foreach.swift > > >>> Swift trunk swift-r7023 cog-r3768 > > >>> RunID: 20130903-0755-r4lxbb67 > > >>> Warning: The @ syntax for function invocation is deprecated > > >>> [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > >>> declaration > > >>> of element 'config'. > > >>> > > >>> Execution failed: > > >>> java.lang.UnsupportedOperationException > > >>> k:concat @ swift.k, line: 29 > > >>> Caused by: java.lang.UnsupportedOperationException > > >>> at k.rt.Sink.getAll(Sink.java:45) > > >>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > >>> at > > >>> > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > > >>> at k.thr.LWThread.run(LWThread.java:228) > > >>> at > > >>> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > > >>> at > > >>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > >>> at java.lang.Thread.run(Thread.java:679) > > >>> > > >>> -- > > >>> Luiz Gadelha > > >>> http://www.lncc.br/~lgadelha > > >>> _______________________________________________ > > >>> Swift-devel mailing list > > >>> Swift-devel at ci.uchicago.edu > > >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > >>> > > >> _______________________________________________ > > >> Swift-devel mailing list > > >> Swift-devel at ci.uchicago.edu > > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Sun Sep 8 13:04:27 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sun, 08 Sep 2013 11:04:27 -0700 Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: References: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> <1378220885.30133.1.camel@echo> <5226A196.8040605@lncc.br> <1378626679.30263.0.camel@echo> Message-ID: <1378663467.32203.1.camel@echo> On Sun, 2013-09-08 at 10:49 -0500, Ketan Maheshwari wrote: > Hi, > > I add this line to sites.xml suggested by Mike: > > > I have no idea where that came from. The correct incantation is: Mihael From ketancmaheshwari at gmail.com Sun Sep 8 17:20:03 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Sun, 8 Sep 2013 17:20:03 -0500 Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: <1378663467.32203.1.camel@echo> References: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> <1378220885.30133.1.camel@echo> <5226A196.8040605@lncc.br> <1378626679.30263.0.camel@echo> <1378663467.32203.1.camel@echo> Message-ID: Thanks, it works. On Sun, Sep 8, 2013 at 1:04 PM, Mihael Hategan wrote: > On Sun, 2013-09-08 at 10:49 -0500, Ketan Maheshwari wrote: > > Hi, > > > > I add this line to sites.xml suggested by Mike: > > > > > > > > I have no idea where that came from. The correct incantation is: > > > > > Mihael > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lgadelha at lncc.br Sun Sep 8 20:12:50 2013 From: lgadelha at lncc.br (Luiz Gadelha) Date: Sun, 08 Sep 2013 22:12:50 -0300 Subject: [Swift-devel] Trunk failing with provenance.log=true In-Reply-To: <1378626679.30263.0.camel@echo> References: <1985469863.706186.1378212184229.JavaMail.root@mcs.anl.gov> <1378220885.30133.1.camel@echo> <5226A196.8040605@lncc.br> <1378626679.30263.0.camel@echo> Message-ID: <522D2092.1020306@lncc.br> Mihael, I've just tested the same scripts and they're now working with provenance.log=true. Thanks, Luiz On 09/08/2013 04:51 AM, Mihael Hategan wrote: > Should now be fixed in r7042. > > Mihael > > On Tue, 2013-09-03 at 23:57 -0300, Luiz Gadelha wrote: >> That's indeed what happens, I get the warning about the sites file and >> the scripts execute correctly with provenance.log disabled. >> >> Thanks, >> >> Luiz >> >> On 09/03/2013 12:08 PM, Mihael Hategan wrote: >>> While that is true, schema checking of the sites file is optional. >>> You'll get the warning seen below and swift should continue. >>> >>> I believe that the other error is due to the run-time. I'll take a look >>> when I can. >>> >>> Mihael >>> >>> On Tue, 2013-09-03 at 07:43 -0500, Michael Wilde wrote: >>>> Luiz, in trunk, stricter XML parsing is used now. You need to start sites.xml with these two lines: >>>> >>>> >>>> >>>> >>>> We need to start using the README and Download pages to make people aware of changes like this. >>>> >>>> - Mike >>>> >>>> >>>> ----- Original Message ----- >>>>> From: "Luiz Gadelha" >>>>> To: swift-devel at ci.uchicago.edu >>>>> Sent: Tuesday, September 3, 2013 6:33:09 AM >>>>> Subject: [Swift-devel] Trunk failing with provenance.log=true >>>>> >>>>> Hi, >>>>> >>>>> I'm getting the error message below when setting the provenance.log >>>>> parameter to true in the latest trunk version. Both scripts I tested >>>>> work with provenance.log disabled. >>>>> >>>>> Regards, >>>>> >>>>> Luiz >>>>> >>>>> $ swift -sites.file sites-vortex.xml -tc.file tc-vortex.data >>>>> blast.swift >>>>> -d=genes.pep -i=seq3.seq -n=64 >>>>> Swift trunk swift-r7023 cog-r3768 >>>>> RunID: 20130903-0731-gg3aup4d >>>>> Warning: The @ syntax for function invocation is deprecated >>>>> [Error] sites-vortex.xml, line 1, col 9: cvc-elt.1: Cannot find the >>>>> declaration of element 'config'. >>>>> >>>>> Execution failed: >>>>> java.lang.UnsupportedOperationException >>>>> k:concat @ swift.k, line: 29 >>>>> Caused by: java.lang.UnsupportedOperationException >>>>> at k.rt.Sink.getAll(Sink.java:45) >>>>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>>>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>>>> at k.thr.LWThread.run(LWThread.java:228) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:679) >>>>> >>>>> >>>>> $ swift foreach.swift >>>>> Swift trunk swift-r7023 cog-r3768 >>>>> RunID: 20130903-0755-r4lxbb67 >>>>> Warning: The @ syntax for function invocation is deprecated >>>>> [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the >>>>> declaration >>>>> of element 'config'. >>>>> >>>>> Execution failed: >>>>> java.lang.UnsupportedOperationException >>>>> k:concat @ swift.k, line: 29 >>>>> Caused by: java.lang.UnsupportedOperationException >>>>> at k.rt.Sink.getAll(Sink.java:45) >>>>> at org.globus.cog.karajan.util.TypeUtil.toString(TypeUtil.java:201) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.functions.Misc$Concat.function(Misc.java:448) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:99) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:245) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:69) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>>>> at >>>>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>>>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>>>> at k.thr.LWThread.run(LWThread.java:228) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >>>>> at java.lang.Thread.run(Thread.java:679) >>>>> >>>>> -- >>>>> Luiz Gadelha >>>>> http://www.lncc.br/~lgadelha >>>>> _______________________________________________ >>>>> Swift-devel mailing list >>>>> Swift-devel at ci.uchicago.edu >>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>>>> >>>> _______________________________________________ >>>> Swift-devel mailing list >>>> Swift-devel at ci.uchicago.edu >>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>> >>> >> >> > > -- Luiz Gadelha http://www.lncc.br/~lgadelha From wilde at mcs.anl.gov Tue Sep 10 10:31:40 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Tue, 10 Sep 2013 10:31:40 -0500 (CDT) Subject: [Swift-devel] Why does SGE provider only start one job per second? In-Reply-To: <1712757939.2154476.1378826073039.JavaMail.root@mcs.anl.gov> Message-ID: <2116419181.2155857.1378827100448.JavaMail.root@mcs.anl.gov> Running on the SGE systems orthros, when I configure a sites file for 256 single-core workers, I see the workers starting one per second. It seems that the SGE jobs themselves are only emitted one per second. A log of a typical run is in: http://www.ci.uchicago.edu/~wilde/IndexStrain-20130909-2141-zq403y5f.log It starts off like this: 2013-09-09 21:41:40,195-0500 INFO RuntimeStats$ProgressTicker 2013-09-09 21:41:43,078-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:99 2013-09-09 21:41:44,428-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:98 Active:1 2013-09-09 21:41:45,476-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:97 Active:2 2013-09-09 21:41:46,500-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:96 Active:3 2013-09-09 21:41:47,814-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:95 Active:4 2013-09-09 21:41:49,643-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:93 Active:6 2013-09-09 21:41:50,686-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:92 Active:7 2013-09-09 21:41:51,704-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:91 Active:8 2013-09-09 21:41:52,772-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:90 Active:9 2013-09-09 21:41:53,807-0500 INFO RuntimeStats$ProgressTicker Stage in:1 Submitted:89 Active:10 In this case the Swift script consists of 100 independent apps emitted right away from a foreach loop. As you can see above and in the log, the 100 apps become active at a 1-per-second rate. Can anyone tell me whats throttling the app launches in this fashion? I dont recall this happening elsewhere, so perhaps its an artifact of the SGE provider. The sites.xml file is below. Thanks, - Mike {env.HOME}/swiftwork sec1all.q --> sec1_all --> 1 320 1 1 1 4.00 00:10:00 200000 /tmp/wilde/swiftwork 100 100 10000 From davidk at ci.uchicago.edu Tue Sep 10 10:53:15 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Tue, 10 Sep 2013 10:53:15 -0500 (CDT) Subject: [Swift-devel] Why does SGE provider only start one job per second? In-Reply-To: <2116419181.2155857.1378827100448.JavaMail.root@mcs.anl.gov> Message-ID: <529735131.8593255.1378828395673.JavaMail.root@ci.uchicago.edu> It does seem to be something on the Swift side that is throttling somehow, only one submit file is being written per second. 013-09-09 21:41:42,670-0500 DEBUG AbstractExecutor Wrote SGE script to /clhome/WILDE/.globus/scripts/SGE7757422543183944116.submit 2013-09-09 21:41:44,002-0500 DEBUG AbstractExecutor Wrote SGE script to /clhome/WILDE/.globus/scripts/SGE7192137066159443413.submit 2013-09-09 21:41:45,040-0500 DEBUG AbstractExecutor Wrote SGE script to /clhome/WILDE/.globus/scripts/SGE906390723669036085.submit 2013-09-09 21:41:46,075-0500 DEBUG AbstractExecutor Wrote SGE script to /clhome/WILDE/.globus/scripts/SGE4921900111206072571.submit It could be something with the SGE provider. I'll take a look and see if I can find anything. I also remember seeing the setting below in the user guide. I've never tried changing this value before, but it might be worth a quick try: throttle.host.submit Valid values: , off Default value: 2 Limits the number of concurrent submissions for any of the sites Swift will try to send jobs to. In other words it guarantees that no more than the value of this throttle jobs sent to any site will be concurrently in a state of being submitted. ----- Original Message ----- > From: "Michael Wilde" > To: "Swift Devel" > Sent: Tuesday, September 10, 2013 10:31:40 AM > Subject: [Swift-devel] Why does SGE provider only start one job per > second? > Running on the SGE systems orthros, when I configure a sites file for > 256 single-core workers, I see the workers starting one per second. > It seems that the SGE jobs themselves are only emitted one per > second. > A log of a typical run is in: > http://www.ci.uchicago.edu/~wilde/IndexStrain-20130909-2141-zq403y5f.log > It starts off like this: > 2013-09-09 21:41:40,195-0500 INFO RuntimeStats$ProgressTicker > 2013-09-09 21:41:43,078-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:99 > 2013-09-09 21:41:44,428-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:98 Active:1 > 2013-09-09 21:41:45,476-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:97 Active:2 > 2013-09-09 21:41:46,500-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:96 Active:3 > 2013-09-09 21:41:47,814-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:95 Active:4 > 2013-09-09 21:41:49,643-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:93 Active:6 > 2013-09-09 21:41:50,686-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:92 Active:7 > 2013-09-09 21:41:51,704-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:91 Active:8 > 2013-09-09 21:41:52,772-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:90 Active:9 > 2013-09-09 21:41:53,807-0500 INFO RuntimeStats$ProgressTicker Stage > in:1 Submitted:89 Active:10 > In this case the Swift script consists of 100 independent apps > emitted right away from a foreach loop. > As you can see above and in the log, the 100 apps become active at a > 1-per-second rate. > Can anyone tell me whats throttling the app launches in this fashion? > I dont recall this happening elsewhere, so perhaps its an artifact of > the SGE provider. > The sites.xml file is below. > Thanks, > - Mike > > > > > {env.HOME}/swiftwork > > > > > sec1all.q --> > sec1_all --> > 1 > > > 320 > 1 > 1 > 1 > 4.00 > > 00:10:00 > 200000 > > > /tmp/wilde/swiftwork > > > 100 > 100 > 10000 > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidk at ci.uchicago.edu Tue Sep 10 12:05:05 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Tue, 10 Sep 2013 12:05:05 -0500 (CDT) Subject: [Swift-devel] Why does SGE provider only start one job per second? In-Reply-To: <529735131.8593255.1378828395673.JavaMail.root@ci.uchicago.edu> Message-ID: <1123118655.8661336.1378832705416.JavaMail.root@ci.uchicago.edu> I tried running similar tests with the slurm and pbs providers. I see a very similar rate of job submission (0.94.1 RC3) Submit script times on midway (slurm): Modify: 2013-09-10 11:43:31.266499000 -0500 Modify: 2013-09-10 11:43:32.287140000 -0500 Modify: 2013-09-10 11:43:33.308472000 -0500 Modify: 2013-09-10 11:43:34.328796000 -0500 Modify: 2013-09-10 11:43:35.329563000 -0500 Submit script times on raven (pbs): Modify: 2013-09-10 11:54:17.000000000 -0500 Modify: 2013-09-10 11:54:18.000000000 -0500 Modify: 2013-09-10 11:54:19.000000000 -0500 Modify: 2013-09-10 11:54:20.000000000 -0500 Modify: 2013-09-10 11:54:21.000000000 -0500 Modify: 2013-09-10 11:54:22.000000000 -0500 ----- Original Message ----- > From: "David Kelly" > To: "Michael Wilde" > Cc: "Swift Devel" > Sent: Tuesday, September 10, 2013 10:53:15 AM > Subject: Re: [Swift-devel] Why does SGE provider only start one job > per second? > It does seem to be something on the Swift side that is throttling > somehow, only one submit file is being written per second. > 013-09-09 21:41:42,670-0500 DEBUG AbstractExecutor Wrote SGE script > to /clhome/WILDE/.globus/scripts/SGE7757422543183944116.submit > 2013-09-09 21:41:44,002-0500 DEBUG AbstractExecutor Wrote SGE script > to /clhome/WILDE/.globus/scripts/SGE7192137066159443413.submit > 2013-09-09 21:41:45,040-0500 DEBUG AbstractExecutor Wrote SGE script > to /clhome/WILDE/.globus/scripts/SGE906390723669036085.submit > 2013-09-09 21:41:46,075-0500 DEBUG AbstractExecutor Wrote SGE script > to /clhome/WILDE/.globus/scripts/SGE4921900111206072571.submit > It could be something with the SGE provider. I'll take a look and see > if I can find anything. > I also remember seeing the setting below in the user guide. I've > never tried changing this value before, but it might be worth a > quick try: > throttle.host.submit > Valid values: , off > Default value: 2 > Limits the number of concurrent submissions for any of the sites > Swift will try to send jobs to. In other words it guarantees that no > more than the value of this throttle jobs sent to any site will be > concurrently in a state of being submitted. > ----- Original Message ----- > > From: "Michael Wilde" > > > To: "Swift Devel" > > > Sent: Tuesday, September 10, 2013 10:31:40 AM > > > Subject: [Swift-devel] Why does SGE provider only start one job per > > second? > > > Running on the SGE systems orthros, when I configure a sites file > > for > > 256 single-core workers, I see the workers starting one per second. > > It seems that the SGE jobs themselves are only emitted one per > > second. > > > A log of a typical run is in: > > > http://www.ci.uchicago.edu/~wilde/IndexStrain-20130909-2141-zq403y5f.log > > > It starts off like this: > > > 2013-09-09 21:41:40,195-0500 INFO RuntimeStats$ProgressTicker > > > 2013-09-09 21:41:43,078-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:99 > > > 2013-09-09 21:41:44,428-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:98 Active:1 > > > 2013-09-09 21:41:45,476-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:97 Active:2 > > > 2013-09-09 21:41:46,500-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:96 Active:3 > > > 2013-09-09 21:41:47,814-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:95 Active:4 > > > 2013-09-09 21:41:49,643-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:93 Active:6 > > > 2013-09-09 21:41:50,686-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:92 Active:7 > > > 2013-09-09 21:41:51,704-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:91 Active:8 > > > 2013-09-09 21:41:52,772-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:90 Active:9 > > > 2013-09-09 21:41:53,807-0500 INFO RuntimeStats$ProgressTicker Stage > > in:1 Submitted:89 Active:10 > > > In this case the Swift script consists of 100 independent apps > > emitted right away from a foreach loop. > > > As you can see above and in the log, the 100 apps become active at > > a > > 1-per-second rate. > > > Can anyone tell me whats throttling the app launches in this > > fashion? > > > I dont recall this happening elsewhere, so perhaps its an artifact > > of > > the SGE provider. > > > The sites.xml file is below. > > > Thanks, > > > - Mike > > > > > > > > > > > > > > > {env.HOME}/swiftwork > > > > > > > > > > > > > > > sec1all.q --> > > > sec1_all --> > > > 1 > > > > > > > > > 320 > > > 1 > > > 1 > > > 1 > > > 4.00 > > > > > > 00:10:00 > > > 200000 > > > > > > > > > /tmp/wilde/swiftwork > > > > > > > > > 100 > > > 100 > > > 10000 > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Sep 10 14:20:50 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 10 Sep 2013 12:20:50 -0700 Subject: [Swift-devel] Why does SGE provider only start one job per second? In-Reply-To: <529735131.8593255.1378828395673.JavaMail.root@ci.uchicago.edu> References: <529735131.8593255.1378828395673.JavaMail.root@ci.uchicago.edu> Message-ID: <1378840850.25100.1.camel@echo> On Tue, 2013-09-10 at 10:53 -0500, David Kelly wrote: > 2013-09-09 21:41:44,428-0500 INFO RuntimeStats$ProgressTicker Stage in:1 > Submitted:98 Active:1 > I also remember seeing the setting below in the user guide. I've never tried changing this value before, but it might be worth a quick try: > > throttle.host.submit > That would make a difference in how quick the "Submitted" state changes, but once things get submitted, that setting is not relevant any more. Mihael From iraicu at cs.iit.edu Wed Sep 11 23:53:56 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Wed, 11 Sep 2013 23:53:56 -0500 Subject: [Swift-devel] CFP: The 23nd International ACM Symposium on, High-Performance Parallel and Distributed Computing (HPDC) 2014 -- in Vancouver Canada. Message-ID: <523148E4.5020304@cs.iit.edu> **** CALL FOR PAPERS **** The 23nd International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC-2014) Vancouver, Canada - June 23-27, 2014 http://www.hpdc.org/2014 The ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) is the premier annual conference for presenting the latest research on the design, implementation, evaluation, and application of parallel and distributed systems for high-end computing. In 2014, the 23nd HPDC and affiliated workshops will take place in the beautiful city of Vancouver, Canada during June 23-27, 2014. **** IMPORTANT DATES **** Abstracts Due: January 20, 2014 Papers Due: January 27, 2014 (no extensions) Author Rebuttal: March 14-18, 2014 Author Notifications: March 28, 2014 **** SCOPE AND TOPICS **** Submissions are welcomed on high-performance parallel and distributed computing topics including but not limited to: clouds, clusters, grids, Big Data, massively multicore, and global-scale computing systems. New scholarly research showing empirical and reproducible results in architectures, systems, and networks is strongly encouraged, as are experience reports of operational deployments that can provide insights for future research on HPDC applications and systems. All papers will be evaluated for their originality, technical depth and correctness, potential impact, relevance to the conference, and quality of presentation. Research papers must clearly demonstrate research contributions and novelty, while experience reports must clearly describe lessons learned and demonstrate impact. In the context of high-performance parallel and distributed computing, the topics of interest include, but are not limited to: o Systems, networks, and architectures for high-end computing o Massively multicore systems o Resource virtualization o Programming languages and environments o I/O, storage systems, and data management o Resource management and scheduling, including energy-aware techniques o Performance modeling and analysis o Fault tolerance, reliability, and availability o Data-intensive computing o Applications of parallel and distributed computing **** PAPER SUBMISSION GUIDELINES **** Authors are invited to submit technical papers of at most 12 pages in PDF format, including figures and references. Papers should be formatted in the ACM Proceedings Style (http://www.acm.org/sigs/publications/proceedings-templates) and submitted via the conference web site. No changes to the margins, spacing, or font sizes as specified by the style file are allowed. Accepted papers will appear in the conference proceedings, and will be incorporated into the ACM Digital Library. A limited number of papers will be accepted as posters. Papers must be self-contained and provide the technical substance required for the program committee to evaluate their contributions. Papers should thoughtfully address all related work, particularly work presented at previous HPDC events. Submitted papers must be original work that has not appeared in and is not under consideration for another conference or a journal. See the ACM Prior Publication Policy (http://www.acm.org/publications/policies/sim_submissions) for more details. **** HPDC'14 GENERAL CO-CHAIRS **** Beth Plale, Indiana University Matei Ripeanu, University of British Columbia **** HPDC'14 PROGRAM CO-CHAIRS **** Franck Cappello, Argonne National Lab and INRIA Dongyan Xu, Purdue University **** HPDC'14 PUBLICITY CHAIR **** Ioan Raicu, Illinois Institute of Technology, USA **** HPDC'14 PROGRAM COMMITTEE **** To be announced on the conference web site. **** HPDC STEERING COMMITTEE **** Henri Bal, Vrije Universiteit Andrew A. Chien, University of Chicago Peter Dinda, Northwestern University Dick Epema, Delft University of Technology Renato Figueiredo, University of Florida Ian Foster, Univ. of Chicago and Argonne National Laboratory Salim Hariri, University of Arizona Thilo Kielmann, Vrije Universiteit Amsterdam Arthur "Barney" Maccabe, Oak Ridge National Laboratory Manish Parashar, Rutgers University Matei Ripeanu, University of British Columbia Karsten Schwan, Georgia Tech Doug Thain, University of Notre Dame Jon Weissman, University of Minnesota (Chair) -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud, IEEE/ACM DataCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= From ketancmaheshwari at gmail.com Mon Sep 16 10:49:07 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Mon, 16 Sep 2013 10:49:07 -0500 Subject: [Swift-devel] iterate fails Message-ID: Hi, I am trying a catsn example with foreach replaced by iterate as follows: === type file; app (file o) cat (file i) { cat @i stdout=@o; } file out[]; string d=@arg("data"); #foreach j in [1:1000] { iterate j { file data; out[j] = cat(data); } until (j==3); === The run fails with error message as follows: $ swift -tc.file tc -config cf -sites.file sites.xml catsn.swift -data=data.txt Swift trunk swift-r7065 (swift modified locally) cog-r3782 RunID: 20130916-1045-tldyyexe Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 Execution failed: org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not found for out of size 2 swift:getFieldSubscript @ catsn, line: 93 Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not found for out of size 2 Any suggestions? Thanks, -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Mon Sep 16 11:20:28 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Mon, 16 Sep 2013 11:20:28 -0500 (CDT) Subject: [Swift-devel] iterate fails In-Reply-To: Message-ID: <1301634470.369394.1379348428886.JavaMail.root@mcs.anl.gov> Ketan, this worked for me (see below). What version of swift did you run where? I ran on swift.rcc, on localhost. - Mike swift$ swift km2.swift -data=data.txt Swift 0.94.1 RC2 swift-r6895 cog-r3765 RunID: 20130916-1617-i570fay7 Progress: time: Mon, 16 Sep 2013 16:17:32 +0000 Final status: Mon, 16 Sep 2013 16:17:33 +0000 Finished successfully:3 swift$ ls -lt outdir | head total 16032 -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0002.out -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0001.out -rw-rw-r-- 1 wilde wilde 9 Sep 16 11:17 f.0000.out -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0483.out -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0490.out -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0488.out -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0500.out -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0498.out -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0491.out swift$ swift -version Swift 0.94.1 RC2 swift-r6895 cog-r3765 swift$ ----- Original Message ----- > From: "Ketan Maheshwari" > To: "Swift Devel" > Sent: Monday, September 16, 2013 10:49:07 AM > Subject: [Swift-devel] iterate fails > > > > > Hi, > > I am trying a catsn example with foreach replaced by iterate as > follows: > > === > type file; > > app (file o) cat (file i) { > cat @i stdout=@o; > } > file out[] prefix="f.",suffix=".out">; > > string d=@arg("data"); > > #foreach j in [1:1000] { > iterate j { > file data; > out[j] = cat(data); > } until (j==3); > > > === > > > The run fails with error message as follows: > $ swift -tc.file tc -config cf -sites.file sites.xml catsn.swift > -data=data.txt > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > RunID: 20130916-1045-tldyyexe > Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 > > Execution failed: > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not > found for out of size 2 > swift:getFieldSubscript @ catsn, line: 93 > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array index > '[2]' not found for out of size 2 > > > > Any suggestions? > > > Thanks, > > -- > Ketan > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From ketancmaheshwari at gmail.com Mon Sep 16 11:23:54 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Mon, 16 Sep 2013 11:23:54 -0500 Subject: [Swift-devel] iterate fails In-Reply-To: <1301634470.369394.1379348428886.JavaMail.root@mcs.anl.gov> References: <1301634470.369394.1379348428886.JavaMail.root@mcs.anl.gov> Message-ID: Mike, I am using the latest trunk: Swift trunk swift-r7065 (swift modified locally) cog-r3782 On Mon, Sep 16, 2013 at 11:20 AM, Michael Wilde wrote: > Ketan, this worked for me (see below). What version of swift did you run > where? > > I ran on swift.rcc, on localhost. > > - Mike > > swift$ swift km2.swift -data=data.txt > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > RunID: 20130916-1617-i570fay7 > Progress: time: Mon, 16 Sep 2013 16:17:32 +0000 > Final status: Mon, 16 Sep 2013 16:17:33 +0000 Finished successfully:3 > > swift$ ls -lt outdir | head > total 16032 > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0002.out > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0001.out > -rw-rw-r-- 1 wilde wilde 9 Sep 16 11:17 f.0000.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0483.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0490.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0488.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0500.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0498.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0491.out > swift$ swift -version > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > swift$ > > > ----- Original Message ----- > > From: "Ketan Maheshwari" > > To: "Swift Devel" > > Sent: Monday, September 16, 2013 10:49:07 AM > > Subject: [Swift-devel] iterate fails > > > > > > > > > > Hi, > > > > I am trying a catsn example with foreach replaced by iterate as > > follows: > > > > === > > type file; > > > > app (file o) cat (file i) { > > cat @i stdout=@o; > > } > > file out[] > prefix="f.",suffix=".out">; > > > > string d=@arg("data"); > > > > #foreach j in [1:1000] { > > iterate j { > > file data; > > out[j] = cat(data); > > } until (j==3); > > > > > > === > > > > > > The run fails with error message as follows: > > $ swift -tc.file tc -config cf -sites.file sites.xml catsn.swift > > -data=data.txt > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > RunID: 20130916-1045-tldyyexe > > Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 > > > > Execution failed: > > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not > > found for out of size 2 > > swift:getFieldSubscript @ catsn, line: 93 > > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array index > > '[2]' not found for out of size 2 > > > > > > > > Any suggestions? > > > > > > Thanks, > > > > -- > > Ketan > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Mon Sep 16 11:29:01 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Mon, 16 Sep 2013 11:29:01 -0500 (CDT) Subject: [Swift-devel] iterate fails In-Reply-To: Message-ID: <652671398.372560.1379348941140.JavaMail.root@mcs.anl.gov> If this is indeed failing on trunk, hopefully the test suite will see the same thing. Yadu, can you look into this and file a ticket if its indeed a trunk bug? Thanks, - Mike ----- Original Message ----- > From: "Ketan Maheshwari" > To: "Michael Wilde" > Cc: "Swift Devel" > Sent: Monday, September 16, 2013 11:23:54 AM > Subject: Re: [Swift-devel] iterate fails > > > > Mike, > > I am using the latest trunk: > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > > > On Mon, Sep 16, 2013 at 11:20 AM, Michael Wilde < wilde at mcs.anl.gov > > wrote: > > > Ketan, this worked for me (see below). What version of swift did you > run where? > > I ran on swift.rcc, on localhost. > > - Mike > > swift$ swift km2.swift -data=data.txt > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > RunID: 20130916-1617-i570fay7 > Progress: time: Mon, 16 Sep 2013 16:17:32 +0000 > Final status: Mon, 16 Sep 2013 16:17:33 +0000 Finished successfully:3 > > swift$ ls -lt outdir | head > total 16032 > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0002.out > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0001.out > -rw-rw-r-- 1 wilde wilde 9 Sep 16 11:17 f.0000.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0483.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0490.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0488.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0500.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0498.out > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0491.out > swift$ swift -version > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > swift$ > > > > > ----- Original Message ----- > > From: "Ketan Maheshwari" < ketancmaheshwari at gmail.com > > > To: "Swift Devel" < swift-devel at ci.uchicago.edu > > > Sent: Monday, September 16, 2013 10:49:07 AM > > Subject: [Swift-devel] iterate fails > > > > > > > > > > Hi, > > > > I am trying a catsn example with foreach replaced by iterate as > > follows: > > > > === > > type file; > > > > app (file o) cat (file i) { > > cat @i stdout=@o; > > } > > file out[] > prefix="f.",suffix=".out">; > > > > string d=@arg("data"); > > > > #foreach j in [1:1000] { > > iterate j { > > file data; > > out[j] = cat(data); > > } until (j==3); > > > > > > === > > > > > > The run fails with error message as follows: > > $ swift -tc.file tc -config cf -sites.file sites.xml catsn.swift > > -data=data.txt > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > RunID: 20130916-1045-tldyyexe > > Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 > > > > Execution failed: > > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not > > found for out of size 2 > > swift:getFieldSubscript @ catsn, line: 93 > > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array > > index > > '[2]' not found for out of size 2 > > > > > > > > Any suggestions? > > > > > > Thanks, > > > > -- > > Ketan > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > -- > Ketan > > From hategan at mcs.anl.gov Mon Sep 16 13:52:32 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Mon, 16 Sep 2013 11:52:32 -0700 Subject: [Swift-devel] iterate fails In-Reply-To: <652671398.372560.1379348941140.JavaMail.root@mcs.anl.gov> References: <652671398.372560.1379348941140.JavaMail.root@mcs.anl.gov> Message-ID: <1379357552.27463.0.camel@echo> I can confirm that this is an issue with write reference counting in trunk. I'm working on it. On Mon, 2013-09-16 at 11:29 -0500, Michael Wilde wrote: > If this is indeed failing on trunk, hopefully the test suite will see the same thing. > Yadu, can you look into this and file a ticket if its indeed a trunk bug? > > Thanks, > > - Mike > > ----- Original Message ----- > > From: "Ketan Maheshwari" > > To: "Michael Wilde" > > Cc: "Swift Devel" > > Sent: Monday, September 16, 2013 11:23:54 AM > > Subject: Re: [Swift-devel] iterate fails > > > > > > > > Mike, > > > > I am using the latest trunk: > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > > > > > > > > On Mon, Sep 16, 2013 at 11:20 AM, Michael Wilde < wilde at mcs.anl.gov > > > wrote: > > > > > > Ketan, this worked for me (see below). What version of swift did you > > run where? > > > > I ran on swift.rcc, on localhost. > > > > - Mike > > > > swift$ swift km2.swift -data=data.txt > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > RunID: 20130916-1617-i570fay7 > > Progress: time: Mon, 16 Sep 2013 16:17:32 +0000 > > Final status: Mon, 16 Sep 2013 16:17:33 +0000 Finished successfully:3 > > > > swift$ ls -lt outdir | head > > total 16032 > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0002.out > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0001.out > > -rw-rw-r-- 1 wilde wilde 9 Sep 16 11:17 f.0000.out > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0483.out > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0490.out > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0488.out > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0500.out > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0498.out > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0491.out > > swift$ swift -version > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > swift$ > > > > > > > > > > ----- Original Message ----- > > > From: "Ketan Maheshwari" < ketancmaheshwari at gmail.com > > > > To: "Swift Devel" < swift-devel at ci.uchicago.edu > > > > Sent: Monday, September 16, 2013 10:49:07 AM > > > Subject: [Swift-devel] iterate fails > > > > > > > > > > > > > > > Hi, > > > > > > I am trying a catsn example with foreach replaced by iterate as > > > follows: > > > > > > === > > > type file; > > > > > > app (file o) cat (file i) { > > > cat @i stdout=@o; > > > } > > > file out[] > > prefix="f.",suffix=".out">; > > > > > > string d=@arg("data"); > > > > > > #foreach j in [1:1000] { > > > iterate j { > > > file data; > > > out[j] = cat(data); > > > } until (j==3); > > > > > > > > > === > > > > > > > > > The run fails with error message as follows: > > > $ swift -tc.file tc -config cf -sites.file sites.xml catsn.swift > > > -data=data.txt > > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > RunID: 20130916-1045-tldyyexe > > > Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 > > > > > > Execution failed: > > > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not > > > found for out of size 2 > > > swift:getFieldSubscript @ catsn, line: 93 > > > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array > > > index > > > '[2]' not found for out of size 2 > > > > > > > > > > > > Any suggestions? > > > > > > > > > Thanks, > > > > > > -- > > > Ketan > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > -- > > Ketan > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From yadudoc1729 at gmail.com Mon Sep 16 14:39:34 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Mon, 16 Sep 2013 14:39:34 -0500 Subject: [Swift-devel] iterate fails In-Reply-To: <1379357552.27463.0.camel@echo> References: <652671398.372560.1379348941140.JavaMail.root@mcs.anl.gov> <1379357552.27463.0.camel@echo> Message-ID: Hi Mihael, Is this the same issue that was discussed in this mail -> http://lists.ci.uchicago.edu/pipermail/swift-devel/2013-March/010109.html I am seeing this script failing on trunk (but not on 0.94). The internals stress test runs this script which does an assignment to an array within an iterate loop -> https://github.com/yadudoc/swift-basics/blob/master/stress/x_iterate.swift The test-suite has some pending issues on trunk, which I haven't cleaned up yet, and which is why this went under the radar. Here's the error I get when running stress/x_iterate.swift: [yadunand at midway001 x_iterate-142712]$ swift x_iterate.swift -loops=5 Swift trunk swift-r7071 (swift modified locally) cog-r3787 (cog modified locally) RunID: 20130916-1936-i6mqe9qa [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the declaration of element 'config'. Progress: time: Mon, 16 Sep 2013 19:36:50 +0000 Result[0] = 0 Execution failed: java.lang.IndexOutOfBoundsException: Invalid index [4] for result tracef @ x_iterate, line: 56 Caused by: java.lang.IndexOutOfBoundsException: Invalid index [4] for result at org.griphyn.vdl.mapping.AbstractDataNode.checkNoValue(AbstractDataNode.java:716) at org.griphyn.vdl.mapping.AbstractDataNode.waitFor(AbstractDataNode.java:682) at org.griphyn.vdl.karajan.lib.SwiftFunction.waitForAll(SwiftFunction.java:343) at org.griphyn.vdl.karajan.lib.swiftscript.Tracef.function(Tracef.java:57) at org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) at k.thr.LWThread.run(LWThread.java:228) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) Here's the error I'm getting when running Ketan's code : Execution failed: org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not found for out of size 2 swift:getFieldSubscript @ iterate, line: 93 Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not found for out of size 2 Are these coming from the same bug ? Thanks, -Yadu On Mon, Sep 16, 2013 at 1:52 PM, Mihael Hategan wrote: > I can confirm that this is an issue with write reference counting in > trunk. I'm working on it. > > On Mon, 2013-09-16 at 11:29 -0500, Michael Wilde wrote: > > If this is indeed failing on trunk, hopefully the test suite will see > the same thing. > > Yadu, can you look into this and file a ticket if its indeed a trunk bug? > > > > Thanks, > > > > - Mike > > > > ----- Original Message ----- > > > From: "Ketan Maheshwari" > > > To: "Michael Wilde" > > > Cc: "Swift Devel" > > > Sent: Monday, September 16, 2013 11:23:54 AM > > > Subject: Re: [Swift-devel] iterate fails > > > > > > > > > > > > Mike, > > > > > > I am using the latest trunk: > > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > > > > > > > > > > > > > On Mon, Sep 16, 2013 at 11:20 AM, Michael Wilde < wilde at mcs.anl.gov > > > > wrote: > > > > > > > > > Ketan, this worked for me (see below). What version of swift did you > > > run where? > > > > > > I ran on swift.rcc, on localhost. > > > > > > - Mike > > > > > > swift$ swift km2.swift -data=data.txt > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > > > RunID: 20130916-1617-i570fay7 > > > Progress: time: Mon, 16 Sep 2013 16:17:32 +0000 > > > Final status: Mon, 16 Sep 2013 16:17:33 +0000 Finished successfully:3 > > > > > > swift$ ls -lt outdir | head > > > total 16032 > > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0002.out > > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0001.out > > > -rw-rw-r-- 1 wilde wilde 9 Sep 16 11:17 f.0000.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0483.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0490.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0488.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0500.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0498.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0491.out > > > swift$ swift -version > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > > > swift$ > > > > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Ketan Maheshwari" < ketancmaheshwari at gmail.com > > > > > To: "Swift Devel" < swift-devel at ci.uchicago.edu > > > > > Sent: Monday, September 16, 2013 10:49:07 AM > > > > Subject: [Swift-devel] iterate fails > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > I am trying a catsn example with foreach replaced by iterate as > > > > follows: > > > > > > > > === > > > > type file; > > > > > > > > app (file o) cat (file i) { > > > > cat @i stdout=@o; > > > > } > > > > file out[] > > > prefix="f.",suffix=".out">; > > > > > > > > string d=@arg("data"); > > > > > > > > #foreach j in [1:1000] { > > > > iterate j { > > > > file data; > > > > out[j] = cat(data); > > > > } until (j==3); > > > > > > > > > > > > === > > > > > > > > > > > > The run fails with error message as follows: > > > > $ swift -tc.file tc -config cf -sites.file sites.xml catsn.swift > > > > -data=data.txt > > > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > > RunID: 20130916-1045-tldyyexe > > > > Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 > > > > > > > > Execution failed: > > > > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not > > > > found for out of size 2 > > > > swift:getFieldSubscript @ catsn, line: 93 > > > > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array > > > > index > > > > '[2]' not found for out of size 2 > > > > > > > > > > > > > > > > Any suggestions? > > > > > > > > > > > > Thanks, > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > -- > > > Ketan > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Mon Sep 16 15:04:21 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Mon, 16 Sep 2013 13:04:21 -0700 Subject: [Swift-devel] iterate fails In-Reply-To: References: <652671398.372560.1379348941140.JavaMail.root@mcs.anl.gov> <1379357552.27463.0.camel@echo> Message-ID: <1379361861.30928.0.camel@echo> Not the same. The link points to a compilation issue, while this is a run-time issue. Mihael On Mon, 2013-09-16 at 14:39 -0500, Yadu Nand wrote: > Hi Mihael, > > Is this the same issue that was discussed in this mail -> > http://lists.ci.uchicago.edu/pipermail/swift-devel/2013-March/010109.html > > I am seeing this script failing on trunk (but not on 0.94). The internals > stress test > runs this script which does an assignment to an array within an iterate > loop -> > https://github.com/yadudoc/swift-basics/blob/master/stress/x_iterate.swift > The test-suite has some pending issues on trunk, which I haven't cleaned up > yet, > and which is why this went under the radar. > > Here's the error I get when running stress/x_iterate.swift: > [yadunand at midway001 x_iterate-142712]$ swift x_iterate.swift -loops=5 > Swift trunk swift-r7071 (swift modified locally) cog-r3787 (cog modified > locally) > RunID: 20130916-1936-i6mqe9qa > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the declaration of > element 'config'. > Progress: time: Mon, 16 Sep 2013 19:36:50 +0000 > Result[0] = 0 > > Execution failed: > java.lang.IndexOutOfBoundsException: Invalid index [4] for result > tracef @ x_iterate, line: 56 > Caused by: java.lang.IndexOutOfBoundsException: Invalid index [4] for result > at > org.griphyn.vdl.mapping.AbstractDataNode.checkNoValue(AbstractDataNode.java:716) > at > org.griphyn.vdl.mapping.AbstractDataNode.waitFor(AbstractDataNode.java:682) > at > org.griphyn.vdl.karajan.lib.SwiftFunction.waitForAll(SwiftFunction.java:343) > at org.griphyn.vdl.karajan.lib.swiftscript.Tracef.function(Tracef.java:57) > at > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) > at k.thr.LWThread.run(LWThread.java:228) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) > > Here's the error I'm getting when running Ketan's code : > Execution failed: > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not found > for out of size 2 > swift:getFieldSubscript @ iterate, line: 93 > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' > not found for out of size 2 > > Are these coming from the same bug ? > > Thanks, > -Yadu > > > > On Mon, Sep 16, 2013 at 1:52 PM, Mihael Hategan wrote: > > > I can confirm that this is an issue with write reference counting in > > trunk. I'm working on it. > > > > On Mon, 2013-09-16 at 11:29 -0500, Michael Wilde wrote: > > > If this is indeed failing on trunk, hopefully the test suite will see > > the same thing. > > > Yadu, can you look into this and file a ticket if its indeed a trunk bug? > > > > > > Thanks, > > > > > > - Mike > > > > > > ----- Original Message ----- > > > > From: "Ketan Maheshwari" > > > > To: "Michael Wilde" > > > > Cc: "Swift Devel" > > > > Sent: Monday, September 16, 2013 11:23:54 AM > > > > Subject: Re: [Swift-devel] iterate fails > > > > > > > > > > > > > > > > Mike, > > > > > > > > I am using the latest trunk: > > > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > > > > > > > > > > > > > > > > > > On Mon, Sep 16, 2013 at 11:20 AM, Michael Wilde < wilde at mcs.anl.gov > > > > > wrote: > > > > > > > > > > > > Ketan, this worked for me (see below). What version of swift did you > > > > run where? > > > > > > > > I ran on swift.rcc, on localhost. > > > > > > > > - Mike > > > > > > > > swift$ swift km2.swift -data=data.txt > > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > > > > > RunID: 20130916-1617-i570fay7 > > > > Progress: time: Mon, 16 Sep 2013 16:17:32 +0000 > > > > Final status: Mon, 16 Sep 2013 16:17:33 +0000 Finished successfully:3 > > > > > > > > swift$ ls -lt outdir | head > > > > total 16032 > > > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0002.out > > > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0001.out > > > > -rw-rw-r-- 1 wilde wilde 9 Sep 16 11:17 f.0000.out > > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0483.out > > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0490.out > > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0488.out > > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0500.out > > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0498.out > > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0491.out > > > > swift$ swift -version > > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > > > > > swift$ > > > > > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > > From: "Ketan Maheshwari" < ketancmaheshwari at gmail.com > > > > > > To: "Swift Devel" < swift-devel at ci.uchicago.edu > > > > > > Sent: Monday, September 16, 2013 10:49:07 AM > > > > > Subject: [Swift-devel] iterate fails > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > I am trying a catsn example with foreach replaced by iterate as > > > > > follows: > > > > > > > > > > === > > > > > type file; > > > > > > > > > > app (file o) cat (file i) { > > > > > cat @i stdout=@o; > > > > > } > > > > > file out[] > > > > prefix="f.",suffix=".out">; > > > > > > > > > > string d=@arg("data"); > > > > > > > > > > #foreach j in [1:1000] { > > > > > iterate j { > > > > > file data; > > > > > out[j] = cat(data); > > > > > } until (j==3); > > > > > > > > > > > > > > > === > > > > > > > > > > > > > > > The run fails with error message as follows: > > > > > $ swift -tc.file tc -config cf -sites.file sites.xml catsn.swift > > > > > -data=data.txt > > > > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > > > RunID: 20130916-1045-tldyyexe > > > > > Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 > > > > > > > > > > Execution failed: > > > > > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not > > > > > found for out of size 2 > > > > > swift:getFieldSubscript @ catsn, line: 93 > > > > > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array > > > > > index > > > > > '[2]' not found for out of size 2 > > > > > > > > > > > > > > > > > > > > Any suggestions? > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > -- > > > > > Ketan > > > > > > > > > > > > > > > _______________________________________________ > > > > > Swift-devel mailing list > > > > > Swift-devel at ci.uchicago.edu > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > Hi Mihael, > > > Is this the same issue that was discussed in this mail -> > http://lists.ci.uchicago.edu/pipermail/swift-devel/2013-March/010109.html > > > I am seeing this script failing on trunk (but not on 0.94). The > internals stress test > runs this script which does an assignment to an array within an > iterate loop -> > https://github.com/yadudoc/swift-basics/blob/master/stress/x_iterate.swift > The test-suite has some pending issues on trunk, which I haven't > cleaned up yet, > and which is why this went under the radar. > > > Here's the error I get when running stress/x_iterate.swift: > [yadunand at midway001 x_iterate-142712]$ swift x_iterate.swift -loops=5 > Swift trunk swift-r7071 (swift modified locally) cog-r3787 (cog > modified locally) > RunID: 20130916-1936-i6mqe9qa > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > declaration of element 'config'. > Progress: time: Mon, 16 Sep 2013 19:36:50 +0000 > Result[0] = 0 > > > Execution failed: > java.lang.IndexOutOfBoundsException: Invalid index [4] for result > tracef @ x_iterate, line: 56 > Caused by: java.lang.IndexOutOfBoundsException: Invalid index [4] for > result > at > org.griphyn.vdl.mapping.AbstractDataNode.checkNoValue(AbstractDataNode.java:716) > at > org.griphyn.vdl.mapping.AbstractDataNode.waitFor(AbstractDataNode.java:682) > at > org.griphyn.vdl.karajan.lib.SwiftFunction.waitForAll(SwiftFunction.java:343) > at > org.griphyn.vdl.karajan.lib.swiftscript.Tracef.function(Tracef.java:57) > at > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.UParallel > $1.run(UParallel.java:68) > at k.thr.LWThread.run(LWThread.java:228) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) > > > Here's the error I'm getting when running Ketan's code : > Execution failed: > org.griphyn.vdl.mapping.InvalidPathException: Array index '[2]' not > found for out of size 2 > swift:getFieldSubscript @ iterate, line: 93 > Caused by: org.griphyn.vdl.mapping.InvalidPathException: Array index > '[2]' not found for out of size 2 > > > Are these coming from the same bug ? > > > Thanks, > -Yadu > > > > > > On Mon, Sep 16, 2013 at 1:52 PM, Mihael Hategan > wrote: > I can confirm that this is an issue with write reference > counting in > trunk. I'm working on it. > > On Mon, 2013-09-16 at 11:29 -0500, Michael Wilde wrote: > > If this is indeed failing on trunk, hopefully the test suite > will see the same thing. > > Yadu, can you look into this and file a ticket if its indeed > a trunk bug? > > > > Thanks, > > > > - Mike > > > > ----- Original Message ----- > > > From: "Ketan Maheshwari" > > > To: "Michael Wilde" > > > Cc: "Swift Devel" > > > Sent: Monday, September 16, 2013 11:23:54 AM > > > Subject: Re: [Swift-devel] iterate fails > > > > > > > > > > > > Mike, > > > > > > I am using the latest trunk: > > > Swift trunk swift-r7065 (swift modified locally) cog-r3782 > > > > > > > > > > > > > > > On Mon, Sep 16, 2013 at 11:20 AM, Michael Wilde < > wilde at mcs.anl.gov > > > > wrote: > > > > > > > > > Ketan, this worked for me (see below). What version of > swift did you > > > run where? > > > > > > I ran on swift.rcc, on localhost. > > > > > > - Mike > > > > > > swift$ swift km2.swift -data=data.txt > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > > > RunID: 20130916-1617-i570fay7 > > > Progress: time: Mon, 16 Sep 2013 16:17:32 +0000 > > > Final status: Mon, 16 Sep 2013 16:17:33 +0000 Finished > successfully:3 > > > > > > swift$ ls -lt outdir | head > > > total 16032 > > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0002.out > > > -rw-r--r-- 1 wilde wilde 9 Sep 16 11:17 f.0001.out > > > -rw-rw-r-- 1 wilde wilde 9 Sep 16 11:17 f.0000.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0483.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0490.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0488.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0500.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0498.out > > > -rw-rw-r-- 1 wilde wilde 9 Jul 27 17:05 f.0491.out > > > swift$ swift -version > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > > > swift$ > > > > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Ketan Maheshwari" < ketancmaheshwari at gmail.com > > > > > To: "Swift Devel" < swift-devel at ci.uchicago.edu > > > > > Sent: Monday, September 16, 2013 10:49:07 AM > > > > Subject: [Swift-devel] iterate fails > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > I am trying a catsn example with foreach replaced by > iterate as > > > > follows: > > > > > > > > === > > > > type file; > > > > > > > > app (file o) cat (file i) { > > > > cat @i stdout=@o; > > > > } > > > > file out[] > > > prefix="f.",suffix=".out">; > > > > > > > > string d=@arg("data"); > > > > > > > > #foreach j in [1:1000] { > > > > iterate j { > > > > file data; > > > > out[j] = cat(data); > > > > } until (j==3); > > > > > > > > > > > > === > > > > > > > > > > > > The run fails with error message as follows: > > > > $ swift -tc.file tc -config cf -sites.file sites.xml > catsn.swift > > > > -data=data.txt > > > > Swift trunk swift-r7065 (swift modified locally) > cog-r3782 > > > > RunID: 20130916-1045-tldyyexe > > > > Progress: time: Mon, 16 Sep 2013 10:45:41 -0500 > > > > > > > > Execution failed: > > > > org.griphyn.vdl.mapping.InvalidPathException: Array > index '[2]' not > > > > found for out of size 2 > > > > swift:getFieldSubscript @ catsn, line: 93 > > > > Caused by: org.griphyn.vdl.mapping.InvalidPathException: > Array > > > > index > > > > '[2]' not found for out of size 2 > > > > > > > > > > > > > > > > Any suggestions? > > > > > > > > > > > > Thanks, > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > -- > > > Ketan > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > -- > Yadu Nand B > > From hategan at mcs.anl.gov Mon Sep 16 15:50:00 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Mon, 16 Sep 2013 13:50:00 -0700 Subject: [Swift-devel] iterate fails In-Reply-To: References: Message-ID: <1379364600.30928.1.camel@echo> Should work now (r7072). Mihael On Mon, 2013-09-16 at 10:49 -0500, Ketan Maheshwari wrote: > Hi, > > I am trying a catsn example with foreach replaced by iterate as follows: > > === > type file; > > app (file o) cat (file i) { > cat @i stdout=@o; > } > file out[]; > > string d=@arg("data"); > > #foreach j in [1:1000] { > iterate j { > file data; > out[j] = cat(data); > } until (j==3); > === [...] From ketancmaheshwari at gmail.com Mon Sep 16 16:06:34 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Mon, 16 Sep 2013 16:06:34 -0500 Subject: [Swift-devel] iterate fails In-Reply-To: <1379364600.30928.1.camel@echo> References: <1379364600.30928.1.camel@echo> Message-ID: Works now, thanks! On Mon, Sep 16, 2013 at 3:50 PM, Mihael Hategan wrote: > Should work now (r7072). > > Mihael > > On Mon, 2013-09-16 at 10:49 -0500, Ketan Maheshwari wrote: > > Hi, > > > > I am trying a catsn example with foreach replaced by iterate as follows: > > > > === > > type file; > > > > app (file o) cat (file i) { > > cat @i stdout=@o; > > } > > file out[]; > > > > string d=@arg("data"); > > > > #foreach j in [1:1000] { > > iterate j { > > file data; > > out[j] = cat(data); > > } until (j==3); > > === > [...] > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadudoc1729 at gmail.com Mon Sep 16 16:29:13 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Mon, 16 Sep 2013 16:29:13 -0500 Subject: [Swift-devel] Use wrapper staging parameter in Trunk required? Message-ID: Hi, In 0.94 the config used by tests do not specify the line use.wrapper.staging in the config file and they pass.The same tests on trunk fail with the error pasted here -> [yadunand at midway001 x_append_iter-142706]$ swift -tc.file tc.data -sites.file sites.xml -config swift.properties x_append_iter.swift Swift trunk swift-r7071 (swift modified locally) cog-r3787 (cog modified locally) RunID: 20130916-2120-ovvu8v0g Could not start execution: Import of 'swift.k' failed: Swift config property "use.wrapper.staging" not found in Swift configuration ([swift.properties]): {throttle.score.job.factor=4, throttle.file.operations=8, throttle.host.submit=2, pgraph.node.options=color="seagreen", style="filled", wrapper.parameter.mode=args, provenance.log=false, throttle.submit=4, ticker.prefix=Progress: time:, provider.staging.pin.swiftfiles=false, clustering.enabled=false, replication.min.queue.time=60, sites.file=sites.xml, cdm.broadcast.mode=file, kickstart.enabled=maybe, wrapperlog.always.transfer=true, caching.algorithm=LRU, use.provider.staging=false, status.mode=provider, clustering.min.time=60, ticker.date.format=, kickstart.always.transfer=false, file.gc.enabled=true, throttle.transfers=4, execution.retries=0, sitedir.keep=true, pgraph.graph.options=splines="compound", rankdir="TB", replication.enabled=false, clustering.queue.delay=4, pgraph=false, mapping.checker=on, wrapper.invocation.mode=absolute, tc.file=tc.data, replication.limit=3, lazy.errors=false}: Swift config property "use.wrapper.staging" not found in Swift configuration ([swift.properties]): {throttle.score.job.factor=4, throttle.file.operations=8, throttle.host.submit=2, pgraph.node.options=color="seagreen", style="filled", wrapper.parameter.mode=args, provenance.log=false, throttle.submit=4, ticker.prefix=Progress: time:, provider.staging.pin.swiftfiles=false, clustering.enabled=false, replication.min.queue.time=60, sites.file=sites.xml, cdm.broadcast.mode=file, kickstart.enabled=maybe, wrapperlog.always.transfer=true, caching.algorithm=LRU, use.provider.staging=false, status.mode=provider, clustering.min.time=60, ticker.date.format=, kickstart.always.transfer=false, file.gc.enabled=true, throttle.transfers=4, execution.retries=0, sitedir.keep=true, pgraph.graph.options=splines="compound", rankdir="TB", replication.enabled=false, clustering.queue.delay=4, pgraph=false, mapping.checker=on, wrapper.invocation.mode=absolute, tc.file=tc.data, replication.limit=3, lazy.errors=false} Is this expected behavior ? -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Mon Sep 16 16:52:21 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Mon, 16 Sep 2013 14:52:21 -0700 Subject: [Swift-devel] Use wrapper staging parameter in Trunk required? In-Reply-To: References: Message-ID: <1379368341.30928.4.camel@echo> There was a commit to validate configuration settings. This is a consequence of that and the fact that there is no default value for this setting. I added a default value in r7073. Mihael On Mon, 2013-09-16 at 16:29 -0500, Yadu Nand wrote: > Hi, > > In 0.94 the config used by tests do not specify the line > use.wrapper.staging in the > config file and they pass.The same tests on trunk fail with the error > pasted here -> > > [yadunand at midway001 x_append_iter-142706]$ swift -tc.file tc.data > -sites.file sites.xml -config swift.properties x_append_iter.swift > Swift trunk swift-r7071 (swift modified locally) cog-r3787 (cog modified > locally) > RunID: 20130916-2120-ovvu8v0g > Could not start execution: > Import of 'swift.k' failed: > Swift config property "use.wrapper.staging" not found in Swift > configuration ([swift.properties]): {throttle.score.job.factor=4, > throttle.file.operations=8, throttle.host.submit=2, > pgraph.node.options=color="seagreen", style="filled", > wrapper.parameter.mode=args, provenance.log=false, throttle.submit=4, > ticker.prefix=Progress: time:, provider.staging.pin.swiftfiles=false, > clustering.enabled=false, replication.min.queue.time=60, > sites.file=sites.xml, cdm.broadcast.mode=file, kickstart.enabled=maybe, > wrapperlog.always.transfer=true, caching.algorithm=LRU, > use.provider.staging=false, status.mode=provider, clustering.min.time=60, > ticker.date.format=, kickstart.always.transfer=false, file.gc.enabled=true, > throttle.transfers=4, execution.retries=0, sitedir.keep=true, > pgraph.graph.options=splines="compound", rankdir="TB", > replication.enabled=false, clustering.queue.delay=4, pgraph=false, > mapping.checker=on, wrapper.invocation.mode=absolute, tc.file=tc.data, > replication.limit=3, lazy.errors=false}: > Swift config property "use.wrapper.staging" not found in Swift > configuration ([swift.properties]): {throttle.score.job.factor=4, > throttle.file.operations=8, throttle.host.submit=2, > pgraph.node.options=color="seagreen", style="filled", > wrapper.parameter.mode=args, provenance.log=false, throttle.submit=4, > ticker.prefix=Progress: time:, provider.staging.pin.swiftfiles=false, > clustering.enabled=false, replication.min.queue.time=60, > sites.file=sites.xml, cdm.broadcast.mode=file, kickstart.enabled=maybe, > wrapperlog.always.transfer=true, caching.algorithm=LRU, > use.provider.staging=false, status.mode=provider, clustering.min.time=60, > ticker.date.format=, kickstart.always.transfer=false, file.gc.enabled=true, > throttle.transfers=4, execution.retries=0, sitedir.keep=true, > pgraph.graph.options=splines="compound", rankdir="TB", > replication.enabled=false, clustering.queue.delay=4, pgraph=false, > mapping.checker=on, wrapper.invocation.mode=absolute, tc.file=tc.data, > replication.limit=3, lazy.errors=false} > > Is this expected behavior ? > > Hi, > > > In 0.94 the config used by tests do not specify the line > use.wrapper.staging in the > config file and they pass.The same tests on trunk fail with the error > pasted here -> > > > [yadunand at midway001 x_append_iter-142706]$ swift -tc.file tc.data > -sites.file sites.xml -config swift.properties x_append_iter.swift > Swift trunk swift-r7071 (swift modified locally) cog-r3787 (cog > modified locally) > RunID: 20130916-2120-ovvu8v0g > Could not start execution: > Import of 'swift.k' failed: > Swift config property "use.wrapper.staging" not found in Swift > configuration ([swift.properties]): {throttle.score.job.factor=4, > throttle.file.operations=8, throttle.host.submit=2, > pgraph.node.options=color="seagreen", style="filled", > wrapper.parameter.mode=args, provenance.log=false, throttle.submit=4, > ticker.prefix=Progress: time:, provider.staging.pin.swiftfiles=false, > clustering.enabled=false, replication.min.queue.time=60, > sites.file=sites.xml, cdm.broadcast.mode=file, > kickstart.enabled=maybe, wrapperlog.always.transfer=true, > caching.algorithm=LRU, use.provider.staging=false, > status.mode=provider, clustering.min.time=60, ticker.date.format=, > kickstart.always.transfer=false, file.gc.enabled=true, > throttle.transfers=4, execution.retries=0, sitedir.keep=true, > pgraph.graph.options=splines="compound", rankdir="TB", > replication.enabled=false, clustering.queue.delay=4, pgraph=false, > mapping.checker=on, wrapper.invocation.mode=absolute, tc.file=tc.data, > replication.limit=3, lazy.errors=false}: > Swift config property "use.wrapper.staging" not found in Swift > configuration ([swift.properties]): {throttle.score.job.factor=4, > throttle.file.operations=8, throttle.host.submit=2, > pgraph.node.options=color="seagreen", style="filled", > wrapper.parameter.mode=args, provenance.log=false, throttle.submit=4, > ticker.prefix=Progress: time:, provider.staging.pin.swiftfiles=false, > clustering.enabled=false, replication.min.queue.time=60, > sites.file=sites.xml, cdm.broadcast.mode=file, > kickstart.enabled=maybe, wrapperlog.always.transfer=true, > caching.algorithm=LRU, use.provider.staging=false, > status.mode=provider, clustering.min.time=60, ticker.date.format=, > kickstart.always.transfer=false, file.gc.enabled=true, > throttle.transfers=4, execution.retries=0, sitedir.keep=true, > pgraph.graph.options=splines="compound", rankdir="TB", > replication.enabled=false, clustering.queue.delay=4, pgraph=false, > mapping.checker=on, wrapper.invocation.mode=absolute, tc.file=tc.data, > replication.limit=3, lazy.errors=false} > > > Is this expected behavior ? > > > -- > Yadu Nand B > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From hategan at mcs.anl.gov Mon Sep 16 22:26:54 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Mon, 16 Sep 2013 20:26:54 -0700 Subject: [Swift-devel] returning primitive values from apps Message-ID: <1379388414.8674.1.camel@echo> Please take a look at https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=293 I think we should discuss it. Mihael From ketancmaheshwari at gmail.com Tue Sep 17 09:28:55 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Tue, 17 Sep 2013 09:28:55 -0500 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: <1379388414.8674.1.camel@echo> References: <1379388414.8674.1.camel@echo> Message-ID: I am in favor of this feature. Mike's suggestion of built-in ReadData logic in app invocation routine sounds like a good idea. On Mon, Sep 16, 2013 at 10:26 PM, Mihael Hategan wrote: > Please take a look at > https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=293 > > I think we should discuss it. > > Mihael > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Sep 17 10:37:23 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 17 Sep 2013 08:37:23 -0700 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: References: <1379388414.8674.1.camel@echo> Message-ID: <1379432243.15304.1.camel@echo> I'm in favor of the idea too, but I think we need a discussion about how it will look like in the language and how it would look like to a user. Mihael On Tue, 2013-09-17 at 09:28 -0500, Ketan Maheshwari wrote: > I am in favor of this feature. Mike's suggestion of built-in ReadData logic > in app invocation routine sounds like a good idea. > > > On Mon, Sep 16, 2013 at 10:26 PM, Mihael Hategan wrote: > > > Please take a look at > > https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=293 > > > > I think we should discuss it. > > > > Mihael > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > From hereld at mcs.anl.gov Tue Sep 17 11:10:04 2013 From: hereld at mcs.anl.gov (Mark Hereld) Date: Tue, 17 Sep 2013 11:10:04 -0500 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: <1379432243.15304.1.camel@echo> References: <1379388414.8674.1.camel@echo> <1379432243.15304.1.camel@echo> Message-ID: <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> do you have a list of candidate syntax ideas for this? -mark On Sep 17, 2013, at 10:37 AM, Mihael Hategan wrote: > I'm in favor of the idea too, but I think we need a discussion about how > it will look like in the language and how it would look like to a user. > > Mihael > > On Tue, 2013-09-17 at 09:28 -0500, Ketan Maheshwari wrote: >> I am in favor of this feature. Mike's suggestion of built-in ReadData logic >> in app invocation routine sounds like a good idea. >> >> >> On Mon, Sep 16, 2013 at 10:26 PM, Mihael Hategan wrote: >> >>> Please take a look at >>> https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=293 >>> >>> I think we should discuss it. >>> >>> Mihael >>> >>> _______________________________________________ >>> Swift-devel mailing list >>> Swift-devel at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>> >> > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel ------------------------------------------------------- Mark Hereld Senior Fellow - Computation Institute Experimental Systems Engineer - Mathematics and Computer Science Visualization and Analysis Lead - Argonne Leadership Computing Facility Argonne National Laboratory The University of Chicago Cell: 630.327.2088 Voice: 630.252.4170 -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadudoc1729 at gmail.com Tue Sep 17 11:26:08 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Tue, 17 Sep 2013 11:26:08 -0500 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> References: <1379388414.8674.1.camel@echo> <1379432243.15304.1.camel@echo> <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> Message-ID: Hi Mihael, If say the app has the following syntax : app (int ret_value) do_math (int x, int y) { add x y stdout=ret_value; } Would this (hypothetical) syntax entirely avoid the overhead of writing several small files as we see in larger runs ? Thanks, Yadu On Tue, Sep 17, 2013 at 11:10 AM, Mark Hereld wrote: > do you have a list of candidate syntax ideas for this? > -mark > > On Sep 17, 2013, at 10:37 AM, Mihael Hategan wrote: > > I'm in favor of the idea too, but I think we need a discussion about how > it will look like in the language and how it would look like to a user. > > Mihael > > On Tue, 2013-09-17 at 09:28 -0500, Ketan Maheshwari wrote: > > I am in favor of this feature. Mike's suggestion of built-in ReadData logic > in app invocation routine sounds like a good idea. > > > On Mon, Sep 16, 2013 at 10:26 PM, Mihael Hategan >wrote: > > Please take a look at > https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=293 > > I think we should discuss it. > > Mihael > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > ------------------------------------------------------- > Mark Hereld > Senior Fellow - Computation Institute > Experimental Systems Engineer - Mathematics and Computer Science > Visualization and Analysis Lead - Argonne Leadership Computing Facility > Argonne National Laboratory > The University of Chicago > > Cell: 630.327.2088 > Voice: 630.252.4170 > > > > > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Sep 17 11:35:38 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 17 Sep 2013 09:35:38 -0700 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> References: <1379388414.8674.1.camel@echo> <1379432243.15304.1.camel@echo> <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> Message-ID: <1379435738.15957.16.camel@echo> Hi Mark, Sadly, not much. Ketan, a while ago, mentioned that he would like to see something of the form: app ( value) f() { f stdout = filename(value); } I complained that the idea of filename() on a primitive type would be confusing. But then it just occurred to me that the idea of "stdout = " is also confusing. So it might be that we could have something like this: f stdout > value; If value is a file type, this would be interpreted as what today we would express as stdout = filename(value), except in a more natural form if you come from shell scripting. It would also then make sense to use that for primitive types and "compile" it to something like value = readData(someFileThatHasTheStdoutOfTheProcess). I'm not sure how to extend this to non-stdout/err things. The only other standard way to communicate with a process is through command line parameters. We use those sometimes to supply the app input and output file names, but how we would make it such that primitive values go through a file and the scheme is expressed in a way that makes it unambiguous for the user to tell what it means, that I'm not quite sure of. Perhaps something more explicit might work: app (int r1, int r2) f(int p1, int p2) { file tmpr1, tmpr2, tmpp1, tmpp2; p1 > tmpp1; p2 > tmpp2; f @tmpp1 @tmpp2 @tmpr1 @tmpr2; r1 < tmpr1; r2 < tmpr2; } Mihael On Tue, 2013-09-17 at 11:10 -0500, Mark Hereld wrote: > do you have a list of candidate syntax ideas for this? > -mark > > On Sep 17, 2013, at 10:37 AM, Mihael Hategan wrote: > > > I'm in favor of the idea too, but I think we need a discussion about how > > it will look like in the language and how it would look like to a user. > > > > Mihael > > > > On Tue, 2013-09-17 at 09:28 -0500, Ketan Maheshwari wrote: > >> I am in favor of this feature. Mike's suggestion of built-in ReadData logic > >> in app invocation routine sounds like a good idea. > >> > >> > >> On Mon, Sep 16, 2013 at 10:26 PM, Mihael Hategan wrote: > >> > >>> Please take a look at > >>> https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=293 > >>> > >>> I think we should discuss it. > >>> > >>> Mihael > >>> > >>> _______________________________________________ > >>> Swift-devel mailing list > >>> Swift-devel at ci.uchicago.edu > >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >>> > >> > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > ------------------------------------------------------- > Mark Hereld > Senior Fellow - Computation Institute > Experimental Systems Engineer - Mathematics and Computer Science > Visualization and Analysis Lead - Argonne Leadership Computing Facility > Argonne National Laboratory > The University of Chicago > > Cell: 630.327.2088 > Voice: 630.252.4170 > > > > > > > > do you have a list of candidate syntax ideas for this? > -mark > > On Sep 17, 2013, at 10:37 AM, Mihael Hategan > wrote: > > > I'm in favor of the idea too, but I think we need a discussion about > > how > > it will look like in the language and how it would look like to a > > user. > > > > Mihael > > > > On Tue, 2013-09-17 at 09:28 -0500, Ketan Maheshwari wrote: > > > I am in favor of this feature. Mike's suggestion of built-in > > > ReadData logic > > > in app invocation routine sounds like a good idea. > > > > > > > > > On Mon, Sep 16, 2013 at 10:26 PM, Mihael Hategan > > > wrote: > > > > > > > Please take a look at > > > > https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=293 > > > > > > > > I think we should discuss it. > > > > > > > > Mihael > > > > > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > ------------------------------------------------------- > Mark Hereld > Senior Fellow - Computation Institute > Experimental Systems Engineer - Mathematics and Computer Science > Visualization and Analysis Lead - Argonne Leadership Computing > Facility > Argonne National Laboratory > The University of Chicago > > > Cell: 630.327.2088 > Voice: 630.252.4170 > > > > > > > > > > > > > From hategan at mcs.anl.gov Tue Sep 17 11:43:21 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 17 Sep 2013 09:43:21 -0700 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: References: <1379388414.8674.1.camel@echo> <1379432243.15304.1.camel@echo> <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> Message-ID: <1379436201.15957.23.camel@echo> On Tue, 2013-09-17 at 11:26 -0500, Yadu Nand wrote: > Hi Mihael, > > If say the app has the following syntax : > app (int ret_value) do_math (int x, int y) { > add x y stdout=ret_value; > } > > Would this (hypothetical) syntax entirely avoid the overhead of writing > several small files as > we see in larger runs ? Maybe. If you can re-direct stdout straight to memory (only some providers support this), then you could read the value without writing files on the client side. From hockyg at uchicago.edu Tue Sep 17 12:03:38 2013 From: hockyg at uchicago.edu (Glen Hocky) Date: Tue, 17 Sep 2013 13:03:38 -0400 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: <1379436201.15957.23.camel@echo> References: <1379388414.8674.1.camel@echo> <1379432243.15304.1.camel@echo> <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> <1379436201.15957.23.camel@echo> Message-ID: Currently, schematically I would do something like this as ------------------ struct outdata { logfile l; returnfile r; } (outdata o) app ( a , b ,c ... ){ run "-logfile" @filename(o.l) "-returnfile" @filename(o.r) } o = app(a,b,c) int r = readData(o.r) -------------- It wouldn't be as general as the options just suggested, but would one option be to have a meta-datatype like "intfile" "stringfile" "floatfile" and "booleanfile" that automatically performs a readdata to extract a single value from a file and store it in a variable of that standard type? On Tue, Sep 17, 2013 at 12:43 PM, Mihael Hategan wrote: > On Tue, 2013-09-17 at 11:26 -0500, Yadu Nand wrote: > > Hi Mihael, > > > > If say the app has the following syntax : > > app (int ret_value) do_math (int x, int y) { > > add x y stdout=ret_value; > > } > > > > Would this (hypothetical) syntax entirely avoid the overhead of writing > > several small files as > > we see in larger runs ? > > Maybe. If you can re-direct stdout straight to memory (only some > providers support this), then you could read the value without writing > files on the client side. > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Sep 17 16:06:19 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 17 Sep 2013 14:06:19 -0700 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: References: <1379388414.8674.1.camel@echo> <1379432243.15304.1.camel@echo> <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> <1379436201.15957.23.camel@echo> Message-ID: <1379451979.16530.11.camel@echo> On Tue, 2013-09-17 at 13:03 -0400, Glen Hocky wrote: > Currently, schematically I would do something like this as > > ------------------ > > struct outdata { > logfile l; > returnfile r; > } > > (outdata o) app ( a , b ,c ... ){ > run "-logfile" @filename(o.l) "-returnfile" @filename(o.r) > } > > o = app(a,b,c) > > int r = readData(o.r) > > -------------- > > It wouldn't be as general as the options just suggested, but would one > option be to have a meta-datatype like "intfile" "stringfile" "floatfile" > and "booleanfile" > that automatically performs a readdata to extract a single value from a > file and store it in a variable of that standard type? Thanks for the feedback Glen! The swift compiler could be able to properly infer that in situations like the one you describe where a file pointer is requested for a primitive type, it really means that it should pass through read/writeData. So I don't think separate types are necessary. What I'm looking for is an unambiguous and non-confusing way to express this in the language. I'm not sure about filename(int) since it evokes feelings of the int not being an int, but then it also makes it clear that the app will not receive an int value on the command line, but a file name instead. So maybe that is the way to go. It also has the advantage that the parser doesn't need to be modified (and that is a BIG advantage from my perspective). Mihael From hategan at mcs.anl.gov Tue Sep 17 17:08:16 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 17 Sep 2013 15:08:16 -0700 Subject: [Swift-devel] returning primitive values from apps In-Reply-To: References: <1379388414.8674.1.camel@echo> <1379432243.15304.1.camel@echo> <656AF7FD-BDCC-497B-B81E-3AE0C5EB7617@mcs.anl.gov> <1379436201.15957.23.camel@echo> <1379451979.16530.11.camel@echo> Message-ID: <1379455696.19089.2.camel@echo> [getting this back on the mailing list] On Tue, 2013-09-17 at 18:02 -0400, Glen Hocky wrote: > Maybe you could add a primitive function like extract() or output() which > is an alias to filename() but could make the syntax more clear in > situations such as this? From iraicu at cs.iit.edu Tue Sep 17 23:12:38 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Tue, 17 Sep 2013 23:12:38 -0500 Subject: [Swift-devel] CFP: IEEE/ACM Int. Symp. on Cluster, Cloud and Grid Computing (CCGrid'14) -- Chicago May 26-29, 2014 Message-ID: <52392836.706@cs.iit.edu> -------------------------- IEEE/ACM CCGrid 2014 -------------------------- 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing May 26-29, 2014 Chicago, IL, USA http://datasys.cs.iit.edu/events/CCGrid2014/ Rapid advances in architectures, networks, and systems and middleware technologies are leading to new concepts and platforms for computing, ranging from Clusters and Grids to Clouds and Datacenters. The 14th Annual IEEE/ACM International Symposium in Cluster, Cloud, and Grid Computing (CCGrid 2014) is a forum bringing together international researchers, developers, and practitioners to present leading research activities and results on a broad range of topics related to these concepts and platforms, and their applications. The conference features keynotes, technical presentations, workshops, tutorials, and posters, as well as the SCALE challenge featuring live demonstrations. In 2014, CCGrid will return to the USA and be held in Chicago, the third largest city in the United States. The main conference will be held on May 27-29, 2014, with tutorials and affiliated workshops taking place on May 26, 2014. IMPORTANT DATES (Anywhere on Earth) Workshop Proposals Due: November 3rd, 2013 Papers Due: November 11th, 2013 Author Notifications: January 24th, 2014 Camera Ready Papers Due: February 17th, 2014 Posters Due: February 24th, 2014 Tutorials Due: March 4th, 2014 TOPICS OF INTEREST CCGrid 2014 will have a focus on important and immediate issues that are significantly influencing all aspects of cluster, cloud and grid computing. Topics of interest include, but are not limited to: * Applications and Experiences: Applications to real and complex problems in science, engineering, business, and society; User studies; Experiences with large-scale deployments, systems, or applications * Architecture and Accelerators: Design and use of emergent system architectures including but not limited to accelerators; Multicores; Power and cooling; Security and reliability; High availability solutions * Autonomic Computing and Cyberinfrastructure: Self-managed behavior, models and technologies; Autonomic paradigms and systems (control-based, bio-inspired, emergent, etc.); Bio-inspired optimizations and computing * Cloud Computing: Development, evaluation, and optimization of cloud architectures; Software tools and techniques for clouds * Modeling and Evaluation of Performance and Energy: Prediction, modeling and analysis; Monitoring and evaluation tools; Benchmarks and testbeds * Programming Models, Systems, and Fault-Tolerant Computing: Programming models, systems software and environments for cluster, cloud, and grid computing; Fault-tolerant systems, programs and algorithms * Scheduling and Resource Management: Techniques to schedule jobs and resources on cluster, cloud, and grid computing platforms; SLA definition and enforcement * Storage and I/O Systems: System design and optimization for big data processing in cluster and cloud environment PAPER SUBMISSION GUIDELINES Authors are invited to submit papers electronically in PDF format. Submitted manuscripts should be structured as technical papers and may not exceed 10 letter-size (8.5 x 11) pages including figures, tables and references using the IEEE format for conference proceedings. Submissions not conforming to these guidelines may be returned without review. The official language of the conference is English. All manuscripts will be reviewed and judged on correctness, originality, technical strength, significance, quality of presentation, and interest and relevance to the conference attendees. Paper submissions are limited to 10 pages in 2-column IEEE format including all figures and references. Submitted manuscripts exceeding this limit will be returned without review. For the final camera-ready version, authors with accepted papers may purchase additional pages at the following rates: 100 USD for each of the first two additional pages; 200 USD for each of the third and fourth additional pages. Submitted papers must represent original unpublished research that is not currently under review for any other conference or journal. Papers not following these guidelines will be rejected without review and further action may be taken, including (but not limited to) notifications sent to the heads of the institutions of the authors and sponsors of the conference. Submissions received after the due date, exceeding the page limit, or not appropriately structured may not be considered. Authors may contact the conference chairs for more information. The proceedings will be published through the IEEE Computer Society Press, USA, and will be made available online through the IEEE Digital Library. The paper submission online system will be available in late September 2013. JOURNAL SPECIAL ISSUE IEEE Transactions on Cloud Computing (TCC: http://computer.org/TCC) is organizing a Special Issue which encourages submission of revised and extended versions of 2-3 best/top rated papers in the area of Cloud Computing from our conference. The special issue also seeks direct submission of papers that present 'new' ideas for the first time in TCC. All papers will be peer-reviewed and selected competitively based on their originality and merit as per requirement of TCC. All queries on this special issue should be directed to its guest editors. Details on this special issue will be informed about in a separate Call for Papers at http://www.computer.org/cms/Computer.org/transactions/cfps/cfp_tccsi_apbdac.pdf. CALL FOR TUTORIAL, WORKSHOP PROPOSALS, AND POSTERS Tutorials and workshops affiliated with CCGrid 2014 will be held on May 26, 2014. Poster presentations will be done during the main conference dates between May 27th and May 29th 2014. For more information on the tutorials, workshops, posters, and doctoral symposium, see the following links: - Tutorials: http://datasys.cs.iit.edu/events/CCGrid2014/call-tutorials.html - Workshops: http://datasys.cs.iit.edu/events/CCGrid2014/call-workshops.html - Posters: http://datasys.cs.iit.edu/events/CCGrid2014/call-posters.html - Doctoral Symposium: http://datasys.cs.iit.edu/events/CCGrid2014/call-doctoral.html ORGANIZATION Honorary General Chair Daniel A. Reed, University of Iowa, USA General Chairs Xian-He Sun, Illinois Institute of Technology, USA Ian T. Foster, University of Chicago & Argonne National Laboratory, USA Program Committee Chair Kirk W. Cameron, Virginia Tech, USA Dimitris S. Nikolopoulos, Queen's University of Belfast, UK APPLICATIONS Esmond G. Ng, Lawrence Berkeley National Laboratory, USA ARCHITECTURE & ACCELERATORS Suren Byna, Lawrence Berkeley National Laboratory, USA AUTONOMIC COMPUTING AND CYBERINFRASTUCTURE Vladimir Getov, University of Westminster, UK Xiaobo Zhou, University of Colorado, USA CLOUD COMPUTING Ali Butt, Virginia Tech, USA Xiaosong Ma, North Carolina State University, USA MODELING AND EVALUATION OF PERFORMANCE AND ENERGY David Lowenthal, University of Arizona, USA PROGRAMMING MODELS, SYSTEMS, AND FAULT-TOLERANT COMPUTING Kasidit Chanchio, Thammasat University, Thailand Christian Engelmann, Oak Ridge National Laboratory, USA SCHEDULING AND RESOURCE MANAGEMENT Matthew Grove, Rackspace, USA Rizos Sakellariou, University of Manchester, UK STORAGE AND I/O SYSTEMS Dan Feng, Huazhong University of Science and Technology, China Workshops Co-Chairs Zhiling Lan, Illinois Institute of Technology, USA Matei Ripeanu, University of British Columbia, Canada Tutorials Co-Chairs Kate Keahey, University of Chicago & Argonne National Laboratory, USA Radu Prodan, University of Innsbruck, Austria Doctoral Symposium Co-Chairs Judy Qiu, Indiana University, USA Poster and Research Demo Chair Borja Sotomayor, University of Chicago, USA Hui Jin, Oracle, USA SCALE Challenge Coordinator Doug Thain, Notre Dame, USA Student Award Chair Yong Chen, Texas Tech University, USA Publicity Chair Bruno Schulze, LNCC, Brazil Cho-Li Wang, The University of Hong Kong, China Darren J. Kerbyson, Pacific Northwest National Laboratory, USA Toni Cortes UPC/BSC, Spain Cyber Co-Chairs Rong Ge, Marquette University, USA Wei Tang, Argonne National Laboratory, USA Proceedings Chair Pavan Balaji, Argonne National Laboratory, USA Local Organizing Committee Ioan Raicu, Illinois Institute of Technology, USA Kyle Chard, University of Chicago, USA Finance Chairs Dawn DeBartolo, Illinois Institute of Technology, USA Steering Committee Rajkumar Buyya, University of Melbourne, Australia (Chair) Craig Lee, The Aerospace Corporation, USA (Co-Chair) Henri Bal, Vrije University, The Netherlands Pavan Balaji, Argonne National Laboratory, USA Franck Capello, University of Paris-Sud, France Jack Dongarra, University of Tennessee & ORNL, USA Dick Epema, Technical University of Delft, The Netherlands Thomas Fahringer, University of Innsbruck, Austria Ian Foster, University of Chicago, USA Wolfgang Gentzsch, DEISA, Germany Hai Jin, Huazhong University of Science & Technology, China Laurent Lefevre, INRIA, France Geng Lin, Dell Inc., USA Manish Parashar, Rutgers: The State University of New Jersey, USA Shikharesh Majumdar, Carleton University, Canada Satoshi Matsuoaka, Tokyo Institute of Technology, Japan Omer Rana, Cardiff University, UK Paul Roe, Queensland University of Technology, Australia Bruno Schulze, LNCC, Brazil Nalini Venkatasubramanian, University of California, USA Carlos Varela, Rensselaer Polytechnic Institute, USA Technical Program Committee Applications: Bill Barth, University of Texas at Austin Costas Bekas, IBM Research - Zurich Sanjukta Bhowmick, University of Nebraska Yifeng Chen, Peking University Olivier Coulaud, INRIA Frederic Desprez, INRIA Daniel Martin, Lawrence Berkeley National Laboratory Kengo Nakajima, University of Tokyo Hai Ah Nam, Oak Ridge National Laboratory Esmond Ng, Lawrence Berkeley National Laboratory Dana Petcu, West University of Timisoara Judy Qiu, Indiana University Ashok Srinivasan, Florida State University Gerhard Wellein, Erlangen Regional Computing Center Rio Yokota, KAUST Yunquan Zhang, Institute of Software - Chinese Academy of Sciences ARCHITECTURE & ACCELERATORS Michela Becchi, University of Missouri Suren Byna, Lawrence Berkeley National Laboratory Srihari Cadambi, NEC Labs America Inc. Wenguang Chen, Tsinghua University Cong Du, Arista Networks Natalie Enright Jerger, University of Toronto Michael Ferdman, Stony Brook University Dong Li, Oak Rdige National Lab Sonia Lopez, Rochester Institute of Technology Kamesh Madduri, Pennsylvania State University Jiayuan Meng, Argonne National Lab Ahmad Samih, Intel Corporation Vinod Tipparaju, AMD Didem Unat, Lawrence Berkeley National Laboratory Sathish Vadhiyar, Indian Institute of Science ? Bangalore Abhinav Vishnu, Pacific Northwest National Laboratory Yili Zheng, LBNL AUTONOMIC COMPUTING AND CYBERINFRASTUCTURE Marco Aldinucci, University of Torino Francoise Baude, Universite de Nice Sophia-Antipolis Alexander Bolotov, University of Westminster Marian Bubak, AGH Krakow PL and University of Amsterdam Abhishek Chandra, University of Minnesota Yong Chen, Texas Tech University Yuan Chen, HP Labs Marco Danelutto, University of Pisa Xiaoning Ding, New Jersey Institute of Technology Jose Fortes, University of Florida Paraskevi Fragopoulou, FORTH-ICS Vladimir Getov, University of Westminster Sergei Gorlatch, University of Muenster Palden Lama, University of Texas at San Antonio Zhiqiang Lin, University of Texas at Dallas Cristian Lumezanu, NEC Laboratories Ningfang Mi, Northeastern University Christine Morin, INRIA Jia Rao, University of Colorado, Colorado Springs Ian Taylor, Cardiff University and Louisiana State University Thomas Weigold, IBM Research Timothy Wood, George Washington University Weikuan Yu, Auburn University Ming Zhao, Florida International University Xiaobo Zhou, University of Colorado CLOUD COMPUTING Gagan Agrawal, Ohio State University Raouf Boutaba, University of Waterloo Ali Butt, Virginia Tech Zhihui Du, Tsinghua University Dick Epema, Delft University of Technology Renato Figueiredo, University of Florida Geoffrey C. Fox, Indiana University Chuntao Hong, Microsoft Research Peter Kilpatrick, Queen's University Belfast Jong Kim, HPC Lab Youngjae Kim, Oak Ridge National Laboratory Shan Lu, University of Wisconsin at Madison Xiaosong Ma, North Carolina State University Kostas Magoutis, ICS-FORTH Gaurav Makkar, NetApp M. Mustafa Rafique, IBM Research Ioan Raicu, Illinois Institute of Technology Lavanya Ramakrishnan, Lawrence Berkeley National Lab Kai Sachs, SAP AG Prasenjit Sarkar, IBM Radu Sion, Stony Brook University Yang Song, IBM Osamu Tatebe, University of Tsukuba Douglas Thain, University of Notre Dame Devesh Tiwari, Oak Ridge National Laboratory Dimitrios Tsoumakos, Ionian University Guanying Wang, Google Jon Weissman, University of Minnesota Gianluigi Zanetti, CRS4 Jianfeng Zhan, Institute of Computing Technology - Chinese Academy of Sciences MODELING AND EVALUATION OF PERFORMANCE AND ENERGY Sadaf Alam, Swiss National Supercomputing Centre Dorian Arnold, University of New Mexico Frank Bellosa, University of Karlsruhe Peter Dinda, Northwestern University Todd Gamblin, Lawrence Livermore National Laboratory Lizy John, University of Texas at Austin Karen L. Karavanic, Portland State University Darren Kerbyson, Pacific Northwest National Lab James Laros, Sandia National Labs David Lowenthal, University of Arizona Naoya Maruyama, RIKEN AICS Barry Rountree, Lawrence Livermore National Laboratory Valerie Taylor, Texas A&M University Hans Vandierendonck, Queen's University Belfast Ana Varbanescu, University of Amsterdam Xin Yuan, Flroida State University PROGRAMMING MODELS, SYSTEMS, AND FAULT-TOLERANT COMPUTING Vassil Alexandrov, BSC Christos Antonopoulos, University of Thessaly Pavan Balaji, Argonne National Laboratory David E. Bernholdt, Oak Ridge National Laboratory Filip Blagojevic, Hitachi Research George Bosilca, University of Tennessee Patrick Bridges, University of New Mexico Greg Bronevetsky, Lawrence Livermore National Laboratory Franck Cappello, INRIA and University of Illinois at Urbana Champaign Kasidit Chanchio, Thammasat University Zizhong Chen, University of California at Riverside Andrew Chien, University of Chicago Nathan Debardeleben, Los Alamos National Laboratory Christian Engelmann, Oak Ridge National Laboratory Kurt Ferreira, Sandia National Laboratories Ada Gavrilovska, Georgia Institute of Technology Cecile Germain, LRI Michael Gerndt, Technische Universit?t M?nchen William Gropp, University of Illinois at Urbana-Champaign Thilo Kielmann, Vrije Universiteit Bettina Krammer, Universit? de Versailles St-Quentin-en-Yvelines (UVSQ) Dieter Kranzlm?ller, Ludwig-Maximilians-Universitaet Muenchen Sriram Krishnamoorthy, Pacific Northwest National Lab Hatem Ltaief, KAUST Satoshi Matsuoka, Tokyo Institute of Technology Celso Mendes, University of Illinois Kathryn Mohror, LLNL Frank Mueller, NCSU Wolfgang E. Nagel, ZIH, TU Dresden Bogdan Nicolae, IBM Research (Ireland) Dhabaleswar Panda, Ohio State University Prapaporn Rattanatamrong, Thammasat University Alexander Reinefeld, Zuse Institute Berlin Rolf Riesen, IBM Research Scott Schneider, IBM Research Martin Schulz, Lawrence Livermore National Laboratory Stephen Scott, Oak Ridge National Laboratory Shuaiwen Song, Pacific Northwest National Lab Wei Tang, Argonne National Laboratory Michela Taufer, University of Delaware Putchong Uthayopas, Kasetsart University Felix Wolf, German Research School for Simulation Sciences SCHEDULING AND RESOURCE MANAGEMENT Carl Albing, Cray Inc. and University of Reading Henri Bal, Vrije Universiteit Anne Benoit, ENS Lyon - LIP Luiz F. Bittencourt, University of Campinas Richard Boakes, University of Portsmouth Ivona Brandic, TU Wien Julita Corbalan, Barcelona Supercomputing Center Ewa Deelman, USC Information Sciences Institute Erik Elmroth, Ume? University Thomas Fahringer, University of Innsbruck Matthew Grove, Rackspace Emmanuel Jeannot, Inria Gideon Juve, USC Information Sciences Institute Helen Karatza, Aristotle University of Thessaloniki Alexey Lastovetsky, University College Dublin Charles Lively, DeVry University Maciej Malawski, AGH Nandini Mukherjee, Jadavpur University Jarek Nabrzyski, University of Notre Dame Thomas Naughton, ORNL George Pallis, University of Cyprus Rizos Sakellariou, University of Manchester Uwe Schwiegelshohn, TU Dortmund University Domenico Talia, University of Calabria Matthew E. Tolentino, Intel Corporation Denis Trystram, Grenoble university Kurt Vanmechelen, University of Antwerp Carlos A. Varela, Rensselaer Polytechnic Institute Vladimir Vlassov, Royal Institute of Technology (KTH) Ramin Yahyapour, University of G?ttingen Wolfgang Ziegler, Fraunhofer Institute SCAI STORAGE AND I/O SYSTEMS Andre Brinkmann, Johannes Gutenberg-Universit?t Mainz Toni Cortes, Barcelona Supercomputing Center Yafei Dai, Peking University Dan Feng, Huazhong University of Science and Technology Xubin He, Virginia Commonwealth University Hong Jiang, University of Nebraska Lincoln Xiao Qin, Auburn University Fang Wang, Huazhong University of Science and Technology Chunxiao Xing, Tsinghua University Zhenquan Xu, Wuhan University Lu Xu, Institute of Computing Technology - Chinese Academy of Sciences Zhao Zhang, Iowa State University Yifeng Zhu, University of Maine Zhichun Zhu, University of Illinois at Chicago -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud, IEEE/ACM DataCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= From yadudoc1729 at gmail.com Wed Sep 18 14:44:46 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Wed, 18 Sep 2013 14:44:46 -0500 Subject: [Swift-devel] Is the use of args within functions legal? Message-ID: Ketan reported a script failing for him on trunk alone and this is the condensed form of his script that fails consistently on trunk: func() { string FOO=arg("foo"); string BAR="hello"; tracef("FOO : %s | BAR : %s\n", FOO, BAR); } func(); This script does pass on 0.94RC3, but not on Trunk. Could someone confirm if this is valid syntax ? Reversing the order of the assignments to the following, gets the test to pass: func() { string BAR="hello"; string FOO=arg("foo"); tracef("FOO : %s | BAR : %s\n", FOO, BAR); } func(); ?Thanks,? Yadu -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Wed Sep 18 15:09:35 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 18 Sep 2013 13:09:35 -0700 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: References: Message-ID: <1379534975.27977.0.camel@echo> It is. As usual, when something fails, the exact error message is one very useful bit of information. Mihael On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > Ketan reported a script failing for him on trunk alone and this is the > condensed > form of his script that fails consistently on trunk: > > func() { > string FOO=arg("foo"); > string BAR="hello"; > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > This script does pass on 0.94RC3, but not on Trunk. Could someone confirm > if this > is valid syntax ? > > Reversing the order of the assignments to the following, gets the test to > pass: > > func() { > string BAR="hello"; > string FOO=arg("foo"); > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > ?Thanks,? > > Yadu > Ketan reported a script failing for him on trunk alone and this is the > condensed > > form of his script that fails consistently on trunk: > > func() { > string FOO=arg("foo"); > string BAR="hello"; > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > confirm if this > > is valid syntax ? > > > Reversing the order of the assignments to the following, gets the test > to pass: > > func() { > string BAR="hello"; > string FOO=arg("foo"); > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > > > ?Thanks,? > > Yadu > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From hategan at mcs.anl.gov Wed Sep 18 15:11:07 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 18 Sep 2013 13:11:07 -0700 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: References: Message-ID: <1379535067.27977.1.camel@echo> Ok, the error is due to the optional value missing. There's shouldn't be an error to begin with, but a temporary workaround is: string FOO = arg("foo", "defaultValue") On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > Ketan reported a script failing for him on trunk alone and this is the > condensed > form of his script that fails consistently on trunk: > > func() { > string FOO=arg("foo"); > string BAR="hello"; > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > This script does pass on 0.94RC3, but not on Trunk. Could someone confirm > if this > is valid syntax ? > > Reversing the order of the assignments to the following, gets the test to > pass: > > func() { > string BAR="hello"; > string FOO=arg("foo"); > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > ?Thanks,? > > Yadu > Ketan reported a script failing for him on trunk alone and this is the > condensed > > form of his script that fails consistently on trunk: > > func() { > string FOO=arg("foo"); > string BAR="hello"; > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > confirm if this > > is valid syntax ? > > > Reversing the order of the assignments to the following, gets the test > to pass: > > func() { > string BAR="hello"; > string FOO=arg("foo"); > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > > > ?Thanks,? > > Yadu > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From yadudoc1729 at gmail.com Wed Sep 18 15:17:02 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Wed, 18 Sep 2013 15:17:02 -0500 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: <1379535067.27977.1.camel@echo> References: <1379535067.27977.1.camel@echo> Message-ID: Sorry Mihael, I've pasted the error below. Why does the error go away, when the string assignment statements are reversed in order? swift args_trunk.swift -foo="helloFOO" Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog modified locally) RunID: 20130918-2014-t0fyb8m1 [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the declaration of element 'config'. Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 Execution failed: java.lang.ClassCastException: Invalid type: expected a org.griphyn.vdl.mapping.AbstractDataNode but got a org.globus.cog.karajan.analyzer.InvalidArgChannel swiftscript:arg @ args_trunk, line: 22 Caused by: java.lang.ClassCastException: Invalid type: expected a org.griphyn.vdl.mapping.AbstractDataNode but got a org.globus.cog.karajan.analyzer.InvalidArgChannel at org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) at org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) at org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) at k.thr.LWThread.run(LWThread.java:228) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) -Yadu On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan wrote: > Ok, the error is due to the optional value missing. There's shouldn't be > an error to begin with, but a temporary workaround is: > > string FOO = arg("foo", "defaultValue") > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > Ketan reported a script failing for him on trunk alone and this is the > > condensed > > form of his script that fails consistently on trunk: > > > > func() { > > string FOO=arg("foo"); > > string BAR="hello"; > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone confirm > > if this > > is valid syntax ? > > > > Reversing the order of the assignments to the following, gets the test to > > pass: > > > > func() { > > string BAR="hello"; > > string FOO=arg("foo"); > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > ?Thanks,? > > > > Yadu > > Ketan reported a script failing for him on trunk alone and this is the > > condensed > > > > form of his script that fails consistently on trunk: > > > > func() { > > string FOO=arg("foo"); > > string BAR="hello"; > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > > confirm if this > > > > is valid syntax ? > > > > > > Reversing the order of the assignments to the following, gets the test > > to pass: > > > > func() { > > string BAR="hello"; > > string FOO=arg("foo"); > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > > > > > ?Thanks,? > > > > Yadu > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Wed Sep 18 16:03:39 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 18 Sep 2013 14:03:39 -0700 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: References: <1379535067.27977.1.camel@echo> Message-ID: <1379538219.29135.1.camel@echo> On Wed, 2013-09-18 at 15:17 -0500, Yadu Nand wrote: > Sorry Mihael, I've pasted the error below. > Why does the error go away, when the string assignment statements are > reversed in order? I assume because there is a string value on the stack frame from the previous invocation that the second invocation interprets as being the second argument. Or something like that. > > swift args_trunk.swift -foo="helloFOO" > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog modified > locally) > RunID: 20130918-2014-t0fyb8m1 > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the declaration of > element 'config'. > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > Execution failed: > java.lang.ClassCastException: Invalid type: expected a > org.griphyn.vdl.mapping.AbstractDataNode but got a > org.globus.cog.karajan.analyzer.InvalidArgChannel > swiftscript:arg @ args_trunk, line: 22 > Caused by: java.lang.ClassCastException: Invalid type: expected a > org.griphyn.vdl.mapping.AbstractDataNode but got a > org.globus.cog.karajan.analyzer.InvalidArgChannel > at org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > at > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) > at k.thr.LWThread.run(LWThread.java:228) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) > > -Yadu > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan wrote: > > > Ok, the error is due to the optional value missing. There's shouldn't be > > an error to begin with, but a temporary workaround is: > > > > string FOO = arg("foo", "defaultValue") > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > Ketan reported a script failing for him on trunk alone and this is the > > > condensed > > > form of his script that fails consistently on trunk: > > > > > > func() { > > > string FOO=arg("foo"); > > > string BAR="hello"; > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone confirm > > > if this > > > is valid syntax ? > > > > > > Reversing the order of the assignments to the following, gets the test to > > > pass: > > > > > > func() { > > > string BAR="hello"; > > > string FOO=arg("foo"); > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > > > > ?Thanks,? > > > > > > Yadu > > > Ketan reported a script failing for him on trunk alone and this is the > > > condensed > > > > > > form of his script that fails consistently on trunk: > > > > > > func() { > > > string FOO=arg("foo"); > > > string BAR="hello"; > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > > > confirm if this > > > > > > is valid syntax ? > > > > > > > > > Reversing the order of the assignments to the following, gets the test > > > to pass: > > > > > > func() { > > > string BAR="hello"; > > > string FOO=arg("foo"); > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > > > > > > > > > > ?Thanks,? > > > > > > Yadu > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > Sorry Mihael, I've pasted the error below. > > Why does the error go away, when the string assignment statements are > reversed in order? > > > swift args_trunk.swift -foo="helloFOO" > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > modified locally) > RunID: 20130918-2014-t0fyb8m1 > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > declaration of element 'config'. > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > Execution failed: > java.lang.ClassCastException: Invalid type: expected a > org.griphyn.vdl.mapping.AbstractDataNode but got a > org.globus.cog.karajan.analyzer.InvalidArgChannel > swiftscript:arg @ args_trunk, line: 22 > Caused by: java.lang.ClassCastException: Invalid type: expected a > org.griphyn.vdl.mapping.AbstractDataNode but got a > org.globus.cog.karajan.analyzer.InvalidArgChannel > at > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > at > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > at > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.UParallel > $1.run(UParallel.java:68) > at k.thr.LWThread.run(LWThread.java:228) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) > > > -Yadu > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > wrote: > Ok, the error is due to the optional value missing. There's > shouldn't be > an error to begin with, but a temporary workaround is: > > string FOO = arg("foo", "defaultValue") > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > Ketan reported a script failing for him on trunk alone and > this is the > > condensed > > form of his script that fails consistently on trunk: > > > > func() { > > string FOO=arg("foo"); > > string BAR="hello"; > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > This script does pass on 0.94RC3, but not on Trunk. Could > someone confirm > > if this > > is valid syntax ? > > > > Reversing the order of the assignments to the following, > gets the test to > > pass: > > > > func() { > > string BAR="hello"; > > string FOO=arg("foo"); > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > ?Thanks,? > > > > Yadu > > > Ketan reported a script failing for him on trunk alone and > this is the > > condensed > > > > form of his script that fails consistently on trunk: > > > > func() { > > string FOO=arg("foo"); > > string BAR="hello"; > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > someone > > confirm if this > > > > is valid syntax ? > > > > > > Reversing the order of the assignments to the following, > gets the test > > to pass: > > > > func() { > > string BAR="hello"; > > string FOO=arg("foo"); > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > > > > > ?Thanks,? > > > > Yadu > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > -- > Yadu Nand B > > From hategan at mcs.anl.gov Thu Sep 19 01:50:52 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 18 Sep 2013 23:50:52 -0700 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: <1379538219.29135.1.camel@echo> References: <1379535067.27977.1.camel@echo> <1379538219.29135.1.camel@echo> Message-ID: <1379573452.18738.4.camel@echo> Should now be fixed in svn. There were two issues. One was that in trunk the value parameter to arg() was not optional. I changed that back. The other was that a missing positional parameter in karajan was not well detected at run-time. In this case it was picking up stuff that was left on the stack from other invocations. This should also be fixed. This should have been caught by the swift compiler, but it was set to treat the second parameter to arg() as optional, so it fell through the cracks. Mihael On Wed, 2013-09-18 at 14:03 -0700, Mihael Hategan wrote: > On Wed, 2013-09-18 at 15:17 -0500, Yadu Nand wrote: > > Sorry Mihael, I've pasted the error below. > > Why does the error go away, when the string assignment statements are > > reversed in order? > > I assume because there is a string value on the stack frame from the > previous invocation that the second invocation interprets as being the > second argument. Or something like that. > > > > > swift args_trunk.swift -foo="helloFOO" > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog modified > > locally) > > RunID: 20130918-2014-t0fyb8m1 > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the declaration of > > element 'config'. > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > Execution failed: > > java.lang.ClassCastException: Invalid type: expected a > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > swiftscript:arg @ args_trunk, line: 22 > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > at org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > at > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) > > at k.thr.LWThread.run(LWThread.java:228) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:722) > > > > -Yadu > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan wrote: > > > > > Ok, the error is due to the optional value missing. There's shouldn't be > > > an error to begin with, but a temporary workaround is: > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > Ketan reported a script failing for him on trunk alone and this is the > > > > condensed > > > > form of his script that fails consistently on trunk: > > > > > > > > func() { > > > > string FOO=arg("foo"); > > > > string BAR="hello"; > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone confirm > > > > if this > > > > is valid syntax ? > > > > > > > > Reversing the order of the assignments to the following, gets the test to > > > > pass: > > > > > > > > func() { > > > > string BAR="hello"; > > > > string FOO=arg("foo"); > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > ?Thanks,? > > > > > > > > Yadu > > > > Ketan reported a script failing for him on trunk alone and this is the > > > > condensed > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > func() { > > > > string FOO=arg("foo"); > > > > string BAR="hello"; > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > > > > confirm if this > > > > > > > > is valid syntax ? > > > > > > > > > > > > Reversing the order of the assignments to the following, gets the test > > > > to pass: > > > > > > > > func() { > > > > string BAR="hello"; > > > > string FOO=arg("foo"); > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > Yadu > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > Sorry Mihael, I've pasted the error below. > > > > Why does the error go away, when the string assignment statements are > > reversed in order? > > > > > > swift args_trunk.swift -foo="helloFOO" > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > > modified locally) > > RunID: 20130918-2014-t0fyb8m1 > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > declaration of element 'config'. > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > Execution failed: > > java.lang.ClassCastException: Invalid type: expected a > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > swiftscript:arg @ args_trunk, line: 22 > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > at > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > at > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at org.globus.cog.karajan.compiled.nodes.UParallel > > $1.run(UParallel.java:68) > > at k.thr.LWThread.run(LWThread.java:228) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at java.util.concurrent.ThreadPoolExecutor > > $Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:722) > > > > > > -Yadu > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > > wrote: > > Ok, the error is due to the optional value missing. There's > > shouldn't be > > an error to begin with, but a temporary workaround is: > > > > string FOO = arg("foo", "defaultValue") > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > Ketan reported a script failing for him on trunk alone and > > this is the > > > condensed > > > form of his script that fails consistently on trunk: > > > > > > func() { > > > string FOO=arg("foo"); > > > string BAR="hello"; > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > someone confirm > > > if this > > > is valid syntax ? > > > > > > Reversing the order of the assignments to the following, > > gets the test to > > > pass: > > > > > > func() { > > > string BAR="hello"; > > > string FOO=arg("foo"); > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > > > > ?Thanks,? > > > > > > Yadu > > > > > Ketan reported a script failing for him on trunk alone and > > this is the > > > condensed > > > > > > form of his script that fails consistently on trunk: > > > > > > func() { > > > string FOO=arg("foo"); > > > string BAR="hello"; > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > someone > > > confirm if this > > > > > > is valid syntax ? > > > > > > > > > Reversing the order of the assignments to the following, > > gets the test > > > to pass: > > > > > > func() { > > > string BAR="hello"; > > > string FOO=arg("foo"); > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > } > > > func(); > > > > > > > > > > > > > > > ?Thanks,? > > > > > > Yadu > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > -- > > Yadu Nand B > > > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From ketancmaheshwari at gmail.com Thu Sep 19 10:08:07 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Thu, 19 Sep 2013 10:08:07 -0500 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: <1379573452.18738.4.camel@echo> References: <1379535067.27977.1.camel@echo> <1379538219.29135.1.camel@echo> <1379573452.18738.4.camel@echo> Message-ID: Mihael, Trying the r7088 arg fix with the same script Yadu tested, I get the following: $ swift tmp.swift -foo="helloFOO" Swift trunk swift-r7088 cog-r3789 RunID: 20130919-1105-tpzasdg5 Execution failed: Missing argument 'value' export @ swift-lib.k, line: 45 The script is: $ cat tmp.swift func() { string FOO=arg("foo"); string BAR="hello"; tracef("FOO : %s | BAR : %s\n", FOO, BAR); } func(); Thanks, Ketan On Thu, Sep 19, 2013 at 1:50 AM, Mihael Hategan wrote: > Should now be fixed in svn. > > There were two issues. One was that in trunk the value parameter to > arg() was not optional. I changed that back. > > The other was that a missing positional parameter in karajan was not > well detected at run-time. In this case it was picking up stuff that was > left on the stack from other invocations. This should also be fixed. > > This should have been caught by the swift compiler, but it was set to > treat the second parameter to arg() as optional, so it fell through the > cracks. > > Mihael > > > On Wed, 2013-09-18 at 14:03 -0700, Mihael Hategan wrote: > > On Wed, 2013-09-18 at 15:17 -0500, Yadu Nand wrote: > > > Sorry Mihael, I've pasted the error below. > > > Why does the error go away, when the string assignment statements are > > > reversed in order? > > > > I assume because there is a string value on the stack frame from the > > previous invocation that the second invocation interprets as being the > > second argument. Or something like that. > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > modified > > > locally) > > > RunID: 20130918-2014-t0fyb8m1 > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > declaration of > > > element 'config'. > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > Execution failed: > > > java.lang.ClassCastException: Invalid type: expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > swiftscript:arg @ args_trunk, line: 22 > > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > at > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > at > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) > > > at k.thr.LWThread.run(LWThread.java:228) > > > at > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > at > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:722) > > > > > > -Yadu > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > wrote: > > > > > > > Ok, the error is due to the optional value missing. There's > shouldn't be > > > > an error to begin with, but a temporary workaround is: > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > Ketan reported a script failing for him on trunk alone and this is > the > > > > > condensed > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > confirm > > > > > if this > > > > > is valid syntax ? > > > > > > > > > > Reversing the order of the assignments to the following, gets the > test to > > > > > pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > Ketan reported a script failing for him on trunk alone and this is > the > > > > > condensed > > > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > > > > > confirm if this > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > Reversing the order of the assignments to the following, gets the > test > > > > > to pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > _______________________________________________ > > > > > Swift-devel mailing list > > > > > Swift-devel at ci.uchicago.edu > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > Sorry Mihael, I've pasted the error below. > > > > > > Why does the error go away, when the string assignment statements are > > > reversed in order? > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > > > modified locally) > > > RunID: 20130918-2014-t0fyb8m1 > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > > declaration of element 'config'. > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > Execution failed: > > > java.lang.ClassCastException: Invalid type: expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > swiftscript:arg @ args_trunk, line: 22 > > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > at > > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > at > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at org.globus.cog.karajan.compiled.nodes.UParallel > > > $1.run(UParallel.java:68) > > > at k.thr.LWThread.run(LWThread.java:228) > > > at > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > at java.util.concurrent.ThreadPoolExecutor > > > $Worker.run(ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > > -Yadu > > > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > > > wrote: > > > Ok, the error is due to the optional value missing. There's > > > shouldn't be > > > an error to begin with, but a temporary workaround is: > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > > > Ketan reported a script failing for him on trunk alone and > > > this is the > > > > condensed > > > > form of his script that fails consistently on trunk: > > > > > > > > func() { > > > > string FOO=arg("foo"); > > > > string BAR="hello"; > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > > someone confirm > > > > if this > > > > is valid syntax ? > > > > > > > > Reversing the order of the assignments to the following, > > > gets the test to > > > > pass: > > > > > > > > func() { > > > > string BAR="hello"; > > > > string FOO=arg("foo"); > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > ?Thanks,? > > > > > > > > Yadu > > > > > > > Ketan reported a script failing for him on trunk alone and > > > this is the > > > > condensed > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > func() { > > > > string FOO=arg("foo"); > > > > string BAR="hello"; > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > > someone > > > > confirm if this > > > > > > > > is valid syntax ? > > > > > > > > > > > > Reversing the order of the assignments to the following, > > > gets the test > > > > to pass: > > > > > > > > func() { > > > > string BAR="hello"; > > > > string FOO=arg("foo"); > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > Yadu > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > -- > > > Yadu Nand B > > > > > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Thu Sep 19 13:03:28 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 19 Sep 2013 11:03:28 -0700 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: References: <1379535067.27977.1.camel@echo> <1379538219.29135.1.camel@echo> <1379573452.18738.4.camel@echo> Message-ID: <1379613808.25070.0.camel@echo> Ah, sorry. Forgot to commit that one. Can you try now? On Thu, 2013-09-19 at 10:08 -0500, Ketan Maheshwari wrote: > Mihael, > > Trying the r7088 arg fix with the same script Yadu tested, I get the > following: > > $ swift tmp.swift -foo="helloFOO" > Swift trunk swift-r7088 cog-r3789 > RunID: 20130919-1105-tpzasdg5 > > Execution failed: > Missing argument 'value' > export @ swift-lib.k, line: 45 > > > The script is: > > $ cat tmp.swift > func() { > string FOO=arg("foo"); > string BAR="hello"; > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > > Thanks, > Ketan > > > On Thu, Sep 19, 2013 at 1:50 AM, Mihael Hategan wrote: > > > Should now be fixed in svn. > > > > There were two issues. One was that in trunk the value parameter to > > arg() was not optional. I changed that back. > > > > The other was that a missing positional parameter in karajan was not > > well detected at run-time. In this case it was picking up stuff that was > > left on the stack from other invocations. This should also be fixed. > > > > This should have been caught by the swift compiler, but it was set to > > treat the second parameter to arg() as optional, so it fell through the > > cracks. > > > > Mihael > > > > > > On Wed, 2013-09-18 at 14:03 -0700, Mihael Hategan wrote: > > > On Wed, 2013-09-18 at 15:17 -0500, Yadu Nand wrote: > > > > Sorry Mihael, I've pasted the error below. > > > > Why does the error go away, when the string assignment statements are > > > > reversed in order? > > > > > > I assume because there is a string value on the stack frame from the > > > previous invocation that the second invocation interprets as being the > > > second argument. Or something like that. > > > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > > modified > > > > locally) > > > > RunID: 20130918-2014-t0fyb8m1 > > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > declaration of > > > > element 'config'. > > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > > > Execution failed: > > > > java.lang.ClassCastException: Invalid type: expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > swiftscript:arg @ args_trunk, line: 22 > > > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > at > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > > at > > > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) > > > > at k.thr.LWThread.run(LWThread.java:228) > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > -Yadu > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > > wrote: > > > > > > > > > Ok, the error is due to the optional value missing. There's > > shouldn't be > > > > > an error to begin with, but a temporary workaround is: > > > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > > Ketan reported a script failing for him on trunk alone and this is > > the > > > > > > condensed > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > func() { > > > > > > string FOO=arg("foo"); > > > > > > string BAR="hello"; > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > > confirm > > > > > > if this > > > > > > is valid syntax ? > > > > > > > > > > > > Reversing the order of the assignments to the following, gets the > > test to > > > > > > pass: > > > > > > > > > > > > func() { > > > > > > string BAR="hello"; > > > > > > string FOO=arg("foo"); > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > Yadu > > > > > > Ketan reported a script failing for him on trunk alone and this is > > the > > > > > > condensed > > > > > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > func() { > > > > > > string FOO=arg("foo"); > > > > > > string BAR="hello"; > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could someone > > > > > > confirm if this > > > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > > > > Reversing the order of the assignments to the following, gets the > > test > > > > > > to pass: > > > > > > > > > > > > func() { > > > > > > string BAR="hello"; > > > > > > string FOO=arg("foo"); > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > Yadu > > > > > > _______________________________________________ > > > > > > Swift-devel mailing list > > > > > > Swift-devel at ci.uchicago.edu > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry Mihael, I've pasted the error below. > > > > > > > > Why does the error go away, when the string assignment statements are > > > > reversed in order? > > > > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > > > > modified locally) > > > > RunID: 20130918-2014-t0fyb8m1 > > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > > > declaration of element 'config'. > > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > > > Execution failed: > > > > java.lang.ClassCastException: Invalid type: expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > swiftscript:arg @ args_trunk, line: 22 > > > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > at > > > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > > at > > > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > > at > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at org.globus.cog.karajan.compiled.nodes.UParallel > > > > $1.run(UParallel.java:68) > > > > at k.thr.LWThread.run(LWThread.java:228) > > > > at > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at java.util.concurrent.ThreadPoolExecutor > > > > $Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > > > > > -Yadu > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > > > > wrote: > > > > Ok, the error is due to the optional value missing. There's > > > > shouldn't be > > > > an error to begin with, but a temporary workaround is: > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > > > > > Ketan reported a script failing for him on trunk alone and > > > > this is the > > > > > condensed > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > > > someone confirm > > > > > if this > > > > > is valid syntax ? > > > > > > > > > > Reversing the order of the assignments to the following, > > > > gets the test to > > > > > pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > > > > > Ketan reported a script failing for him on trunk alone and > > > > this is the > > > > > condensed > > > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > > > someone > > > > > confirm if this > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > Reversing the order of the assignments to the following, > > > > gets the test > > > > > to pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > _______________________________________________ > > > > > Swift-devel mailing list > > > > > Swift-devel at ci.uchicago.edu > > > > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Yadu Nand B > > > > > > > > > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > Mihael, > > > Trying the r7088 arg fix with the same script Yadu tested, I get the > following: > > $ swift tmp.swift -foo="helloFOO" > Swift trunk swift-r7088 cog-r3789 > RunID: 20130919-1105-tpzasdg5 > > Execution failed: > Missing argument 'value' > export @ swift-lib.k, line: 45 > > > > The script is: > > $ cat tmp.swift > func() { > string FOO=arg("foo"); > string BAR="hello"; > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > } > func(); > > > > > Thanks, > > Ketan > > > > On Thu, Sep 19, 2013 at 1:50 AM, Mihael Hategan > wrote: > Should now be fixed in svn. > > There were two issues. One was that in trunk the value > parameter to > arg() was not optional. I changed that back. > > The other was that a missing positional parameter in karajan > was not > well detected at run-time. In this case it was picking up > stuff that was > left on the stack from other invocations. This should also be > fixed. > > This should have been caught by the swift compiler, but it was > set to > treat the second parameter to arg() as optional, so it fell > through the > cracks. > > Mihael > > > On Wed, 2013-09-18 at 14:03 -0700, Mihael Hategan wrote: > > On Wed, 2013-09-18 at 15:17 -0500, Yadu Nand wrote: > > > Sorry Mihael, I've pasted the error below. > > > Why does the error go away, when the string assignment > statements are > > > reversed in order? > > > > I assume because there is a string value on the stack frame > from the > > previous invocation that the second invocation interprets as > being the > > second argument. Or something like that. > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 > (cog modified > > > locally) > > > RunID: 20130918-2014-t0fyb8m1 > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find > the declaration of > > > element 'config'. > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > Execution failed: > > > java.lang.ClassCastException: Invalid type: expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > swiftscript:arg @ args_trunk, line: 22 > > > Caused by: java.lang.ClassCastException: Invalid type: > expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > at > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > at > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > org.globus.cog.karajan.compiled.nodes.UParallel > $1.run(UParallel.java:68) > > > at k.thr.LWThread.run(LWThread.java:228) > > > at > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > at > > > java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:722) > > > > > > -Yadu > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > wrote: > > > > > > > Ok, the error is due to the optional value missing. > There's shouldn't be > > > > an error to begin with, but a temporary workaround is: > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > Ketan reported a script failing for him on trunk alone > and this is the > > > > > condensed > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. > Could someone confirm > > > > > if this > > > > > is valid syntax ? > > > > > > > > > > Reversing the order of the assignments to the > following, gets the test to > > > > > pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > Ketan reported a script failing for him on trunk alone > and this is the > > > > > condensed > > > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. > Could someone > > > > > confirm if this > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > Reversing the order of the assignments to the > following, gets the test > > > > > to pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > _______________________________________________ > > > > > Swift-devel mailing list > > > > > Swift-devel at ci.uchicago.edu > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > Sorry Mihael, I've pasted the error below. > > > > > > Why does the error go away, when the string assignment > statements are > > > reversed in order? > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 > (cog > > > modified locally) > > > RunID: 20130918-2014-t0fyb8m1 > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find > the > > > declaration of element 'config'. > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > Execution failed: > > > java.lang.ClassCastException: Invalid type: expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > swiftscript:arg @ args_trunk, line: 22 > > > Caused by: java.lang.ClassCastException: Invalid type: > expected a > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > at > > > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > at > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > at > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > at org.globus.cog.karajan.compiled.nodes.UParallel > > > $1.run(UParallel.java:68) > > > at k.thr.LWThread.run(LWThread.java:228) > > > at > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > at java.util.concurrent.ThreadPoolExecutor > > > $Worker.run(ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > > -Yadu > > > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > > > > wrote: > > > Ok, the error is due to the optional value > missing. There's > > > shouldn't be > > > an error to begin with, but a temporary workaround > is: > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand > wrote: > > > > > > > Ketan reported a script failing for him on trunk > alone and > > > this is the > > > > condensed > > > > form of his script that fails consistently on > trunk: > > > > > > > > func() { > > > > string FOO=arg("foo"); > > > > string BAR="hello"; > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > This script does pass on 0.94RC3, but not on > Trunk. Could > > > someone confirm > > > > if this > > > > is valid syntax ? > > > > > > > > Reversing the order of the assignments to the > following, > > > gets the test to > > > > pass: > > > > > > > > func() { > > > > string BAR="hello"; > > > > string FOO=arg("foo"); > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > ?Thanks,? > > > > > > > > Yadu > > > > > > > Ketan reported a script failing for him on trunk > alone and > > > this is the > > > > condensed > > > > > > > > form of his script that fails consistently on > trunk: > > > > > > > > func() { > > > > string FOO=arg("foo"); > > > > string BAR="hello"; > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > This script does pass on 0.94RC3, but not on > Trunk. Could > > > someone > > > > confirm if this > > > > > > > > is valid syntax ? > > > > > > > > > > > > Reversing the order of the assignments to the > following, > > > gets the test > > > > to pass: > > > > > > > > func() { > > > > string BAR="hello"; > > > > string FOO=arg("foo"); > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > } > > > > func(); > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > Yadu > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > -- > > > Yadu Nand B > > > > > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > -- > Ketan > > From yadudoc1729 at gmail.com Thu Sep 19 13:17:46 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Thu, 19 Sep 2013 13:17:46 -0500 Subject: [Swift-devel] Is the use of args within functions legal? In-Reply-To: <1379613808.25070.0.camel@echo> References: <1379535067.27977.1.camel@echo> <1379538219.29135.1.camel@echo> <1379573452.18738.4.camel@echo> <1379613808.25070.0.camel@echo> Message-ID: Hi Mihael, Ketan, The test is passing now, with trunk swift-r7089. -Yadu On Thu, Sep 19, 2013 at 1:03 PM, Mihael Hategan wrote: > Ah, sorry. Forgot to commit that one. Can you try now? > > On Thu, 2013-09-19 at 10:08 -0500, Ketan Maheshwari wrote: > > Mihael, > > > > Trying the r7088 arg fix with the same script Yadu tested, I get the > > following: > > > > $ swift tmp.swift -foo="helloFOO" > > Swift trunk swift-r7088 cog-r3789 > > RunID: 20130919-1105-tpzasdg5 > > > > Execution failed: > > Missing argument 'value' > > export @ swift-lib.k, line: 45 > > > > > > The script is: > > > > $ cat tmp.swift > > func() { > > string FOO=arg("foo"); > > string BAR="hello"; > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > > > Thanks, > > Ketan > > > > > > On Thu, Sep 19, 2013 at 1:50 AM, Mihael Hategan > wrote: > > > > > Should now be fixed in svn. > > > > > > There were two issues. One was that in trunk the value parameter to > > > arg() was not optional. I changed that back. > > > > > > The other was that a missing positional parameter in karajan was not > > > well detected at run-time. In this case it was picking up stuff that > was > > > left on the stack from other invocations. This should also be fixed. > > > > > > This should have been caught by the swift compiler, but it was set to > > > treat the second parameter to arg() as optional, so it fell through the > > > cracks. > > > > > > Mihael > > > > > > > > > On Wed, 2013-09-18 at 14:03 -0700, Mihael Hategan wrote: > > > > On Wed, 2013-09-18 at 15:17 -0500, Yadu Nand wrote: > > > > > Sorry Mihael, I've pasted the error below. > > > > > Why does the error go away, when the string assignment statements > are > > > > > reversed in order? > > > > > > > > I assume because there is a string value on the stack frame from the > > > > previous invocation that the second invocation interprets as being > the > > > > second argument. Or something like that. > > > > > > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > > > modified > > > > > locally) > > > > > RunID: 20130918-2014-t0fyb8m1 > > > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > > declaration of > > > > > element 'config'. > > > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > > > > > Execution failed: > > > > > java.lang.ClassCastException: Invalid type: expected a > > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > > swiftscript:arg @ args_trunk, line: 22 > > > > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > > at > > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > > > at > > > > > > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.UParallel$1.run(UParallel.java:68) > > > > > at k.thr.LWThread.run(LWThread.java:228) > > > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > > > -Yadu > > > > > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan < > hategan at mcs.anl.gov> > > > wrote: > > > > > > > > > > > Ok, the error is due to the optional value missing. There's > > > shouldn't be > > > > > > an error to begin with, but a temporary workaround is: > > > > > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > > > Ketan reported a script failing for him on trunk alone and > this is > > > the > > > > > > > condensed > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > > > func() { > > > > > > > string FOO=arg("foo"); > > > > > > > string BAR="hello"; > > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > > } > > > > > > > func(); > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > someone > > > confirm > > > > > > > if this > > > > > > > is valid syntax ? > > > > > > > > > > > > > > Reversing the order of the assignments to the following, gets > the > > > test to > > > > > > > pass: > > > > > > > > > > > > > > func() { > > > > > > > string BAR="hello"; > > > > > > > string FOO=arg("foo"); > > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > > } > > > > > > > func(); > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > > > Yadu > > > > > > > Ketan reported a script failing for him on trunk alone and > this is > > > the > > > > > > > condensed > > > > > > > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > > > func() { > > > > > > > string FOO=arg("foo"); > > > > > > > string BAR="hello"; > > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > > } > > > > > > > func(); > > > > > > > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > someone > > > > > > > confirm if this > > > > > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > > > > > > > Reversing the order of the assignments to the following, gets > the > > > test > > > > > > > to pass: > > > > > > > > > > > > > > func() { > > > > > > > string BAR="hello"; > > > > > > > string FOO=arg("foo"); > > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > > } > > > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > > > Yadu > > > > > > > _______________________________________________ > > > > > > > Swift-devel mailing list > > > > > > > Swift-devel at ci.uchicago.edu > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry Mihael, I've pasted the error below. > > > > > > > > > > Why does the error go away, when the string assignment statements > are > > > > > reversed in order? > > > > > > > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 (cog > > > > > modified locally) > > > > > RunID: 20130918-2014-t0fyb8m1 > > > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find the > > > > > declaration of element 'config'. > > > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > > > > > Execution failed: > > > > > java.lang.ClassCastException: Invalid type: expected a > > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > > swiftscript:arg @ args_trunk, line: 22 > > > > > Caused by: java.lang.ClassCastException: Invalid type: expected a > > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > > at > > > > > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > > > at > > > > > > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > > > at > > > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > > at org.globus.cog.karajan.compiled.nodes.UParallel > > > > > $1.run(UParallel.java:68) > > > > > at k.thr.LWThread.run(LWThread.java:228) > > > > > at > > > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > > at java.util.concurrent.ThreadPoolExecutor > > > > > $Worker.run(ThreadPoolExecutor.java:615) > > > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > > > > > > > > -Yadu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan < > hategan at mcs.anl.gov> > > > > > wrote: > > > > > Ok, the error is due to the optional value missing. There's > > > > > shouldn't be > > > > > an error to begin with, but a temporary workaround is: > > > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > > > > > > > Ketan reported a script failing for him on trunk alone > and > > > > > this is the > > > > > > condensed > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > func() { > > > > > > string FOO=arg("foo"); > > > > > > string BAR="hello"; > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > > > > someone confirm > > > > > > if this > > > > > > is valid syntax ? > > > > > > > > > > > > Reversing the order of the assignments to the following, > > > > > gets the test to > > > > > > pass: > > > > > > > > > > > > func() { > > > > > > string BAR="hello"; > > > > > > string FOO=arg("foo"); > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > Yadu > > > > > > > > > > > Ketan reported a script failing for him on trunk alone > and > > > > > this is the > > > > > > condensed > > > > > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > func() { > > > > > > string FOO=arg("foo"); > > > > > > string BAR="hello"; > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. Could > > > > > someone > > > > > > confirm if this > > > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > > > > Reversing the order of the assignments to the following, > > > > > gets the test > > > > > > to pass: > > > > > > > > > > > > func() { > > > > > > string BAR="hello"; > > > > > > string FOO=arg("foo"); > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > Yadu > > > > > > _______________________________________________ > > > > > > Swift-devel mailing list > > > > > > Swift-devel at ci.uchicago.edu > > > > > > > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Yadu Nand B > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > Mihael, > > > > > > Trying the r7088 arg fix with the same script Yadu tested, I get the > > following: > > > > $ swift tmp.swift -foo="helloFOO" > > Swift trunk swift-r7088 cog-r3789 > > RunID: 20130919-1105-tpzasdg5 > > > > Execution failed: > > Missing argument 'value' > > export @ swift-lib.k, line: 45 > > > > > > > > The script is: > > > > $ cat tmp.swift > > func() { > > string FOO=arg("foo"); > > string BAR="hello"; > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > } > > func(); > > > > > > > > > > Thanks, > > > > Ketan > > > > > > > > On Thu, Sep 19, 2013 at 1:50 AM, Mihael Hategan > > wrote: > > Should now be fixed in svn. > > > > There were two issues. One was that in trunk the value > > parameter to > > arg() was not optional. I changed that back. > > > > The other was that a missing positional parameter in karajan > > was not > > well detected at run-time. In this case it was picking up > > stuff that was > > left on the stack from other invocations. This should also be > > fixed. > > > > This should have been caught by the swift compiler, but it was > > set to > > treat the second parameter to arg() as optional, so it fell > > through the > > cracks. > > > > Mihael > > > > > > On Wed, 2013-09-18 at 14:03 -0700, Mihael Hategan wrote: > > > On Wed, 2013-09-18 at 15:17 -0500, Yadu Nand wrote: > > > > Sorry Mihael, I've pasted the error below. > > > > Why does the error go away, when the string assignment > > statements are > > > > reversed in order? > > > > > > I assume because there is a string value on the stack frame > > from the > > > previous invocation that the second invocation interprets as > > being the > > > second argument. Or something like that. > > > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 > > (cog modified > > > > locally) > > > > RunID: 20130918-2014-t0fyb8m1 > > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find > > the declaration of > > > > element 'config'. > > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > > > Execution failed: > > > > java.lang.ClassCastException: Invalid type: expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > swiftscript:arg @ args_trunk, line: 22 > > > > Caused by: java.lang.ClassCastException: Invalid type: > > expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > at > > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > > at > > > > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > org.globus.cog.karajan.compiled.nodes.UParallel > > $1.run(UParallel.java:68) > > > > at k.thr.LWThread.run(LWThread.java:228) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at > > > > java.util.concurrent.ThreadPoolExecutor > > $Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > -Yadu > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > > wrote: > > > > > > > > > Ok, the error is due to the optional value missing. > > There's shouldn't be > > > > > an error to begin with, but a temporary workaround is: > > > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand wrote: > > > > > > Ketan reported a script failing for him on trunk alone > > and this is the > > > > > > condensed > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > func() { > > > > > > string FOO=arg("foo"); > > > > > > string BAR="hello"; > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. > > Could someone confirm > > > > > > if this > > > > > > is valid syntax ? > > > > > > > > > > > > Reversing the order of the assignments to the > > following, gets the test to > > > > > > pass: > > > > > > > > > > > > func() { > > > > > > string BAR="hello"; > > > > > > string FOO=arg("foo"); > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > Yadu > > > > > > Ketan reported a script failing for him on trunk alone > > and this is the > > > > > > condensed > > > > > > > > > > > > form of his script that fails consistently on trunk: > > > > > > > > > > > > func() { > > > > > > string FOO=arg("foo"); > > > > > > string BAR="hello"; > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on Trunk. > > Could someone > > > > > > confirm if this > > > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > > > > Reversing the order of the assignments to the > > following, gets the test > > > > > > to pass: > > > > > > > > > > > > func() { > > > > > > string BAR="hello"; > > > > > > string FOO=arg("foo"); > > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > > } > > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > > > Yadu > > > > > > _______________________________________________ > > > > > > Swift-devel mailing list > > > > > > Swift-devel at ci.uchicago.edu > > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > Sorry Mihael, I've pasted the error below. > > > > > > > > Why does the error go away, when the string assignment > > statements are > > > > reversed in order? > > > > > > > > > > > > swift args_trunk.swift -foo="helloFOO" > > > > Swift trunk swift-r7073 (swift modified locally) cog-r3787 > > (cog > > > > modified locally) > > > > RunID: 20130918-2014-t0fyb8m1 > > > > [Error] sites.xml, line 1, col 9: cvc-elt.1: Cannot find > > the > > > > declaration of element 'config'. > > > > Progress: time: Wed, 18 Sep 2013 20:14:29 +0000 > > > > > > > > Execution failed: > > > > java.lang.ClassCastException: Invalid type: expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > swiftscript:arg @ args_trunk, line: 22 > > > > Caused by: java.lang.ClassCastException: Invalid type: > > expected a > > > > org.griphyn.vdl.mapping.AbstractDataNode but got a > > > > org.globus.cog.karajan.analyzer.InvalidArgChannel > > > > at > > > > > > > org.griphyn.vdl.karajan.lib.swiftscript.FnArg.function(FnArg.java:63) > > > > at > > > > > > > org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:108) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:22) > > > > at > > > > > > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > > > at org.globus.cog.karajan.compiled.nodes.UParallel > > > > $1.run(UParallel.java:68) > > > > at k.thr.LWThread.run(LWThread.java:228) > > > > at > > > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > at java.util.concurrent.ThreadPoolExecutor > > > > $Worker.run(ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > > > > > -Yadu > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 18, 2013 at 3:11 PM, Mihael Hategan > > > > > > wrote: > > > > Ok, the error is due to the optional value > > missing. There's > > > > shouldn't be > > > > an error to begin with, but a temporary workaround > > is: > > > > > > > > string FOO = arg("foo", "defaultValue") > > > > > > > > On Wed, 2013-09-18 at 14:44 -0500, Yadu Nand > > wrote: > > > > > > > > > Ketan reported a script failing for him on trunk > > alone and > > > > this is the > > > > > condensed > > > > > form of his script that fails consistently on > > trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > This script does pass on 0.94RC3, but not on > > Trunk. Could > > > > someone confirm > > > > > if this > > > > > is valid syntax ? > > > > > > > > > > Reversing the order of the assignments to the > > following, > > > > gets the test to > > > > > pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > > > > > Ketan reported a script failing for him on trunk > > alone and > > > > this is the > > > > > condensed > > > > > > > > > > form of his script that fails consistently on > > trunk: > > > > > > > > > > func() { > > > > > string FOO=arg("foo"); > > > > > string BAR="hello"; > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > This script does pass on 0.94RC3, but not on > > Trunk. Could > > > > someone > > > > > confirm if this > > > > > > > > > > is valid syntax ? > > > > > > > > > > > > > > > Reversing the order of the assignments to the > > following, > > > > gets the test > > > > > to pass: > > > > > > > > > > func() { > > > > > string BAR="hello"; > > > > > string FOO=arg("foo"); > > > > > tracef("FOO : %s | BAR : %s\n", FOO, BAR); > > > > > } > > > > > func(); > > > > > > > > > > > > > > > > > > > > > > > > > ?Thanks,? > > > > > > > > > > Yadu > > > > > _______________________________________________ > > > > > Swift-devel mailing list > > > > > Swift-devel at ci.uchicago.edu > > > > > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Yadu Nand B > > > > > > > > > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > -- > > Ketan > > > > > > > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketancmaheshwari at gmail.com Thu Sep 19 15:43:10 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Thu, 19 Sep 2013 15:43:10 -0500 Subject: [Swift-devel] ssh-cl how to tell coaster bootstrap to run with limited java heap space Message-ID: SDSC Gordon admins have limited java heap space to 256 on login nodes. This is enabled via the following environment variable: JAVA_TOOL_OPTIONS=-Xmx256m It seems coaster bootstrap does not like this: mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf workflow.swift Swift trunk swift-r7089 cog-r3775 RunID: 20130919-2038-jef0ns83 Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 Execution failed: Exception in matrixgen: Arguments: [2544, 3300, mA.dat] Host: gordon Directory: workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl exception @ swift-int-staging.k, line: 162 Caused by: null Caused by: org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could not submit job Caused by: org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could not start coaster service Caused by: org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task ended before registration was received. Picked up JAVA_TOOL_OPTIONS: -Xmx256m /bin/bash: line 54: 33675 Aborted /usr/java/latest/bin/java -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan -jar /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 https://128.135.112.73:50000 11836079986 Do I understand right that this is indeed the java heap space issue? or is it something else that I could work around with? Thanks for any ideas. -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Thu Sep 19 15:48:30 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 19 Sep 2013 13:48:30 -0700 Subject: [Swift-devel] ssh-cl how to tell coaster bootstrap to run with limited java heap space In-Reply-To: References: Message-ID: <1379623710.27686.0.camel@echo> Are the login nodes 64 bit by any chance? On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > SDSC Gordon admins have limited java heap space to 256 on login nodes. > > This is enabled via the following environment variable: > > JAVA_TOOL_OPTIONS=-Xmx256m > > It seems coaster bootstrap does not like this: > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > workflow.swift > Swift trunk swift-r7089 cog-r3775 > RunID: 20130919-2038-jef0ns83 > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > Execution failed: > Exception in matrixgen: > Arguments: [2544, 3300, mA.dat] > Host: gordon > Directory: workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > exception @ swift-int-staging.k, line: 162 > Caused by: null > Caused by: > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > not submit job > Caused by: > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > not start coaster service > Caused by: > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > ended before registration was received. > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > /bin/bash: line 54: 33675 Aborted /usr/java/latest/bin/java > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan -jar > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > https://128.135.112.73:50000 11836079986 > > > Do I understand right that this is indeed the java heap space issue? or is > it something else that I could work around with? Thanks for any ideas. > > SDSC Gordon admins have limited java heap space to 256 on login nodes. > > > This is enabled via the following environment variable: > > JAVA_TOOL_OPTIONS=-Xmx256m > > > It seems coaster bootstrap does not like this: > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > workflow.swift > Swift trunk swift-r7089 cog-r3775 > RunID: 20130919-2038-jef0ns83 > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > Execution failed: > Exception in matrixgen: > Arguments: [2544, 3300, mA.dat] > Host: gordon > Directory: > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > exception @ swift-int-staging.k, line: 162 > Caused by: null > Caused by: > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > Could not submit job > Caused by: > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > Could not start coaster service > Caused by: > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > Task ended before registration was received. > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > /bin/bash: line 54: 33675 > Aborted /usr/java/latest/bin/java > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan > -jar /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > https://128.135.112.73:50000 11836079986 > > > > > Do I understand right that this is indeed the java heap space issue? > or is it something else that I could work around with? Thanks for any > ideas. > > > -- > Ketan > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From ketancmaheshwari at gmail.com Thu Sep 19 15:53:00 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Thu, 19 Sep 2013 15:53:00 -0500 Subject: [Swift-devel] ssh-cl how to tell coaster bootstrap to run with limited java heap space In-Reply-To: <1379623710.27686.0.camel@echo> References: <1379623710.27686.0.camel@echo> Message-ID: Yes, 64 bit running CentOS release 6.4 On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan wrote: > Are the login nodes 64 bit by any chance? > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > SDSC Gordon admins have limited java heap space to 256 on login nodes. > > > > This is enabled via the following environment variable: > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > It seems coaster bootstrap does not like this: > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > workflow.swift > > Swift trunk swift-r7089 cog-r3775 > > RunID: 20130919-2038-jef0ns83 > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > Execution failed: > > Exception in matrixgen: > > Arguments: [2544, 3300, mA.dat] > > Host: gordon > > Directory: workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > exception @ swift-int-staging.k, line: 162 > > Caused by: null > > Caused by: > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > Could > > not submit job > > Caused by: > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > Could > > not start coaster service > > Caused by: > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > > ended before registration was received. > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > /bin/bash: line 54: 33675 Aborted > /usr/java/latest/bin/java > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan -jar > > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > https://128.135.112.73:50000 11836079986 > > > > > > Do I understand right that this is indeed the java heap space issue? or > is > > it something else that I could work around with? Thanks for any ideas. > > > > SDSC Gordon admins have limited java heap space to 256 on login nodes. > > > > > > This is enabled via the following environment variable: > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > It seems coaster bootstrap does not like this: > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > workflow.swift > > Swift trunk swift-r7089 cog-r3775 > > RunID: 20130919-2038-jef0ns83 > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > Execution failed: > > Exception in matrixgen: > > Arguments: [2544, 3300, mA.dat] > > Host: gordon > > Directory: > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > exception @ swift-int-staging.k, line: 162 > > Caused by: null > > Caused by: > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Could not submit job > > Caused by: > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Could not start coaster service > > Caused by: > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Task ended before registration was received. > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > /bin/bash: line 54: 33675 > > Aborted /usr/java/latest/bin/java > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan > > -jar /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > Do I understand right that this is indeed the java heap space issue? > > or is it something else that I could work around with? Thanks for any > > ideas. > > > > > > -- > > Ketan > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Thu Sep 19 15:56:32 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 19 Sep 2013 13:56:32 -0700 Subject: [Swift-devel] ssh-cl how to tell coaster bootstrap to run with limited java heap space In-Reply-To: References: <1379623710.27686.0.camel@echo> Message-ID: <1379624192.30114.2.camel@echo> The coaster bootstrap starts the service with 512M on 64 bit machines. It can be changed in the code: org.globus.cog.abstraction.impl.execution.coaster.bootstrap.Bootstrap.java, line 191. The other solution is to run the service on a compute node, but I don't think we ever spent enough effort on nailing that issue down. Maybe it's time to do so. On Thu, 2013-09-19 at 15:53 -0500, Ketan Maheshwari wrote: > Yes, 64 bit running CentOS release 6.4 > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan wrote: > > > Are the login nodes 64 bit by any chance? > > > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > > SDSC Gordon admins have limited java heap space to 256 on login nodes. > > > > > > This is enabled via the following environment variable: > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > It seems coaster bootstrap does not like this: > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > > workflow.swift > > > Swift trunk swift-r7089 cog-r3775 > > > RunID: 20130919-2038-jef0ns83 > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > > > Execution failed: > > > Exception in matrixgen: > > > Arguments: [2544, 3300, mA.dat] > > > Host: gordon > > > Directory: workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > exception @ swift-int-staging.k, line: 162 > > > Caused by: null > > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Could > > > not submit job > > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Could > > > not start coaster service > > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > > > ended before registration was received. > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > /bin/bash: line 54: 33675 Aborted > > /usr/java/latest/bin/java > > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan -jar > > > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > Do I understand right that this is indeed the java heap space issue? or > > is > > > it something else that I could work around with? Thanks for any ideas. > > > > > > SDSC Gordon admins have limited java heap space to 256 on login nodes. > > > > > > > > > This is enabled via the following environment variable: > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > > workflow.swift > > > Swift trunk swift-r7089 cog-r3775 > > > RunID: 20130919-2038-jef0ns83 > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > > > Execution failed: > > > Exception in matrixgen: > > > Arguments: [2544, 3300, mA.dat] > > > Host: gordon > > > Directory: > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > exception @ swift-int-staging.k, line: 162 > > > Caused by: null > > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could not submit job > > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could not start coaster service > > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Task ended before registration was received. > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > /bin/bash: line 54: 33675 > > > Aborted /usr/java/latest/bin/java > > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan > > > -jar /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > Do I understand right that this is indeed the java heap space issue? > > > or is it something else that I could work around with? Thanks for any > > > ideas. > > > > > > > > > -- > > > Ketan > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > Yes, 64 bit running CentOS release 6.4 > > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan > wrote: > Are the login nodes 64 bit by any chance? > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > SDSC Gordon admins have limited java heap space to 256 on > login nodes. > > > > This is enabled via the following environment variable: > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > It seems coaster bootstrap does not like this: > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > -config cf > > workflow.swift > > Swift trunk swift-r7089 cog-r3775 > > RunID: 20130919-2038-jef0ns83 > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > Submitting:2 > > > > Execution failed: > > Exception in matrixgen: > > Arguments: [2544, 3300, mA.dat] > > Host: gordon > > Directory: > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > exception @ swift-int-staging.k, line: 162 > > Caused by: null > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > not submit job > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > not start coaster service > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > > ended before registration was received. > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > /bin/bash: line 54: 33675 Aborted > /usr/java/latest/bin/java > > -Djava=/usr/java/latest/bin/java > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > -Duser.home=/home/ketan -jar > > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > https://128.135.112.73:50000 11836079986 > > > > > > Do I understand right that this is indeed the java heap > space issue? or is > > it something else that I could work around with? Thanks for > any ideas. > > > > SDSC Gordon admins have limited java heap space to 256 on > login nodes. > > > > > > This is enabled via the following environment variable: > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > It seems coaster bootstrap does not like this: > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > -config cf > > workflow.swift > > Swift trunk swift-r7089 cog-r3775 > > RunID: 20130919-2038-jef0ns83 > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > Submitting:2 > > > > Execution failed: > > Exception in matrixgen: > > Arguments: [2544, 3300, mA.dat] > > Host: gordon > > Directory: > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > exception @ swift-int-staging.k, line: 162 > > Caused by: null > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Could not submit job > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Could not start coaster service > > Caused by: > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Task ended before registration was received. > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > /bin/bash: line 54: 33675 > > Aborted /usr/java/latest/bin/java > > -Djava=/usr/java/latest/bin/java > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > -Duser.home=/home/ketan > > -jar /tmp/bootstrap.RWIqFu > http://swift.rcc.uchicago.edu:50001 > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > Do I understand right that this is indeed the java heap > space issue? > > or is it something else that I could work around with? > Thanks for any > > ideas. > > > > > > -- > > Ketan > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > -- > Ketan > > From ketancmaheshwari at gmail.com Thu Sep 19 17:10:16 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Thu, 19 Sep 2013 17:10:16 -0500 Subject: [Swift-devel] ssh-cl how to tell coaster bootstrap to run with limited java heap space In-Reply-To: <1379624192.30114.2.camel@echo> References: <1379623710.27686.0.camel@echo> <1379624192.30114.2.camel@echo> Message-ID: Changing the line to 256M and rebuilding Swift somehow did not have any effect on the run. I agree, an ability to run coaster service on compute node will help. XSEDE Stampede also has the same and other constraints in place. On Thu, Sep 19, 2013 at 3:56 PM, Mihael Hategan wrote: > The coaster bootstrap starts the service with 512M on 64 bit machines. > > It can be changed in the code: > org.globus.cog.abstraction.impl.execution.coaster.bootstrap.Bootstrap.java, > line 191. The other solution is to run the service on a compute node, but I > don't think we ever spent enough effort on nailing that issue down. Maybe > it's time to do so. > > On Thu, 2013-09-19 at 15:53 -0500, Ketan Maheshwari wrote: > > Yes, 64 bit running CentOS release 6.4 > > > > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan > wrote: > > > > > Are the login nodes 64 bit by any chance? > > > > > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > > > SDSC Gordon admins have limited java heap space to 256 on login > nodes. > > > > > > > > This is enabled via the following environment variable: > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > > > workflow.swift > > > > Swift trunk swift-r7089 cog-r3775 > > > > RunID: 20130919-2038-jef0ns83 > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > > > > > Execution failed: > > > > Exception in matrixgen: > > > > Arguments: [2544, 3300, mA.dat] > > > > Host: gordon > > > > Directory: > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > exception @ swift-int-staging.k, line: 162 > > > > Caused by: null > > > > Caused by: > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could > > > > not submit job > > > > Caused by: > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could > > > > not start coaster service > > > > Caused by: > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > Task > > > > ended before registration was received. > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > /bin/bash: line 54: 33675 Aborted > > > /usr/java/latest/bin/java > > > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan -jar > > > > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > Do I understand right that this is indeed the java heap space issue? > or > > > is > > > > it something else that I could work around with? Thanks for any > ideas. > > > > > > > > SDSC Gordon admins have limited java heap space to 256 on login > nodes. > > > > > > > > > > > > This is enabled via the following environment variable: > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > > > workflow.swift > > > > Swift trunk swift-r7089 cog-r3775 > > > > RunID: 20130919-2038-jef0ns83 > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > > > > > Execution failed: > > > > Exception in matrixgen: > > > > Arguments: [2544, 3300, mA.dat] > > > > Host: gordon > > > > Directory: > > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > exception @ swift-int-staging.k, line: 162 > > > > Caused by: null > > > > Caused by: > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could not submit job > > > > Caused by: > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could not start coaster service > > > > Caused by: > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Task ended before registration was received. > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > /bin/bash: line 54: 33675 > > > > Aborted /usr/java/latest/bin/java > > > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan > > > > -jar /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > > > > > > Do I understand right that this is indeed the java heap space issue? > > > > or is it something else that I could work around with? Thanks for any > > > > ideas. > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > Yes, 64 bit running CentOS release 6.4 > > > > > > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan > > wrote: > > Are the login nodes 64 bit by any chance? > > > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > > SDSC Gordon admins have limited java heap space to 256 on > > login nodes. > > > > > > This is enabled via the following environment variable: > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > It seems coaster bootstrap does not like this: > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > > -config cf > > > workflow.swift > > > Swift trunk swift-r7089 cog-r3775 > > > RunID: 20130919-2038-jef0ns83 > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > > Submitting:2 > > > > > > Execution failed: > > > Exception in matrixgen: > > > Arguments: [2544, 3300, mA.dat] > > > Host: gordon > > > Directory: > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > exception @ swift-int-staging.k, line: 162 > > > Caused by: null > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > > not submit job > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > > not start coaster service > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > > > ended before registration was received. > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > /bin/bash: line 54: 33675 Aborted > > /usr/java/latest/bin/java > > > -Djava=/usr/java/latest/bin/java > > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > > -Duser.home=/home/ketan -jar > > > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > Do I understand right that this is indeed the java heap > > space issue? or is > > > it something else that I could work around with? Thanks for > > any ideas. > > > > > > SDSC Gordon admins have limited java heap space to 256 on > > login nodes. > > > > > > > > > This is enabled via the following environment variable: > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > > -config cf > > > workflow.swift > > > Swift trunk swift-r7089 cog-r3775 > > > RunID: 20130919-2038-jef0ns83 > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > > Submitting:2 > > > > > > Execution failed: > > > Exception in matrixgen: > > > Arguments: [2544, 3300, mA.dat] > > > Host: gordon > > > Directory: > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > exception @ swift-int-staging.k, line: 162 > > > Caused by: null > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could not submit job > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could not start coaster service > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Task ended before registration was received. > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > /bin/bash: line 54: 33675 > > > Aborted /usr/java/latest/bin/java > > > -Djava=/usr/java/latest/bin/java > > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > > -Duser.home=/home/ketan > > > -jar /tmp/bootstrap.RWIqFu > > http://swift.rcc.uchicago.edu:50001 > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > Do I understand right that this is indeed the java heap > > space issue? > > > or is it something else that I could work around with? > > Thanks for any > > > ideas. > > > > > > > > > -- > > > Ketan > > > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > -- > > Ketan > > > > > > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Thu Sep 19 17:21:14 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 19 Sep 2013 15:21:14 -0700 Subject: [Swift-devel] ssh-cl how to tell coaster bootstrap to run with limited java heap space In-Reply-To: References: <1379623710.27686.0.camel@echo> <1379624192.30114.2.camel@echo> Message-ID: <1379629274.31127.1.camel@echo> Well, if the limit is 256M and only the heap takes 256M, I don't think the process will fit in there. There's more memory in the JVM than just the heap. I suggest you try lower values. However, it's likely that the coaster service will then run out of heap space. Mihael On Thu, 2013-09-19 at 17:10 -0500, Ketan Maheshwari wrote: > Changing the line to 256M and rebuilding Swift somehow did not have any > effect on the run. I agree, an ability to run coaster service on compute > node will help. XSEDE Stampede also has the same and other constraints in > place. > > > On Thu, Sep 19, 2013 at 3:56 PM, Mihael Hategan wrote: > > > The coaster bootstrap starts the service with 512M on 64 bit machines. > > > > It can be changed in the code: > > org.globus.cog.abstraction.impl.execution.coaster.bootstrap.Bootstrap.java, > > line 191. The other solution is to run the service on a compute node, but I > > don't think we ever spent enough effort on nailing that issue down. Maybe > > it's time to do so. > > > > On Thu, 2013-09-19 at 15:53 -0500, Ketan Maheshwari wrote: > > > Yes, 64 bit running CentOS release 6.4 > > > > > > > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan > > wrote: > > > > > > > Are the login nodes 64 bit by any chance? > > > > > > > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > > > > SDSC Gordon admins have limited java heap space to 256 on login > > nodes. > > > > > > > > > > This is enabled via the following environment variable: > > > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > > > > workflow.swift > > > > > Swift trunk swift-r7089 cog-r3775 > > > > > RunID: 20130919-2038-jef0ns83 > > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > > > > > > > Execution failed: > > > > > Exception in matrixgen: > > > > > Arguments: [2544, 3300, mA.dat] > > > > > Host: gordon > > > > > Directory: > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > > exception @ swift-int-staging.k, line: 162 > > > > > Caused by: null > > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could > > > > > not submit job > > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could > > > > > not start coaster service > > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > Task > > > > > ended before registration was received. > > > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > > /bin/bash: line 54: 33675 Aborted > > > > /usr/java/latest/bin/java > > > > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan -jar > > > > > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > Do I understand right that this is indeed the java heap space issue? > > or > > > > is > > > > > it something else that I could work around with? Thanks for any > > ideas. > > > > > > > > > > SDSC Gordon admins have limited java heap space to 256 on login > > nodes. > > > > > > > > > > > > > > > This is enabled via the following environment variable: > > > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps -config cf > > > > > workflow.swift > > > > > Swift trunk swift-r7089 cog-r3775 > > > > > RunID: 20130919-2038-jef0ns83 > > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 Submitting:2 > > > > > > > > > > Execution failed: > > > > > Exception in matrixgen: > > > > > Arguments: [2544, 3300, mA.dat] > > > > > Host: gordon > > > > > Directory: > > > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > > exception @ swift-int-staging.k, line: 162 > > > > > Caused by: null > > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > > Could not submit job > > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > > Could not start coaster service > > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > > Task ended before registration was received. > > > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > > /bin/bash: line 54: 33675 > > > > > Aborted /usr/java/latest/bin/java > > > > > -Djava=/usr/java/latest/bin/java -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org -Duser.home=/home/ketan > > > > > -jar /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > > > > > > > > > > > Do I understand right that this is indeed the java heap space issue? > > > > > or is it something else that I could work around with? Thanks for any > > > > > ideas. > > > > > > > > > > > > > > > -- > > > > > Ketan > > > > > > > > > > > > > > > _______________________________________________ > > > > > Swift-devel mailing list > > > > > Swift-devel at ci.uchicago.edu > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > > > > Yes, 64 bit running CentOS release 6.4 > > > > > > > > > > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan > > > wrote: > > > Are the login nodes 64 bit by any chance? > > > > > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > > > SDSC Gordon admins have limited java heap space to 256 on > > > login nodes. > > > > > > > > This is enabled via the following environment variable: > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > > > -config cf > > > > workflow.swift > > > > Swift trunk swift-r7089 cog-r3775 > > > > RunID: 20130919-2038-jef0ns83 > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > > > Submitting:2 > > > > > > > > Execution failed: > > > > Exception in matrixgen: > > > > Arguments: [2544, 3300, mA.dat] > > > > Host: gordon > > > > Directory: > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > exception @ swift-int-staging.k, line: 162 > > > > Caused by: null > > > > Caused by: > > > > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > > > not submit job > > > > Caused by: > > > > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > > > not start coaster service > > > > Caused by: > > > > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > > > > ended before registration was received. > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > /bin/bash: line 54: 33675 Aborted > > > /usr/java/latest/bin/java > > > > -Djava=/usr/java/latest/bin/java > > > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > > > -Duser.home=/home/ketan -jar > > > > /tmp/bootstrap.RWIqFu http://swift.rcc.uchicago.edu:50001 > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > Do I understand right that this is indeed the java heap > > > space issue? or is > > > > it something else that I could work around with? Thanks for > > > any ideas. > > > > > > > > SDSC Gordon admins have limited java heap space to 256 on > > > login nodes. > > > > > > > > > > > > This is enabled via the following environment variable: > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > > > -config cf > > > > workflow.swift > > > > Swift trunk swift-r7089 cog-r3775 > > > > RunID: 20130919-2038-jef0ns83 > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > > > Submitting:2 > > > > > > > > Execution failed: > > > > Exception in matrixgen: > > > > Arguments: [2544, 3300, mA.dat] > > > > Host: gordon > > > > Directory: > > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > exception @ swift-int-staging.k, line: 162 > > > > Caused by: null > > > > Caused by: > > > > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could not submit job > > > > Caused by: > > > > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could not start coaster service > > > > Caused by: > > > > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Task ended before registration was received. > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > /bin/bash: line 54: 33675 > > > > Aborted /usr/java/latest/bin/java > > > > -Djava=/usr/java/latest/bin/java > > > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > > > -Duser.home=/home/ketan > > > > -jar /tmp/bootstrap.RWIqFu > > > http://swift.rcc.uchicago.edu:50001 > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > > > > > > Do I understand right that this is indeed the java heap > > > space issue? > > > > or is it something else that I could work around with? > > > Thanks for any > > > > ideas. > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > > > > -- > > > Ketan > > > > > > > > > > > > > > > Changing the line to 256M and rebuilding Swift somehow did not have > any effect on the run. I agree, an ability to run coaster service on > compute node will help. XSEDE Stampede also has the same and other > constraints in place. > > > > On Thu, Sep 19, 2013 at 3:56 PM, Mihael Hategan > wrote: > The coaster bootstrap starts the service with 512M on 64 bit > machines. > > It can be changed in the code: > org.globus.cog.abstraction.impl.execution.coaster.bootstrap.Bootstrap.java, line 191. The other solution is to run the service on a compute node, but I don't think we ever spent enough effort on nailing that issue down. Maybe it's time to do so. > > On Thu, 2013-09-19 at 15:53 -0500, Ketan Maheshwari wrote: > > Yes, 64 bit running CentOS release 6.4 > > > > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan > wrote: > > > > > Are the login nodes 64 bit by any chance? > > > > > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari wrote: > > > > SDSC Gordon admins have limited java heap space to 256 > on login nodes. > > > > > > > > This is enabled via the following environment variable: > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > -config cf > > > > workflow.swift > > > > Swift trunk swift-r7089 cog-r3775 > > > > RunID: 20130919-2038-jef0ns83 > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > Submitting:2 > > > > > > > > Execution failed: > > > > Exception in matrixgen: > > > > Arguments: [2544, 3300, mA.dat] > > > > Host: gordon > > > > Directory: > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > exception @ swift-int-staging.k, line: 162 > > > > Caused by: null > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could > > > > not submit job > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could > > > > not start coaster service > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > > > > ended before registration was received. > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > /bin/bash: line 54: 33675 Aborted > > > /usr/java/latest/bin/java > > > > -Djava=/usr/java/latest/bin/java > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > -Duser.home=/home/ketan -jar > > > > /tmp/bootstrap.RWIqFu > http://swift.rcc.uchicago.edu:50001 > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > Do I understand right that this is indeed the java heap > space issue? or > > > is > > > > it something else that I could work around with? Thanks > for any ideas. > > > > > > > > SDSC Gordon admins have limited java heap space to 256 > on login nodes. > > > > > > > > > > > > This is enabled via the following environment variable: > > > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file apps > -config cf > > > > workflow.swift > > > > Swift trunk swift-r7089 cog-r3775 > > > > RunID: 20130919-2038-jef0ns83 > > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > Submitting:2 > > > > > > > > Execution failed: > > > > Exception in matrixgen: > > > > Arguments: [2544, 3300, mA.dat] > > > > Host: gordon > > > > Directory: > > > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > > exception @ swift-int-staging.k, line: 162 > > > > Caused by: null > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could not submit job > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Could not start coaster service > > > > Caused by: > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > > Task ended before registration was received. > > > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > > /bin/bash: line 54: 33675 > > > > Aborted /usr/java/latest/bin/java > > > > -Djava=/usr/java/latest/bin/java > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > -Duser.home=/home/ketan > > > > -jar /tmp/bootstrap.RWIqFu > http://swift.rcc.uchicago.edu:50001 > > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > > > > > > Do I understand right that this is indeed the java heap > space issue? > > > > or is it something else that I could work around with? > Thanks for any > > > > ideas. > > > > > > > > > > > > -- > > > > Ketan > > > > > > > > > > > > _______________________________________________ > > > > Swift-devel mailing list > > > > Swift-devel at ci.uchicago.edu > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > > > > Yes, 64 bit running CentOS release 6.4 > > > > > > > > On Thu, Sep 19, 2013 at 3:48 PM, Mihael Hategan > > > wrote: > > Are the login nodes 64 bit by any chance? > > > > On Thu, 2013-09-19 at 15:43 -0500, Ketan Maheshwari > wrote: > > > SDSC Gordon admins have limited java heap space to > 256 on > > login nodes. > > > > > > This is enabled via the following environment > variable: > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > It seems coaster bootstrap does not like this: > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file > apps > > -config cf > > > workflow.swift > > > Swift trunk swift-r7089 cog-r3775 > > > RunID: 20130919-2038-jef0ns83 > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > > Submitting:2 > > > > > > Execution failed: > > > Exception in matrixgen: > > > Arguments: [2544, 3300, mA.dat] > > > Host: gordon > > > Directory: > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > exception @ swift-int-staging.k, line: 162 > > > Caused by: null > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > > not submit job > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Could > > > not start coaster service > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: Task > > > ended before registration was received. > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > /bin/bash: line 54: 33675 Aborted > > /usr/java/latest/bin/java > > > -Djava=/usr/java/latest/bin/java > > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > > -Duser.home=/home/ketan -jar > > > /tmp/bootstrap.RWIqFu > http://swift.rcc.uchicago.edu:50001 > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > Do I understand right that this is indeed the java > heap > > space issue? or is > > > it something else that I could work around with? > Thanks for > > any ideas. > > > > > > SDSC Gordon admins have limited java heap space to > 256 on > > login nodes. > > > > > > > > > This is enabled via the following environment > variable: > > > > > > JAVA_TOOL_OPTIONS=-Xmx256m > > > > > > > > > It seems coaster bootstrap does not like this: > > > > > > mdw$ swift -sites.file sites.gordon.xml -tc.file > apps > > -config cf > > > workflow.swift > > > Swift trunk swift-r7089 cog-r3775 > > > RunID: 20130919-2038-jef0ns83 > > > Progress: time: Thu, 19 Sep 2013 20:38:42 +0000 > > > Progress: time: Thu, 19 Sep 2013 20:38:43 +0000 > > Submitting:2 > > > > > > Execution failed: > > > Exception in matrixgen: > > > Arguments: [2544, 3300, mA.dat] > > > Host: gordon > > > Directory: > > > > workflow-20130919-2038-jef0ns83/jobs/a/matrixgen-a1rnkhfl > > > exception @ swift-int-staging.k, line: 162 > > > Caused by: null > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could not submit job > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Could not start coaster service > > > Caused by: > > > > > > org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: > > > Task ended before registration was received. > > > > > > Picked up JAVA_TOOL_OPTIONS: -Xmx256m > > > /bin/bash: line 54: 33675 > > > Aborted /usr/java/latest/bin/java > > > -Djava=/usr/java/latest/bin/java > > -DGLOBUS_TCP_PORT_RANGE=50000,51000 > > > > > > -DX509_USER_PROXY=/home/ketan/.globus/sshproxy-316831905-1379663604 > > > > > > -DX509_CERT_DIR=/home/ketan/.globus/sshCAcert-316831905-1379663604.pem > > > -DGLOBUS_HOSTNAME=gordon.sdsc.xsede.org > > -Duser.home=/home/ketan > > > -jar /tmp/bootstrap.RWIqFu > > http://swift.rcc.uchicago.edu:50001 > > > https://128.135.112.73:50000 11836079986 > > > > > > > > > > > > > > > Do I understand right that this is indeed the java > heap > > space issue? > > > or is it something else that I could work around > with? > > Thanks for any > > > ideas. > > > > > > > > > -- > > > Ketan > > > > > > > > > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > > > > > -- > > Ketan > > > > > > > > > > > -- > Ketan > > From ketancmaheshwari at gmail.com Fri Sep 20 13:50:47 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Fri, 20 Sep 2013 13:50:47 -0500 Subject: [Swift-devel] output file not created in _concurrent Message-ID: Hi, I was testing a dummy Swift script with a user with file arrays and noticed that no output files are created in the _concurrent dir after a Swift run completed successfully. Here is the Swift script: $ cat outfile.swift type file; app (file out) A (string in1){ dummyA in1 stdout=@out; } file outfileA[]; foreach i in [0:9]{ outfileA[i] = A ("astring"); } The executable simply sleep a bit and prints date: $ cat da.sh #!/bin/bash echo "========" date sleep 5 echo "========" I do see outfiles being created when I replace the file outfileA[] with: file outfileA[]; This is run under Swift 0.94 swift-r7091 cog-r3789 I think this is a bug but wanted to be sure before filing one. Thanks, -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketancmaheshwari at gmail.com Fri Sep 20 22:23:47 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Fri, 20 Sep 2013 22:23:47 -0500 Subject: [Swift-devel] maxtime and maxwalltime Message-ID: Hi, I am trying to run a small test job on Beagle. I set the maxtime and maxwalltime values in the sites file as follows: 610 00:10:00 At runtime, I see Swift complaining that job wall time is greater than maxTime as follows: Swift 0.94.1 RC2 swift-r6895 cog-r3765 RunID: 20130920-2215-oi4vhbh6 Progress: time: Fri, 20 Sep 2013 22:15:56 -0500 Execution failed: Exception in cat: Arguments: [data.txt] Host: beagle Directory: catsn-20130920-2215-oi4vhbh6/jobs/i/cat-itr4ojfl Caused by: Job walltime > maxTime - reserve (00:10:00 > 00:09:10) cat, catsn.swift, line 11 Here, I do not understand where does the number 00:09:10 comes from? I set maxtime at 610 (00:10:10). Am I missing something here? Thanks, -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Fri Sep 20 22:49:19 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Fri, 20 Sep 2013 20:49:19 -0700 Subject: [Swift-devel] maxtime and maxwalltime In-Reply-To: References: Message-ID: <1379735359.17935.1.camel@echo> You are missing the reserve, which is a buffer of (by default) 60 seconds at the end. So 00:10:10 - 00:01:00 = 00:09:10 Mihael On Fri, 2013-09-20 at 22:23 -0500, Ketan Maheshwari wrote: > Hi, > > I am trying to run a small test job on Beagle. I set the maxtime and > maxwalltime values in the sites file as follows: > > 610 > 00:10:00 > > At runtime, I see Swift complaining that job wall time is greater than > maxTime as follows: > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > RunID: 20130920-2215-oi4vhbh6 > Progress: time: Fri, 20 Sep 2013 22:15:56 -0500 > Execution failed: > Exception in cat: > Arguments: [data.txt] > Host: beagle > Directory: catsn-20130920-2215-oi4vhbh6/jobs/i/cat-itr4ojfl > > Caused by: > Job walltime > maxTime - reserve (00:10:00 > 00:09:10) > cat, catsn.swift, line 11 > > Here, I do not understand where does the number 00:09:10 comes from? I set > maxtime at 610 (00:10:10). Am I missing something here? > > Thanks, > Hi, > > > I am trying to run a small test job on Beagle. I set the maxtime and > maxwalltime values in the sites file as follows: > > 610 > 00:10:00 > > > At runtime, I see Swift complaining that job wall time is greater than > maxTime as follows: > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > RunID: 20130920-2215-oi4vhbh6 > Progress: time: Fri, 20 Sep 2013 22:15:56 -0500 > Execution failed: > Exception in cat: > Arguments: [data.txt] > Host: beagle > Directory: catsn-20130920-2215-oi4vhbh6/jobs/i/cat-itr4ojfl > > Caused by: > Job walltime > maxTime - reserve (00:10:00 > 00:09:10) > cat, catsn.swift, line 11 > > > Here, I do not understand where does the number 00:09:10 comes from? I > set maxtime at 610 (00:10:10). Am I missing something here? > > > > Thanks, > > -- > Ketan > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From ketancmaheshwari at gmail.com Fri Sep 20 23:02:47 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Fri, 20 Sep 2013 23:02:47 -0500 Subject: [Swift-devel] maxtime and maxwalltime In-Reply-To: <1379735359.17935.1.camel@echo> References: <1379735359.17935.1.camel@echo> Message-ID: Ahh ok .. I thought that is a hyphen sign. Thanks. On Fri, Sep 20, 2013 at 10:49 PM, Mihael Hategan wrote: > You are missing the reserve, which is a buffer of (by default) 60 > seconds at the end. > > So 00:10:10 - 00:01:00 = 00:09:10 > > Mihael > > On Fri, 2013-09-20 at 22:23 -0500, Ketan Maheshwari wrote: > > Hi, > > > > I am trying to run a small test job on Beagle. I set the maxtime and > > maxwalltime values in the sites file as follows: > > > > 610 > > 00:10:00 > > > > At runtime, I see Swift complaining that job wall time is greater than > > maxTime as follows: > > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > RunID: 20130920-2215-oi4vhbh6 > > Progress: time: Fri, 20 Sep 2013 22:15:56 -0500 > > Execution failed: > > Exception in cat: > > Arguments: [data.txt] > > Host: beagle > > Directory: catsn-20130920-2215-oi4vhbh6/jobs/i/cat-itr4ojfl > > > > Caused by: > > Job walltime > maxTime - reserve (00:10:00 > 00:09:10) > > cat, catsn.swift, line 11 > > > > Here, I do not understand where does the number 00:09:10 comes from? I > set > > maxtime at 610 (00:10:10). Am I missing something here? > > > > Thanks, > > Hi, > > > > > > I am trying to run a small test job on Beagle. I set the maxtime and > > maxwalltime values in the sites file as follows: > > > > 610 > > 00:10:00 > > > > > > At runtime, I see Swift complaining that job wall time is greater than > > maxTime as follows: > > > > Swift 0.94.1 RC2 swift-r6895 cog-r3765 > > > > RunID: 20130920-2215-oi4vhbh6 > > Progress: time: Fri, 20 Sep 2013 22:15:56 -0500 > > Execution failed: > > Exception in cat: > > Arguments: [data.txt] > > Host: beagle > > Directory: catsn-20130920-2215-oi4vhbh6/jobs/i/cat-itr4ojfl > > > > Caused by: > > Job walltime > maxTime - reserve (00:10:00 > 00:09:10) > > cat, catsn.swift, line 11 > > > > > > Here, I do not understand where does the number 00:09:10 comes from? I > > set maxtime at 610 (00:10:10). Am I missing something here? > > > > > > > > Thanks, > > > > -- > > Ketan > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketancmaheshwari at gmail.com Sat Sep 21 19:50:16 2013 From: ketancmaheshwari at gmail.com (Ketan Maheshwari) Date: Sat, 21 Sep 2013 19:50:16 -0500 Subject: [Swift-devel] coaster is resubmitting LRM job for every burst of jobs Message-ID: Hi, I am running a local:pbs swift-coaster setup on LCRC Blues with the jobspernode value of 2 and walltime of about 70 minutes. There are about 120 jobs in this application. What I observe is that PBS job is being killed and resubmitted every time a new burst of 2 jobs are run. With coasters, my assumption is that the LRM job will continue to run as long as there are more Swift jobs in the wing which seems to be not happening. Swift is trunk: swift-r7079 cog-r3778 Config files are as follows: $ cat sites.blues.xml 4300 01:10:00 2 1 2 1 1 0.01 10000 /home/ketan/swift.workdir $ cat cf wrapperlog.always.transfer=false sitedir.keep=true file.gc.enabled=false status.mode=provider execution.retries=0 lazy.errors=false use.provider.staging=true provider.staging.pin.swiftfiles=true use.wrapper.staging=false This looks like connected to a recent issue where just one round of jobs would get submitted. Thanks, -- Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Sat Sep 21 20:10:26 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sat, 21 Sep 2013 18:10:26 -0700 Subject: [Swift-devel] coaster is resubmitting LRM job for every burst of jobs In-Reply-To: References: Message-ID: <1379812226.10230.1.camel@echo> The service shuts down unused blocks. If you throttle is such that only two jobs make it to the service at a time, there might be a delay sufficiently large to cause the blocks to shut down. Increase you throttle and let the service have a larger pool of jobs to run if you want to avoid that. Mihael On Sat, 2013-09-21 at 19:50 -0500, Ketan Maheshwari wrote: > Hi, > > I am running a local:pbs swift-coaster setup on LCRC Blues with the > jobspernode value of 2 and walltime of about 70 minutes. > > There are about 120 jobs in this application. > > What I observe is that PBS job is being killed and resubmitted every time a > new burst of 2 jobs are run. > > With coasters, my assumption is that the LRM job will continue to run as > long as there are more Swift jobs in the wing which seems to be not > happening. > > Swift is trunk: swift-r7079 cog-r3778 > > Config files are as follows: > $ cat sites.blues.xml > > > > > > 4300 > 01:10:00 > 2 > 1 > 2 > 1 > 1 > 0.01 > 10000 > /home/ketan/swift.workdir > > > > $ cat cf > wrapperlog.always.transfer=false > sitedir.keep=true > file.gc.enabled=false > status.mode=provider > execution.retries=0 > lazy.errors=false > use.provider.staging=true > provider.staging.pin.swiftfiles=true > use.wrapper.staging=false > > This looks like connected to a recent issue where just one round of jobs > would get submitted. > > Thanks, > Hi, > > > I am running a local:pbs swift-coaster setup on LCRC Blues with the > jobspernode value of 2 and walltime of about 70 minutes. > > > There are about 120 jobs in this application. > > > What I observe is that PBS job is being killed and resubmitted every > time a new burst of 2 jobs are run. > > > With coasters, my assumption is that the LRM job will continue to run > as long as there are more Swift jobs in the wing which seems to be not > happening. > > > Swift is trunk: swift-r7079 cog-r3778 > > > Config files are as follows: > $ cat sites.blues.xml > > > > > > 4300 > 01:10:00 > 2 > 1 > 2 > 1 > 1 > 0.01 > 10000 > /home/ketan/swift.workdir > > > > $ cat cf > wrapperlog.always.transfer=false > sitedir.keep=true > file.gc.enabled=false > status.mode=provider > execution.retries=0 > lazy.errors=false > use.provider.staging=true > provider.staging.pin.swiftfiles=true > use.wrapper.staging=false > > > This looks like connected to a recent issue where just one round of > jobs would get submitted. > > > Thanks, > > -- > Ketan > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From marialemos72 at gmail.com Mon Sep 30 17:04:12 2013 From: marialemos72 at gmail.com (Maria Lemos) Date: Mon, 30 Sep 2013 23:04:12 +0100 Subject: [Swift-devel] CFP: WorldCIST'14 - World Conference on IST, at Madeira Island Message-ID: <20130930220419.7BFF77CC4A5@mailrelay.anl.gov> Apologies if you are receiving this mail more than once... ********************************************************************************** WorldCIST'14 The 2014 World Conference on Information Systems and Technologies April 15 - 18, Madeira Island, Portugal http://www.aisti.eu/worldcist14/ ********************************************************************************** The 2014 World Conference on Information Systems and Technologies (WorldCIST'14: http://www.aisti.eu/worldcist14) is a global forum for researchers and practitioners to present and discuss the most recent innovations, trends, results, experiences and concerns in the several perspectives of Information Systems and Technologies. We are pleased to invite you to submit your papers to WorldCISTI'14. All submissions will be reviewed on the basis of relevance, originality, importance and clarity. THEMES Submitted papers should be related with one or more of the main themes proposed for the Conference: A) Information and Knowledge Management (IKM); B) Organizational Models and Information Systems (OMIS); C) Intelligent and Decision Support Systems (IDSS); D) Software Systems, Architectures, Applications and Tools (SSAAT); E) Computer Networks, Mobility and Pervasive Systems (CNMPS); F) Human-Computer Interaction (HCI); G) Health Informatics (HIS); H) Information Technologies in Education (ITE). TYPES OF SUBMISSIONS AND DECISIONS Four types of papers can be submitted: Full paper: Finished or consolidated R&D works, to be included in one of the Conference themes. These papers are assigned a 10-page limit. Short paper: Ongoing works with relevant preliminary results, open to discussion. These papers are assigned a 7-page limit. Poster paper: Initial work with relevant ideas, open to discussion. These papers are assigned to a 4-page limit. Company paper: Companies' papers that show practical experience, R & D, tools, etc., focused on some topics of the conference. These papers are assigned to a 4-page limit. Submitted papers must comply with the format of Advances in Intelligent Systems and Computing Series (see Instructions for Authors at Springer Website or download a DOC example) be written in English, must not have been published before, not be under review for any other conference or publication and not include any information leading to the authors? identification. Therefore, the authors? names, affiliations and bibliographic references should not be included in the version for evaluation by the Program Committee. This information should only be included in the camera-ready version, saved in Word or Latex format and also in PDF format. These files must be accompanied by the Consent to Publication form filled out, in a ZIP file, and uploaded at the conference management system. All papers will be subjected to a ?double-blind review? by at least two members of the Program Committee. Based on Program Committee evaluation, a paper can be rejected or accepted by the Conference Chairs. In the later case, it can be accepted as the type originally submitted or as another type. Thus, full papers can be accepted as short papers or poster papers only. Similarly, short papers can be accepted as poster papers only. In these cases, the authors will be allowed to maintain the original number of pages in the camera-ready version. The authors of accepted poster papers must also build and print a poster to be exhibited during the Conference. This poster must follow an A1 or A2 vertical format. The Conference includes Work Sessions where these posters are presented and orally discussed, with a 5 minute limit per poster. The authors of accepted full papers will have 15 minutes to present their work in a Conference Work Session; approximately 5 minutes of discussion will follow each presentation. The authors of accepted short papers and company papers will have 11 minutes to present their work in a Conference Work Session; approximately 4 minutes of discussion will follow each presentation. PUBLICATION AND INDEXING To ensure that a full paper, short paper, poster paper or company paper is published in the Proceedings, at least one of the authors must be fully registered by the 24th of January 2014, and the paper must comply with the suggested layout and page-limit. Additionally, all recommended changes must be addressed by the authors before they submit the camera-ready version. No more than one paper per registration will be published in the Conference Proceedings. An extra fee must be paid for publication of additional papers, with a maximum of one additional paper per registration. Full and short papers will be published in Proceedings by Springer, in Advances in Intelligent Systems and Computing Series. Poster and company papers will be published in Proceedings by AISTI. Published full and short papers will be submitted for indexation by ISI, EI-Compendex, SCOPUS and DBLP, among others, and will be available in the SpringerLink Digital Library. Published poster and company papers will be submitted for indexation by EI-Compendex and EBSCO. The authors of the best selected papers will be invited to extend them for publication in international journals indexed by ISI, SCOPUS and DBLP, among others, such as: Journal of Information Technology (JIT) Social Science Computer Review (SSCR) Information Processing & Management (IPM) Information Technology & People (ITP) Computer Science and Information Systems (ComSIS) Information Development (IDV) IEEE IT Professional (ITPro) Methods of Information in Medicine (MIM) Computer Methods in Biomechanics and Biomedical Engineering - Imaging & Visualization (CMBBE-IV) Journal of Medical Internet Research (JMIR) International Journal of Health Information Systems & Informatics (IJHISI) International Journal of Web Based Communities (IJWBC) International Journal of Interactive Multimedia and Artificial Intelligence (IJIMAI) EAI Transactions on e-Education and e-Learning (EAI-TEL) INPORTANT DATES Paper Submission: November 15, 2013 Notification of Acceptance: January 10, 2014 Camera-ready Submission: January 19, 2014 Payment of Registration, to ensure the inclusion of an accepted paper in the conference proceedings: January 24, 2014. Regards, WorldCIST'14 Team http://www.aisti.eu/worldcist14/