From iraicu at cs.iit.edu Thu Mar 1 20:02:15 2012 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Thu, 01 Mar 2012 20:02:15 -0600 Subject: [Swift-user] Call for Papers: eScience 2012 and associated workshops and tutorial Message-ID: <4F502A27.6010805@cs.iit.edu> *Call for Papers: eScience 2012 and associated workshops* * * In addition to the eScience conference itself, described below, there are six associated workshops and one tutorial (http://www.ci.uchicago.edu/escience2012/workshops.php) * Extending High-Performance Computing Beyond its Traditional User Communities, http://www.psc.edu/events/escience-2012-workshop/ * 2nd International Workshop on Analyzing and Improving Collaborative eScience with Social Networks (eSoN 12), http://www.ci.uchicago.edu/eson2012/ * Advances in eHealth, http://www.scalalife.eu/content/advances-ehealth-2012-workshop * Maintainable Software Practices in e-Science, http://software.ac.uk/maintainable-software-practice-workshop * eScience Meets the Instrument, https://confluence-vre.its.monash.edu.au/display/escience2012/eScience+Meets+the+Instrument * Collaborative research using eScience infrastructure and high speed networks, http://www.surfnet.nl/en/Hybride_netwerk/SURFlichtpaden/Pages/CollaborativeresearchusingeScienceinfrastructureandhighspeednetworks.aspx * Tutorial: Big Data Processing: Lessons from Industry and Applications in Science, http://www.ci.uchicago.edu/escience2012/tutorial.php CALL FOR PAPERS 8th IEEE International Conference on eScience http://www.ci.uchicago.edu/escience2012/ October 8-12, 2012 Chicago, IL, USA Researchers in all disciplines are increasingly adopting digital tools, techniques and practices, often in communities and projects that span disciplines, laboratories, organizations, and national boundaries. The eScience 2012 conference is designed to bring together leading international and interdisciplinary research communities, developers, and users of eScience applications and enabling IT technologies. The conference serves as a forum to present the results of the latest applications research and product/tool developments and to highlight related activities from around the world. Also, we are now entering the second decade of eScience and the 2012 conference gives an opportunity to take stock of what has been achieved so far and look forward to the challenges and opportunities the next decade will bring. A special emphasis of the 2012 conference is on advances in the application of technology in a particular discipline. Accordingly, significant advances in applications science and technology will be considered as important as the development of new technologies themselves. Further, we welcome contributions in educational activities under any of these disciplines. As a result, the conference will be structured around two e-Science tracks: . eScience Algorithms and Applications . eScience application areas, including: . Physical sciences . Biomedical sciences . Social sciences and humanities . Data-oriented approaches and applications . Compute-oriented approaches and applications . Extreme scale approaches and applications . Cyberinfrastructure to support eScience . Novel hardware . Novel uses of production infrastructure . Software and services . Tools The conference proceedings will be published by the IEEE Computer Society Press, USA and will be made available online through the IEEE Digital Library. Selected papers will be invited to submit extended versions to a special issue of the Future Generation Computer Systems (FGCS)journal. SUBMISSION PROCESS Authors are invited to submit papers with unpublished, original work of not more than 8 pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages, as per IEEE 8.5 x 11 manuscript guidelines. (Up to 2 additional pages may be purchased for US$150/page) Templates are available from http://www.ieee.org/conferences_events/conferences/publishing/templates.html. Authors should submit a PDF file that will print on a PostScript printer to https://www.easychair.org/conferences/?conf=escience2012 (Note that paper submitters also must submit an abstract in advance of the paper deadline. This should be done through the same site where papers are submitted.) It is a requirement that at least one author of each accepted paper attend the conference. IMPORTANT DATES Abstract submission (required): 4 July 2012 Paper submission: 11 July 2012 Paper author notification: 22 August 2012 Camera-ready papers due: 10 September 2012 Conference: 8-12 October 2012 CONFERENCE ORGANIZATION General Chair . Ian Foster, University of Chicago & Argonne National Laboratory, USA Program Co-Chairs . Daniel S. Katz, University of Chicago & Argonne National Laboratory, USA . Heinz Stockinger, SIB Swiss Institute of Bioinformatics, Switzerland Program Vice Co-Chairs . eScience Algorithms and Applications Track . David Abramson, Monash University, Australia . Gabrielle Allen, Louisiana State University, USA . Cyberinfrastructure to support eScience Track . Rosa M. Badia, Barcelona Supercomputing Center / CSIC, Spain . Geoffrey Fox, Indiana University, USA Early Results and Works-in-Progress Posters Chair . Roger Barga, Microsoft, USA Workshops Chair . Ruth Pordes, FNAL, USA Sponsorship Chair . Charlie Catlett, Argonne National Laboratory, USA Conference Manager and Finance Chair . Julie Wulf-Knoerzer, University of Chicago & Argonne National Laboratory, USA Publicity Chairs . Kento Aida, National Institute of Informatics, Japan . Ioan Raicu, Illinois Institute of Technology, USA . David Wallom, Oxford e-Research Centre, UK Local Organizing Committee . Ninfa Mayorga, University of Chicago, USA . Evelyn Rayburn, University of Chicago, USA . Lynn Valentini, Argonne National Laboratory, USA Program Committee . eScience Algorithms and Applications Track . Srinivas Aluru, Iowa State University, USA . Ashiq Anjum, University of Derby, UK . David A. Bader, Georgia Institute of Technology, USA . Jon Blower, University of Reading, UK . Paul Bonnington, Monash University, Australia . Simon Cox, University of Southampton, UK . David De Roure, Oxford e-Research Centre, UK . George Djorgovski, California Institute of Technology, USA . Anshu Dubey, University of Chicago & Argonne National Laboratory, USA . Yuri Estrin, Monash University, Australia . Dan Fay, Microsoft, USA . Jeremy Frey, University of Southampton, UK . Wolfgang Gentzsch, HPC Consultant, Germany . Lutz Gross, The University of Queensland, Austrialia . Sverker Holmgren, Uppsala University, Sweden . Bill Howe, University of Washington, USA . Marina Jirotka, University of Oxford, UK . Timoleon Kipouros, University of Cambridge, UK . Kerstin Kleese van Dam, Pacific Northwest National Laboratory, USA . Arun S. Konagurthu, Monash University, Australia . Peter Kunszt, SystemsX.ch , Switzerland . Alexey Lastovetsky, University College Dublin, Ireland . Andrew Lewis, Griffith University, Australia . Sergio Maffioletti, University of Zurich, Switzerland . Amitava Majumdar, San Diego Supercomputer Center, University of California at San Diego, USA . Rui Mao, Shenzhen University, China . Madhav V. Marathe, Virginia Tech, USA . Maryann Martone, University of California at San Diego, USA . Louis Moresi, Monash University, Australia . Riccardo Murri, University of Zurich, Switzerland . Silvia D. Olabarriaga, Academic Medical Center of the University of Amsterdam, Netherlands . Enrique S. Quintana-Ort?, Universidad Jaume I, Spain . Abani Patra, University at Buffalo, USA . Rob Pennington, NSF, USA . Andrew Perry, Monash University, Australia . Beth Plale, Indiana University, USA . Michael Resch, University of Stuttgart, Germany . Adrian Sandu, Virginia Tech, USA . Mark Savill, Cranfield University, UK . Erik Schnetter, Perimeter Institute for Theoretical Physics, Canada . Edward Seidel, Louisiana State University, USA . Suzanne M. Shontz, The Pennsylvania State University, USA . David Skinner, Lawrence Berkeley National Laboratory, USA . Alan Sussman, University of Maryland, USA . Alex Szalay, Johns Hopkins University, USA . Domenico Talia, ICAR-CNR & University of Calabria, Italy . Jian Tao, Louisiana State University, USA . David Wallom, Oxford e-Research Centre, UK . Shaowen Wang, University of Illinois at Urbana-Champaign, USA . Michael Wilde, Argonne National Laboratory & University of Chicago, USA . Nancy Wilkins-Diehr, San Diego Supercomputer Center, University of California at San Diego, USA . Wu Zhang, Shanghai University, China . Yunquan Zhang, Chinese Academy of Sciences, China . Cyberinfrastructure to support eScience Track . Deb Agarwal, Lawrence Berkeley National Laboratory, USA . Ilkay Altintas, San Diego Supercomputer Center, University of California at San Diego, USA . Henri Bal, Vrije Universiteit, Netherlands . Roger Barga, Microsoft, USA . Martin Berzins, University of Utah, USA . John Brooke, University of Manchester, UK . Thomas Fahringer, University of Innsbruck, Austria . Gilles Fedak, INRIA, France . Jos? A. B. Fortes, University of Florida, USA . Yolanda Gil, ISI/USC, USA . Madhusudhan Govindaraju, SUNY Binghamton, USA . Thomas Hacker, Purdue University, USA . Ken Hawick, Massey University, New Zealand . Marty Humphrey, University of Virginia, USA . Hai Jin, Huazhong University of Science and Technology, China . Thilo Kielmann, Vrije Universiteit, Netherlands . Scott Klasky, Oak Ridge National Laboratory, USA . Isao Kojima, AIST, Japan . Tevfik Kosar, University at Buffalo, USA . Dieter Kranzlmueller, LMU & LRZ Munich, Germany . Erwin Laure, KTH, Sweden . Jysoo Lee, KISTI, Korea . Li Xiaoming, Peking University, China . Bertram Lud?scher, University of California, Davis, USA . Andrew Lumsdaine, Indiana University, USA . Tanu Malik, University of Chicago, USA . Satoshi Matsuoka, Tokyo Institute of Technology, Japan . Reagan Moore, University of North Carolina at Chapel Hill, USA . Shirley Moore, University of Kentucky, USA . Steven Newhouse, EGI, Netherlands . Dhabaleswar K. (DK) Panda, The Ohio State University, USA . Manish Parashar, Rutgers University, USA . Ron Perrott, University of Oxford, UK . Depei Qian, Beihang University, China . Judy Qui, Indiana University, USA . Ioan Raicu, Illinois Institute of Technology, USA . Lavanya Ramakrishnan, Lawrence Berkeley National Laboratory, USA . Omer Rana, Cardiff University, UK . Paul Roe, Queensland University of Technology, Australia . Bruno Schulze, LNCC, Brazil . Marc Snir, Argonne National Laboratory & University of Illinois at Urbana-Champaign, USA . Xian-He Sun, Illinois Institute of Technology, USA . Yoshio Tanaka, AIST, Japan . Michela Taufer, University of Delaware, USA . Kerry Taylor, CSIRO, Australia . Douglas Thain, University of Notre Dame, USA . Paul Watson, Newcastle University, UK . Jian Zhang, Northern Illinois University, USA . Jun Zhao, University of Oxford, UK -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ ================================================================= ================================================================= -------------- next part -------------- An HTML attachment was scrubbed... URL: From iraicu at cs.iit.edu Wed Mar 7 15:15:31 2012 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Wed, 07 Mar 2012 15:15:31 -0600 Subject: [Swift-user] CFP: The 9th Int. Conf. on Autonomic Computing (ICAC) 2012 -- deadline extension to March 16th, 2012 Message-ID: <4F57CFF3.9060107@cs.iit.edu> CALL FOR PAPERS The 9th International Conference on Autonomic Computing (ICAC 2012) September 17-21, 2012. San Jose, CA, USA http://icac2012.cs.fiu.edu/ ----------------------------------------------------------------- IMPORTANT DATES Paper and Poster Submission: March 16, 2012, 11:59pm PST (EXTENDED) Notification: May 18, 2012 Camera-ready Due: June 8, 2012 ----------------------------------------------------------------- OVERVIEW ICAC is the leading conference on autonomic computing techniques, foundations, and applications. Autonomic computing refers to methods and means for automated management of performance, fault, security, and configuration with little involvement of users or administrators. Systems introducing new autonomic features are becoming increasingly prevalent, motivating research that spans a variety of areas, from computer systems, networking, software engineering, and data management to machine learning, control theory, and bio-inspired computing. ICAC brings together researchers and practitioners across these disciplines to address multiple facets of adaptation and self-management in computing systems and applications from different perspectives. Autonomic computing solutions are sought for clouds, grids, data centers, enterprise software, internet services, data services, smart phones, embedded systems, and sensor networks. In these environments, resources and applications must be managed to maximize performance and minimize cost, while maintaining predictable and reliable behavior in the face of varying workloads, failures, and malicious threats. Papers are solicited from all areas of autonomic computing, including (but not limited to): * End-to-end techniques for management of resources, workloads, performance, faults, power/cooling, security, and others. * Self-managing components, such as server, storage, network protocols, or specific application elements, and embedded and mobile end systems such as smart phones. * Decision and analysis techniques and their use, such as machine learning, control theory, predictive methods, probability and stochastic processes, queuing theory methodologies, emergent behavior, rule-based systems, and bio-inspired techniques. * Monitoring systems for autonomic computing. * Hypervisor, operating systems, hardware, or application support for autonomic computing. * Novel human interfaces for monitoring and controlling autonomic systems. * Management topics, such as specification and modeling of service-level agreements, behavior enforcement and tie-in with IT governance. * Toolkits, frameworks, principles and architectures, from software engineering practices and experimental methodologies to agent-based techniques and virtualization. * Fundamental science and theory of self-managing systems: understanding, controlling or exploiting system behaviors to enforce autonomic properties. * Applications of autonomic computing and experiences with prototyped or deployed systems solving real-world problems in science, engineering, business and society. Papers will be judged on originality, significance, interest, correctness, clarity and relevance to the broader community. Papers should report on experiences, measurements, user studies, or other evaluations, as appropriate. Evaluations of a prototype or large-scale deployment of systems and applications is expected. PAPER AND POSTER SUBMISSIONS Full papers (a maximum of 10 pages in the two-column ACM proceedings format) and posters (2 pages) are invited on a wide variety of topics relating to autonomic computing. Submitted papers must be original work, and may not be under consideration for another conference or journal. Complete formatting and submission instructions can be found on the conference web site. Accepted papers and posters will appear in proceedings distributed at the conference and available electronically. Relevant top ICAC'12 papers will be invited for "fast-track" submissions to the ACM Transactions on Autonomous and Adaptive Systems (TAAS). INDUSTRY SESSION One of ICAC's important roles is to bring together researchers and practitioners from academia and industry. In its industry session, ICAC helps fulfill this role by presenting an industry viewpoint on technologies, products, and market needs. The industry session also addresses current challenges, and opportunities for academic and corporate research collaborations. We encourage industry leaders, including entrepreneurs, product developers, architects, managers, marketers and end users, to submit their papers and posters reflecting such industry perspectives as part of the regular submission process. ------------------------------------------------------------------ ORGANIZERS GENERAL CHAIR Dejan Milojicic, HP Labs PROGRAM CHAIRS Dongyan Xu, Purdue University Vanish Talwar, HP Labs INDUSTRY CHAIR Xiaoyun Zhu, VMware WORKSHOPS CHAIR Fred Douglis, EMC POSTERS/DEMO/EXHIBITS CHAIR Eno Thereska, Microsoft Research FINANCE CHAIR Michael Kozuch, Intel LOCAL ARRANGEMENT CHAIR Jessica Blaine PUBLICITY CHAIRS Daniel Batista, University of S?o Paulo Vartan Padaryan, ISP/Russian Academy of Sci. Ioan Raicu, Illinois Inst. of Technology Jianfeng Zhan, ICT/Chinese Academy of Sci. Ming Zhao, Florida Intl. University PROGRAM COMMITTEE Tarek Abdelzaher, UIUC Umesh Bellur, IIT, Bombay Ken Birman, Cornell University Rajkumar Buyya, Univ. of Melbourne Rocky Chang, Hong Kong Polytechnic University Yuan Chen, HP Labs Alva Couch, Tufts University Peter Dinda, Northwestern University Fred Douglis, EMC Renato Figueiredo, University of Florida Mohamed Hefeeda, Qatar Computing Research Institute Joe Hellerstein, Google Geoff Jiang, NEC Labs Jeff Kephart, IBM Research Emre Kiciman, Microsoft Research Fabio Kon, University of S?o Paulo Michael Kozuch, Intel Dejan Milojicic, HP Labs Klara Nahrstedt, UIUC Priya Narasimhan, CMU Manish Parashar, Rutgers University Ioan Raicu, Illinois Inst. of Technology Omer Rana, Cardiff University Masoud Sadjadi, Florida Intl. University Rick Schlichting, AT&T Labs Hartmut Schmeck, KIT Karsten Schwan, Georgia Tech Onn Shehory, IBM Research Eno Thereska, Microsoft Research Xiaoyun Zhu, VMware -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ ================================================================= ================================================================= From mickelso at mcs.anl.gov Fri Mar 16 13:05:12 2012 From: mickelso at mcs.anl.gov (Sheri Mickelson) Date: Fri, 16 Mar 2012 13:05:12 -0500 Subject: [Swift-user] execution error Message-ID: Hi, I'm seeing this error message when I'm trying to execute a Swift script: Execution failed: First operand is null Here's the information from the log file: 2012-03-16 11:54:58,725-0600 DEBUG VDL2ExecutionContext First operand is null First operand is null at org.griphyn.vdl.karajan.lib.Operators.vdlop_eq(Operators.java:181) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun .reflect .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 43) at java.lang.reflect.Method.invoke(Method.java:616) at org .globus .cog .karajan .workflow .nodes.functions.FunctionsCollection.function(FunctionsCollection.java: 82) at org .globus .cog .karajan .workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java: 27) at org .globus .cog .karajan .workflow .nodes .AbstractSequentialWithArguments .futureModified(AbstractSequentialWithArguments.java:208) at org.griphyn.vdl.karajan.DSHandleFutureWrapper $1.run(DSHandleFutureWrapper.java:63) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java: 471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java .util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: 1110) at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) I'm having trouble finding the bug in my Swift code. What type of error usually causes this message? Thanks, Sheri From wilde at mcs.anl.gov Fri Mar 16 14:21:41 2012 From: wilde at mcs.anl.gov (Michael Wilde) Date: Fri, 16 Mar 2012 14:21:41 -0500 (CDT) Subject: [Swift-user] execution error In-Reply-To: Message-ID: <767606716.99604.1331925701801.JavaMail.root@zimbra.anl.gov> Hi Sheri, Can you point us to the entire log file? Does it happen repeatedly? Same script, same point? Or sporadically? Thanks, - Mike ----- Original Message ----- > From: "Sheri Mickelson" > To: swift-user at ci.uchicago.edu > Sent: Friday, March 16, 2012 1:05:12 PM > Subject: [Swift-user] execution error > Hi, > > I'm seeing this error message when I'm trying to execute a Swift > script: > > Execution failed: > First operand is null > > Here's the information from the log file: > > 2012-03-16 11:54:58,725-0600 DEBUG VDL2ExecutionContext First operand > is null > First operand is null > > at org.griphyn.vdl.karajan.lib.Operators.vdlop_eq(Operators.java:181) > at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > at > sun > .reflect > .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: > 43) > at java.lang.reflect.Method.invoke(Method.java:616) > at > org > .globus > .cog > .karajan > .workflow > .nodes.functions.FunctionsCollection.function(FunctionsCollection.java: > 82) > at > org > .globus > .cog > .karajan > .workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java: > 27) > at > org > .globus > .cog > .karajan > .workflow > .nodes > .AbstractSequentialWithArguments > .futureModified(AbstractSequentialWithArguments.java:208) > at org.griphyn.vdl.karajan.DSHandleFutureWrapper > $1.run(DSHandleFutureWrapper.java:63) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java: > 471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java > .util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: > 1110) > at java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > > > I'm having trouble finding the bug in my Swift code. What type of > error usually causes this message? > > Thanks, Sheri > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory From mickelso at mcs.anl.gov Fri Mar 16 14:35:34 2012 From: mickelso at mcs.anl.gov (Sheri Mickelson) Date: Fri, 16 Mar 2012 14:35:34 -0500 Subject: [Swift-user] execution error In-Reply-To: <767606716.99604.1331925701801.JavaMail.root@zimbra.anl.gov> References: <767606716.99604.1331925701801.JavaMail.root@zimbra.anl.gov> Message-ID: <351BF1E2-1B1C-4E54-8A54-804ECACAC1ED@mcs.anl.gov> Hi Mike, Here's a copy of the log file. It happens consistently with the same settings. Thanks, Sheri -------------- next part -------------- A non-text attachment was scrubbed... Name: popdiag-20120316-1328-l17ob5i5.log Type: application/octet-stream Size: 101718 bytes Desc: not available URL: -------------- next part -------------- On Mar 16, 2012, at 2:21 PM, Michael Wilde wrote: > Hi Sheri, > > Can you point us to the entire log file? > > Does it happen repeatedly? Same script, same point? Or sporadically? > > Thanks, > > - Mike > > ----- Original Message ----- >> From: "Sheri Mickelson" >> To: swift-user at ci.uchicago.edu >> Sent: Friday, March 16, 2012 1:05:12 PM >> Subject: [Swift-user] execution error >> Hi, >> >> I'm seeing this error message when I'm trying to execute a Swift >> script: >> >> Execution failed: >> First operand is null >> >> Here's the information from the log file: >> >> 2012-03-16 11:54:58,725-0600 DEBUG VDL2ExecutionContext First operand >> is null >> First operand is null >> >> at org.griphyn.vdl.karajan.lib.Operators.vdlop_eq(Operators.java:181) >> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) >> at >> sun >> .reflect >> .DelegatingMethodAccessorImpl >> .invoke(DelegatingMethodAccessorImpl.java: >> 43) >> at java.lang.reflect.Method.invoke(Method.java:616) >> at >> org >> .globus >> .cog >> .karajan >> .workflow >> .nodes >> .functions.FunctionsCollection.function(FunctionsCollection.java: >> 82) >> at >> org >> .globus >> .cog >> .karajan >> .workflow >> .nodes.functions.AbstractFunction.post(AbstractFunction.java: >> 27) >> at >> org >> .globus >> .cog >> .karajan >> .workflow >> .nodes >> .AbstractSequentialWithArguments >> .futureModified(AbstractSequentialWithArguments.java:208) >> at org.griphyn.vdl.karajan.DSHandleFutureWrapper >> $1.run(DSHandleFutureWrapper.java:63) >> at java.util.concurrent.Executors >> $RunnableAdapter.call(Executors.java: >> 471) >> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >> at >> java >> .util >> .concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >> 1110) >> at java.util.concurrent.ThreadPoolExecutor >> $Worker.run(ThreadPoolExecutor.java:603) >> at java.lang.Thread.run(Thread.java:636) >> >> >> I'm having trouble finding the bug in my Swift code. What type of >> error usually causes this message? >> >> Thanks, Sheri >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > From wilde at mcs.anl.gov Fri Mar 16 17:10:49 2012 From: wilde at mcs.anl.gov (Michael Wilde) Date: Fri, 16 Mar 2012 17:10:49 -0500 (CDT) Subject: [Swift-user] execution error In-Reply-To: <351BF1E2-1B1C-4E54-8A54-804ECACAC1ED@mcs.anl.gov> Message-ID: <1987625475.99872.1331935849632.JavaMail.root@zimbra.anl.gov> Hi Sheri, This is a Swift bug (not seen before to my knowledge). Your script trips into it in these lines starting at line 275: string monHSIfiles[] = readData(monHSIFileList); ... string datapath; if (monHSIfiles[1] == " "){ datapath = msroot; } else { datapath = wrkdir; } I suspect that in your test case, monHSIFileList has less than 2 lines in it. The array monHSIfiles is "closed", I think, because its returned from readData(). But element [1] of the array doesn't exist. Did you mean to test element[0] for being equal to " "? Does monHSIFileList get set with an end-of-filelist sentinel of " "? I think if you work through the intent here, you can avoid testing an element that doesnt exist. How Swift *should* behave here needs to be discussed and defined precisely in the User Guide and implementation. - Mike ps. Here's the test that reproduces it: $ cat 1filename foo $ cat 2filenames foo bar $ cat b1.swift string fnames[] = readData(@arg("flist")); if (fnames[1] == " "){ trace("blank"); } else { trace("nonblank"); } $ swift b1.swift -flist=2filenames no sites file specified, setting to default: /Users/wilde/swift/src/trunk/cog/modules/swift/dist/swift-svn/etc/sites.xml Swift trunk swift-r5690 cog-r3361 RunID: 20120316-1700-f7ev3njf Progress: time: Fri, 16 Mar 2012 17:00:50 -0500 SwiftScript trace: nonblank Final status: Fri, 16 Mar 2012 17:00:50 -0500 $ swift b1.swift -flist=1filename no sites file specified, setting to default: /Users/wilde/swift/src/trunk/cog/modules/swift/dist/swift-svn/etc/sites.xml Swift trunk swift-r5690 cog-r3361 RunID: 20120316-1700-1xah6tge Progress: time: Fri, 16 Mar 2012 17:00:57 -0500 Execution failed: First operand is null $ ----- Original Message ----- > From: "Sheri Mickelson" > To: "Michael Wilde" > Cc: swift-user at ci.uchicago.edu > Sent: Friday, March 16, 2012 2:35:34 PM > Subject: Re: [Swift-user] execution error > Hi Mike, > > Here's a copy of the log file. It happens consistently with the same > settings. > > Thanks, Sheri > > > > > > On Mar 16, 2012, at 2:21 PM, Michael Wilde wrote: > > > Hi Sheri, > > > > Can you point us to the entire log file? > > > > Does it happen repeatedly? Same script, same point? Or sporadically? > > > > Thanks, > > > > - Mike > > > > ----- Original Message ----- > >> From: "Sheri Mickelson" > >> To: swift-user at ci.uchicago.edu > >> Sent: Friday, March 16, 2012 1:05:12 PM > >> Subject: [Swift-user] execution error > >> Hi, > >> > >> I'm seeing this error message when I'm trying to execute a Swift > >> script: > >> > >> Execution failed: > >> First operand is null > >> > >> Here's the information from the log file: > >> > >> 2012-03-16 11:54:58,725-0600 DEBUG VDL2ExecutionContext First > >> operand > >> is null > >> First operand is null > >> > >> at > >> org.griphyn.vdl.karajan.lib.Operators.vdlop_eq(Operators.java:181) > >> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > >> at > >> sun > >> .reflect > >> .DelegatingMethodAccessorImpl > >> .invoke(DelegatingMethodAccessorImpl.java: > >> 43) > >> at java.lang.reflect.Method.invoke(Method.java:616) > >> at > >> org > >> .globus > >> .cog > >> .karajan > >> .workflow > >> .nodes > >> .functions.FunctionsCollection.function(FunctionsCollection.java: > >> 82) > >> at > >> org > >> .globus > >> .cog > >> .karajan > >> .workflow > >> .nodes.functions.AbstractFunction.post(AbstractFunction.java: > >> 27) > >> at > >> org > >> .globus > >> .cog > >> .karajan > >> .workflow > >> .nodes > >> .AbstractSequentialWithArguments > >> .futureModified(AbstractSequentialWithArguments.java:208) > >> at org.griphyn.vdl.karajan.DSHandleFutureWrapper > >> $1.run(DSHandleFutureWrapper.java:63) > >> at java.util.concurrent.Executors > >> $RunnableAdapter.call(Executors.java: > >> 471) > >> at > >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) > >> at > >> java > >> .util > >> .concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: > >> 1110) > >> at java.util.concurrent.ThreadPoolExecutor > >> $Worker.run(ThreadPoolExecutor.java:603) > >> at java.lang.Thread.run(Thread.java:636) > >> > >> > >> I'm having trouble finding the bug in my Swift code. What type of > >> error usually causes this message? > >> > >> Thanks, Sheri > >> _______________________________________________ > >> Swift-user mailing list > >> Swift-user at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > > > -- > > Michael Wilde > > Computation Institute, University of Chicago > > Mathematics and Computer Science Division > > Argonne National Laboratory > > -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory From mickelso at mcs.anl.gov Fri Mar 16 18:05:44 2012 From: mickelso at mcs.anl.gov (Sheri Mickelson) Date: Fri, 16 Mar 2012 18:05:44 -0500 Subject: [Swift-user] execution error In-Reply-To: <1987625475.99872.1331935849632.JavaMail.root@zimbra.anl.gov> References: <1987625475.99872.1331935849632.JavaMail.root@zimbra.anl.gov> Message-ID: Thanks, that fixed it. -Sheri On Mar 16, 2012, at 5:10 PM, Michael Wilde wrote: > Hi Sheri, > > This is a Swift bug (not seen before to my knowledge). Your script > trips into it in these lines starting at line 275: > > string monHSIfiles[] = readData(monHSIFileList); > ... > string datapath; > if (monHSIfiles[1] == " "){ > datapath = msroot; > } else { > datapath = wrkdir; > } > > I suspect that in your test case, monHSIFileList has less than 2 > lines in it. The array monHSIfiles is "closed", I think, because its > returned from readData(). But element [1] of the array doesn't > exist. Did you mean to test element[0] for being equal to " "? Does > monHSIFileList get set with an end-of-filelist sentinel of " "? > > I think if you work through the intent here, you can avoid testing > an element that doesnt exist. > > How Swift *should* behave here needs to be discussed and defined > precisely in the User Guide and implementation. > > - Mike > > ps. > > Here's the test that reproduces it: > > $ cat 1filename > foo > $ cat 2filenames > foo > bar > $ cat b1.swift > string fnames[] = readData(@arg("flist")); > > if (fnames[1] == " "){ > trace("blank"); > } else { > trace("nonblank"); > } > > $ swift b1.swift -flist=2filenames > no sites file specified, setting to default: /Users/wilde/swift/src/ > trunk/cog/modules/swift/dist/swift-svn/etc/sites.xml > Swift trunk swift-r5690 cog-r3361 > > RunID: 20120316-1700-f7ev3njf > Progress: time: Fri, 16 Mar 2012 17:00:50 -0500 > SwiftScript trace: nonblank > Final status: Fri, 16 Mar 2012 17:00:50 -0500 > $ swift b1.swift -flist=1filename > no sites file specified, setting to default: /Users/wilde/swift/src/ > trunk/cog/modules/swift/dist/swift-svn/etc/sites.xml > Swift trunk swift-r5690 cog-r3361 > > RunID: 20120316-1700-1xah6tge > Progress: time: Fri, 16 Mar 2012 17:00:57 -0500 > Execution failed: > First operand is null > $ > > > ----- Original Message ----- >> From: "Sheri Mickelson" >> To: "Michael Wilde" >> Cc: swift-user at ci.uchicago.edu >> Sent: Friday, March 16, 2012 2:35:34 PM >> Subject: Re: [Swift-user] execution error >> Hi Mike, >> >> Here's a copy of the log file. It happens consistently with the same >> settings. >> >> Thanks, Sheri >> >> >> >> >> >> On Mar 16, 2012, at 2:21 PM, Michael Wilde wrote: >> >>> Hi Sheri, >>> >>> Can you point us to the entire log file? >>> >>> Does it happen repeatedly? Same script, same point? Or sporadically? >>> >>> Thanks, >>> >>> - Mike >>> >>> ----- Original Message ----- >>>> From: "Sheri Mickelson" >>>> To: swift-user at ci.uchicago.edu >>>> Sent: Friday, March 16, 2012 1:05:12 PM >>>> Subject: [Swift-user] execution error >>>> Hi, >>>> >>>> I'm seeing this error message when I'm trying to execute a Swift >>>> script: >>>> >>>> Execution failed: >>>> First operand is null >>>> >>>> Here's the information from the log file: >>>> >>>> 2012-03-16 11:54:58,725-0600 DEBUG VDL2ExecutionContext First >>>> operand >>>> is null >>>> First operand is null >>>> >>>> at >>>> org.griphyn.vdl.karajan.lib.Operators.vdlop_eq(Operators.java:181) >>>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) >>>> at >>>> sun >>>> .reflect >>>> .DelegatingMethodAccessorImpl >>>> .invoke(DelegatingMethodAccessorImpl.java: >>>> 43) >>>> at java.lang.reflect.Method.invoke(Method.java:616) >>>> at >>>> org >>>> .globus >>>> .cog >>>> .karajan >>>> .workflow >>>> .nodes >>>> .functions.FunctionsCollection.function(FunctionsCollection.java: >>>> 82) >>>> at >>>> org >>>> .globus >>>> .cog >>>> .karajan >>>> .workflow >>>> .nodes.functions.AbstractFunction.post(AbstractFunction.java: >>>> 27) >>>> at >>>> org >>>> .globus >>>> .cog >>>> .karajan >>>> .workflow >>>> .nodes >>>> .AbstractSequentialWithArguments >>>> .futureModified(AbstractSequentialWithArguments.java:208) >>>> at org.griphyn.vdl.karajan.DSHandleFutureWrapper >>>> $1.run(DSHandleFutureWrapper.java:63) >>>> at java.util.concurrent.Executors >>>> $RunnableAdapter.call(Executors.java: >>>> 471) >>>> at >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >>>> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >>>> at >>>> java >>>> .util >>>> .concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java: >>>> 1110) >>>> at java.util.concurrent.ThreadPoolExecutor >>>> $Worker.run(ThreadPoolExecutor.java:603) >>>> at java.lang.Thread.run(Thread.java:636) >>>> >>>> >>>> I'm having trouble finding the bug in my Swift code. What type of >>>> error usually causes this message? >>>> >>>> Thanks, Sheri >>>> _______________________________________________ >>>> Swift-user mailing list >>>> Swift-user at ci.uchicago.edu >>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >>> >>> -- >>> Michael Wilde >>> Computation Institute, University of Chicago >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > From iraicu at cs.iit.edu Thu Mar 22 17:33:24 2012 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Thu, 22 Mar 2012 17:33:24 -0500 Subject: [Swift-user] Call for Participation: 12th IEEE/ACM Int. Symp. on Cluster, Grid and Cloud, Computing (CCGrid 2012) -- May13-16 in Ottawa, Canada Message-ID: <4F6BA8B4.80008@cs.iit.edu> *** Our apologies if you receive multiple copies of this Call *** 12th IEEE/ACM International Symposium on Cluster, Grid and Cloud Computing (CCGrid 2012) Ottawa, Canada May 13-16, 2012 http://www.cloudbus.org/ccgrid2012 Venue: The Delta Ottawa City Centre Hotel [Special rates for conference attendees. To book visit: www.cloudbus.org/ccgrid2012/accommodations.html.] *************************** CALL FOR PARTICPATION *************************** *** Registration is Open. *** *** Take advantage of the Early Registration Rates until April 9/2012 *** Overview: Rapid advances in processing, communication and systems/middleware technologies are leading to new paradigms and platforms for computing, ranging from computing Clusters to widely distributed Grid and emerging Clouds. CCGrid is a series of very successful conferences, sponsored by the IEEE Computer Society Technical Committee on Scalable Computing (TCSC) and ACM, with the overarching goal of bringing together international researchers, developers, and users and to provide an international forum to present leading research activities and results on a broad range of topics related to these platforms and paradigms and their applications. The conference features keynotes, technical presentations, posters and research demos, workshops, tutorials, as well as the SCALE challenges featuring live demonstrations. In 2012, CCGrid will come to Canada for the first time and will be held in Ottawa, the capital city. The symposium will be held on May 13-16 during which the city will be celebrating its world-famous Tulip Festival. CCGrid 2012 will have a focus on important and immediate issues that are significantly influencing all aspects of cluster, cloud and grid computing. Topics discussed in the technical sessions include: Applications and Experiences; Architecture: System architectures, Design and deployment; Autonomic Computing and Cyberinfrastructure; Performance Modeling and Evaluation; Programming Models, Systems, and Fault-Tolerant Computing; Multicore and Accelerator-based Computing; Scheduling and Resource Management; Cloud Computing: Cloud architectures; Software tools and techniques for clouds. ******************* PROGRAM HIGHLIGHTS ****************** TECHNICAL SESSIONS WILL INCLUDE * Programming Models and File Systems * Map Reduce and Workflows * QoS and Architecture * GPU * Cloud Services I * I/O and File Systems) * Programming Models) * Cloud Computing I * Communication and Networks * Faults, Failures and Reliability * Workflows * Scheduling and Monitoring * Virtualization * Cloud Services * Data on the Cloud * Multicore Architectures * Cloud Computing II * Applications KEYNOTES: * Dr. Alok Chaudhury (North Western University, USA) * Dr. Dick Epema (TU Delfts, Netherlands) * The winner of the TCSC medal on Scalable Computing WORKSHOPS: * International workshop on Cloud for Business, Industry and Enterprises (C4BIE 2012) * Workshop on Cloud Computing Optimization (CCOPT 2012) * 2nd International Workshop on Cloud Computing and Scientific Applications (CCSA 2012) * Workshop on Modeling and Simulation on Grid and Cloud Computing (MSGC 2012) * 1st International Workshop on Data-intensive Process Management in Large-Scale Sensor Systems (DPMSS 2012) TUTORIALS PANELS & INDUSTRIAL SESSIONS DOCTORAL SYMPOSIUM POSTER/DEMO SESSIONS SCALE 2012: The Fourth IEEE International Scalable Computing Challenge CHAIRS General Chair * Shikharesh Majumdar, Carleton University, Canada Program Committee Co-Chairs * Rajkumar Buyya, University of Melbourne, Australia * Pavan Balaji, Argonne National Laboratory, USA Program Committee Vice-chairs * Daniel S. Katz (Applications and Experiences) * Dhabaleswar K. Panda (Architecture) * Manish Parashar (Middleware, Autonomic Computing, and Cyberinfrastructure) * Ahmad Afsahi (Performance Modeling and Analysis) * Xian-He Sun (Performance Measurement and Evaluation) * William Gropp (Programming Models, Systems, and Fault-Tolerant computing) * David Bader (Multicore and Accelerator-based Computing) * Thomas Fahringer (Scheduling and Resource Management) * Ignacio Martin Llorente and Madhusudhan Govindaraju (Cloud Computing) Honorary Chair * Geoffrey Fox, Indiana University, USA IMPORTANT DATES Early Registration: From now until April 9/2012 Late/Onsite Registration: April 10, 2012 onwards Conference: May 13-16, 2012 -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ ================================================================= ================================================================= From iraicu at cs.iit.edu Fri Mar 23 18:59:17 2012 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Fri, 23 Mar 2012 18:59:17 -0500 Subject: [Swift-user] CFP: Special Issue on Cloud Computing in Science & Engineering, in the the IEEE Computing in Science & Engineering (CiSE) Message-ID: <4F6D0E55.9040906@cs.iit.edu> *Call for Papers* *IEEE Computing in Science & Engineering* ** *Special Issue on Cloud Computing in Science & Engineering* http://www.computer.org/portal/web/computingnow/cise ** *Submissions due: November 04, 2012* *Estimated Publication date: July/August, 2013* Cloud computing has emerged as a dominant paradigm that has been widely adopted by enterprises. Clouds provide on-demand access to computing utilities, an abstraction of unlimited computing resources, and support for on-demand scale up, scale down and scale out. Clouds are also rapidly joining more traditional computing platforms as viable platforms for scientific exploration and discovery, and education. As a result, understanding application formulations and usage modes that are meaningful in such a hybrid infrastructure, what are the fundamental conceptual and technological challenges, and how applications can effectively utilize it, is critical. The goal of this special issue of CiSE is to explore how Clouds platforms and abstractions, either by themselves or in combination with other platforms, can be effectively used to support real-world science and engineering applications. Topics of interest include (but are not limited to) algorithmic and application formulations, programming models and systems, runtime systems and middleware, end-to-end application workflows and experiences with real applications. Published by the IEEE Computer Society, Computing in Science & Engineering magazine features the latest computational science and engineering research in an accessible format, along with departments covering news and analysis, CSE in education, and emerging technologies. We strongly encourage submissions that include multimedia, data, and community content, which will be featured on the IEEE Computer Society website along with the accepted papers. ** For more information please see http://www.computer.org/portal/web/computingnow/cscfp4 *Questions?* Contact guest editors *Manish Parashar, *Rutgers University (parashar at rutgers.edu) or *George K. Thiruvathukal, *Loyola University Chicago?(gkt at cs.luc.edu).** ** *Submission Guidelines* Authors are asked to submit high-quality original work that has neither appeared in nor is under consideration by other journals. All submissions will be peer-reviewed following standard journal practices. Manuscripts based on previously published conference papers must be extended substantially to include at least 30 percent new material. Manuscripts should be written in the active voice, should be no longer than 7,200 words (counting each standard figure and table as 250 words), and should follow the style and presentation guidelines of /CiSE /(see *www.computer.org/cise/author* for details). Please submit your article using the online manuscript submission service at *https://mc.manuscriptcentral.com/cs-ieee*. When uploading your article, select the appropriate special-issue title under the category "Manuscript Type." Also include complete contact information for all authors. If you have any questions about submitting your article, contact the peer review coordinator at *cise at computer.org* . -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ ================================================================= ================================================================= -------------- next part -------------- An HTML attachment was scrubbed... URL: From lpesce at uchicago.edu Fri Mar 23 19:30:12 2012 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Fri, 23 Mar 2012 19:30:12 -0500 Subject: [Swift-user] Question about output files In-Reply-To: <1647045050.109132.1332514723911.JavaMail.root@zimbra.anl.gov> References: <1647045050.109132.1332514723911.JavaMail.root@zimbra.anl.gov> Message-ID: Hi -- I am writing a script to run a parameter sweep on Beagle (Cray XE6). I am interested in the output files they write, but somehow I seem to be unable to locate them. The calculations seem to proceed fine according to the log files (stdout), but I can't find the mat files that should contain the actual results. The should be files like this one result_neuron_##_ht_##.mat This is the script. Thanks a lot! // file to run the Margic square example from the matlab web site type file; string MCRPath = "/soft/matlab/7.13"; string spikefile = "X.mat"; app (file outdata) demoreal (string mcr, int chn, int ht, string datafile) { runDemoReal mcr chn ht datafile stdout=@outdata; } int ht[] = [3, 6, 9, 12, 15, 18, 21, 24]; file LogDemoReal[] ; foreach s, i in ht { int chn=1; LogDemoReal[s] = demoreal (MCRPath, chn, s, spikefile); } From davidk at ci.uchicago.edu Sat Mar 24 01:12:05 2012 From: davidk at ci.uchicago.edu (David Kelly) Date: Sat, 24 Mar 2012 01:12:05 -0500 (CDT) Subject: [Swift-user] Question about output files In-Reply-To: Message-ID: <2013749672.58796.1332569525146.JavaMail.root@zimbra-mb2.anl.gov> Lorenzo, The result files will have to be defined in your swift script to get them back from the work directory. There are a few different ways to go about this. Here is one simple example. For this example, I'm going to assume that each app call creates one result file, and that the value of "s" determines the filename. app (file outdata, file result) demoreal (string mcr, int chn, int ht, string datafile) { runDemoReal mcr chn ht datafile stdout=@outdata; } . . . foreach s, i in ht { int chn=1; file result ; (LogDemoReal[s], result) = demoreal (MCRPath, chn, s, spikefile); } David ----- Original Message ----- > From: "Lorenzo Pesce" > To: swift-user at ci.uchicago.edu > Sent: Friday, March 23, 2012 7:30:12 PM > Subject: [Swift-user] Question about output files > Hi -- > > I am writing a script to run a parameter sweep on Beagle (Cray XE6). > > I am interested in the output files they write, but somehow I seem to > be unable to locate them. The calculations seem to proceed fine > according to the log files (stdout), but I can't find the mat files > that should contain the actual results. > The should be files like this one result_neuron_##_ht_##.mat > > This is the script. Thanks a lot! > > // file to run the Margic square example from the matlab web site > type file; > > string MCRPath = "/soft/matlab/7.13"; > string spikefile = "X.mat"; > > app (file outdata) demoreal (string mcr, int chn, int ht, string > datafile) > { > runDemoReal mcr chn ht datafile stdout=@outdata; > } > > int ht[] = [3, 6, 9, 12, 15, 18, 21, 24]; > > file LogDemoReal[] ; > > foreach s, i in ht { > int chn=1; > LogDemoReal[s] = demoreal (MCRPath, chn, s, spikefile); > } > > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user From lpesce at uchicago.edu Sat Mar 24 09:35:07 2012 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Sat, 24 Mar 2012 09:35:07 -0500 Subject: [Swift-user] Question about output files In-Reply-To: <2013749672.58796.1332569525146.JavaMail.root@zimbra-mb2.anl.gov> References: <2013749672.58796.1332569525146.JavaMail.root@zimbra-mb2.anl.gov> Message-ID: <02B5EF48-9ECF-4687-B7F4-5CF798169AC9@uchicago.edu> Thanks a lot. Let me test this and get back to you ASAP. On Mar 24, 2012, at 1:12 AM, David Kelly wrote: > Lorenzo, > > The result files will have to be defined in your swift script to get them back from the work directory. There are a few different ways to go about this. > > Here is one simple example. For this example, I'm going to assume that each app call creates one result file, and that the value of "s" determines the filename. > > app (file outdata, file result) demoreal (string mcr, int chn, int ht, string datafile) > { > runDemoReal mcr chn ht datafile stdout=@outdata; > } > . > . > . > foreach s, i in ht { > int chn=1; > file result ; > (LogDemoReal[s], result) = demoreal (MCRPath, chn, s, spikefile); > } > > David > > ----- Original Message ----- >> From: "Lorenzo Pesce" >> To: swift-user at ci.uchicago.edu >> Sent: Friday, March 23, 2012 7:30:12 PM >> Subject: [Swift-user] Question about output files >> Hi -- >> >> I am writing a script to run a parameter sweep on Beagle (Cray XE6). >> >> I am interested in the output files they write, but somehow I seem to >> be unable to locate them. The calculations seem to proceed fine >> according to the log files (stdout), but I can't find the mat files >> that should contain the actual results. >> The should be files like this one result_neuron_##_ht_##.mat >> >> This is the script. Thanks a lot! >> >> // file to run the Margic square example from the matlab web site >> type file; >> >> string MCRPath = "/soft/matlab/7.13"; >> string spikefile = "X.mat"; >> >> app (file outdata) demoreal (string mcr, int chn, int ht, string >> datafile) >> { >> runDemoReal mcr chn ht datafile stdout=@outdata; >> } >> >> int ht[] = [3, 6, 9, 12, 15, 18, 21, 24]; >> >> file LogDemoReal[] ; >> >> foreach s, i in ht { >> int chn=1; >> LogDemoReal[s] = demoreal (MCRPath, chn, s, spikefile); >> } >> >> >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user From lpesce at uchicago.edu Sat Mar 24 10:18:13 2012 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Sat, 24 Mar 2012 10:18:13 -0500 Subject: [Swift-user] Question about output files In-Reply-To: <2013749672.58796.1332569525146.JavaMail.root@zimbra-mb2.anl.gov> References: <2013749672.58796.1332569525146.JavaMail.root@zimbra-mb2.anl.gov> Message-ID: <4DCCC1B7-3EFB-4826-BC41-26EDF1CC5233@uchicago.edu> Worked perfect. Thanks a million. I have another question. In my tc file I will have a number of alternative definitions for the function runDemoReal and sometimes the name of the files to be written back will change with it. Is there a way to have that be part of the tc file itself without having to be changed in the swift script? On Mar 24, 2012, at 1:12 AM, David Kelly wrote: > Lorenzo, > > The result files will have to be defined in your swift script to get them back from the work directory. There are a few different ways to go about this. > > Here is one simple example. For this example, I'm going to assume that each app call creates one result file, and that the value of "s" determines the filename. > > app (file outdata, file result) demoreal (string mcr, int chn, int ht, string datafile) > { > runDemoReal mcr chn ht datafile stdout=@outdata; > } > . > . > . > foreach s, i in ht { > int chn=1; > file result ; > (LogDemoReal[s], result) = demoreal (MCRPath, chn, s, spikefile); > } > > David > > ----- Original Message ----- >> From: "Lorenzo Pesce" >> To: swift-user at ci.uchicago.edu >> Sent: Friday, March 23, 2012 7:30:12 PM >> Subject: [Swift-user] Question about output files >> Hi -- >> >> I am writing a script to run a parameter sweep on Beagle (Cray XE6). >> >> I am interested in the output files they write, but somehow I seem to >> be unable to locate them. The calculations seem to proceed fine >> according to the log files (stdout), but I can't find the mat files >> that should contain the actual results. >> The should be files like this one result_neuron_##_ht_##.mat >> >> This is the script. Thanks a lot! >> >> // file to run the Margic square example from the matlab web site >> type file; >> >> string MCRPath = "/soft/matlab/7.13"; >> string spikefile = "X.mat"; >> >> app (file outdata) demoreal (string mcr, int chn, int ht, string >> datafile) >> { >> runDemoReal mcr chn ht datafile stdout=@outdata; >> } >> >> int ht[] = [3, 6, 9, 12, 15, 18, 21, 24]; >> >> file LogDemoReal[] ; >> >> foreach s, i in ht { >> int chn=1; >> LogDemoReal[s] = demoreal (MCRPath, chn, s, spikefile); >> } >> >> >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user From davidk at ci.uchicago.edu Sat Mar 24 11:30:44 2012 From: davidk at ci.uchicago.edu (David Kelly) Date: Sat, 24 Mar 2012 11:30:44 -0500 (CDT) Subject: [Swift-user] Question about output files In-Reply-To: <4DCCC1B7-3EFB-4826-BC41-26EDF1CC5233@uchicago.edu> Message-ID: <1470582240.59315.1332606644605.JavaMail.root@zimbra-mb2.anl.gov> Lorenzo, What I would do in this case is have the tc app be a small wrapper shell script. The wrapper would execute the real app and then create a tar file containing the results. Then the only file that you need to stage out is the tar file. This is what I typically do when there are a large number of outputs, or when the output filenames are not known ahead of time. David ----- Original Message ----- > From: "Lorenzo Pesce" > To: "David Kelly" > Cc: swift-user at ci.uchicago.edu > Sent: Saturday, March 24, 2012 10:18:13 AM > Subject: Re: [Swift-user] Question about output files > Worked perfect. Thanks a million. > > I have another question. In my tc file I will have a number of > alternative definitions for the function runDemoReal and sometimes the > name of the files to be written back will change with it. > Is there a way to have that be part of the tc file itself without > having to be changed in the swift script? > > > > On Mar 24, 2012, at 1:12 AM, David Kelly wrote: > > > Lorenzo, > > > > The result files will have to be defined in your swift script to get > > them back from the work directory. There are a few different ways to > > go about this. > > > > Here is one simple example. For this example, I'm going to assume > > that each app call creates one result file, and that the value of > > "s" determines the filename. > > > > app (file outdata, file result) demoreal (string mcr, int chn, int > > ht, string datafile) > > { > > runDemoReal mcr chn ht datafile stdout=@outdata; > > } > > . > > . > > . > > foreach s, i in ht { > > int chn=1; > > file result > "_ht_00.mat)>; > > (LogDemoReal[s], result) = demoreal (MCRPath, chn, s, spikefile); > > } > > > > David > > > > ----- Original Message ----- > >> From: "Lorenzo Pesce" > >> To: swift-user at ci.uchicago.edu > >> Sent: Friday, March 23, 2012 7:30:12 PM > >> Subject: [Swift-user] Question about output files > >> Hi -- > >> > >> I am writing a script to run a parameter sweep on Beagle (Cray > >> XE6). > >> > >> I am interested in the output files they write, but somehow I seem > >> to > >> be unable to locate them. The calculations seem to proceed fine > >> according to the log files (stdout), but I can't find the mat files > >> that should contain the actual results. > >> The should be files like this one result_neuron_##_ht_##.mat > >> > >> This is the script. Thanks a lot! > >> > >> // file to run the Margic square example from the matlab web site > >> type file; > >> > >> string MCRPath = "/soft/matlab/7.13"; > >> string spikefile = "X.mat"; > >> > >> app (file outdata) demoreal (string mcr, int chn, int ht, string > >> datafile) > >> { > >> runDemoReal mcr chn ht datafile stdout=@outdata; > >> } > >> > >> int ht[] = [3, 6, 9, 12, 15, 18, 21, 24]; > >> > >> file LogDemoReal[] >> prefix="demo_real",suffix=".log">; > >> > >> foreach s, i in ht { > >> int chn=1; > >> LogDemoReal[s] = demoreal (MCRPath, chn, s, spikefile); > >> } > >> > >> > >> > >> _______________________________________________ > >> Swift-user mailing list > >> Swift-user at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user From lpesce at uchicago.edu Mon Mar 26 13:38:56 2012 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Mon, 26 Mar 2012 13:38:56 -0500 Subject: [Swift-user] Question about packing jobs in Cray XE6 nodes Message-ID: Hi all -- Thanks a lot for the help so far. Most jobs work fine, but some of them crash. Crashing appears to be caused by either: a) Node runs out of memory (but it seems that it affects only one job, not the whole node -- however, when I send out the job alone it works fine) b) Lack of convergence (algorithm needs to be changed) I am testing my hypothesis right now. Is it possible to split the pool of nodes into two groups, one where I run them more packed and one where the more demanding ones are sent? Thanks a lot, Lorenzo From wilde at mcs.anl.gov Mon Mar 26 14:08:26 2012 From: wilde at mcs.anl.gov (Michael Wilde) Date: Mon, 26 Mar 2012 14:08:26 -0500 (CDT) Subject: [Swift-user] Question about packing jobs in Cray XE6 nodes In-Reply-To: Message-ID: <2045676666.112703.1332788906582.JavaMail.root@zimbra.anl.gov> Hi Lorenzo, ----- Original Message ----- > From: "Lorenzo Pesce" > To: swift-user at ci.uchicago.edu > Sent: Monday, March 26, 2012 1:38:56 PM > Subject: [Swift-user] Question about packing jobs in Cray XE6 nodes > Hi all -- > Thanks a lot for the help so far. > > Most jobs work fine, but some of them crash. Crashing appears to be > caused by either: > a) Node runs out of memory (but it seems that it affects only one job, > not the whole node -- however, when I send out the job alone it works > fine) If a none is truly running out of memory, to the point where the Linux kernel "out of memory" action is triggered, the entire PBS job will be killed. I think that would be more visible to you (likely from PBS errors received by Swift). > b) Lack of convergence (algorithm needs to be changed) > > > I am testing my hypothesis right now. > > Is it possible to split the pool of nodes into two groups, one where I > run them more packed and one where the more demanding ones are sent? Yes; you can create multiple "pool" entries in your sites file with different JobsPerNode values. Then you can create multiple versions of your app entry or entries in your tc file, with a different app name (2nd field) for each site. Then in your Swift script you need to create can call multiple app() function names, using the app() name to determine what site it runs on. Thats a bit crude, but it works. A future Swift enhancement might let you force an app call to run on a specific site via a settable parameter. Depending on what you are trying to vary between sites, you might be able to do something clever by varying an environment variable within a single app function definition. I'll look for that info and post a pointer. - Mike > Thanks a lot, > > Lorenzo > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory From wilde at mcs.anl.gov Mon Mar 26 14:37:07 2012 From: wilde at mcs.anl.gov (Michael Wilde) Date: Mon, 26 Mar 2012 14:37:07 -0500 (CDT) Subject: [Swift-user] Question about packing jobs in Cray XE6 nodes In-Reply-To: <2045676666.112703.1332788906582.JavaMail.root@zimbra.anl.gov> Message-ID: <299891716.112796.1332790627151.JavaMail.root@zimbra.anl.gov> First, correction to my prior message: I meant to say "If a node is truly running out of memory..." Second, I spoke to Justin about existing mechanisms to help you set memory requirements. Nothing implemented at the moment offers a better alternative to the "multiple sites/multiple app versions" approach I described below. You can hide the multiple app() function names behind a higher level compound function that selects the right app() variant based eg on calculated memory needs. I'd start with a simple example that perhaps does, for example, one app for 24 workers per node and one app for one worker per node. Something like this (omotting many details): myapp() { if (needsMuchMem) { myApp01() } else { myApp24() } - Mike ----- Original Message ----- > From: "Michael Wilde" > To: "Lorenzo Pesce" > Cc: swift-user at ci.uchicago.edu > Sent: Monday, March 26, 2012 2:08:26 PM > Subject: Re: [Swift-user] Question about packing jobs in Cray XE6 nodes > Hi Lorenzo, > > ----- Original Message ----- > > From: "Lorenzo Pesce" > > To: swift-user at ci.uchicago.edu > > Sent: Monday, March 26, 2012 1:38:56 PM > > Subject: [Swift-user] Question about packing jobs in Cray XE6 nodes > > Hi all -- > > Thanks a lot for the help so far. > > > > Most jobs work fine, but some of them crash. Crashing appears to be > > caused by either: > > a) Node runs out of memory (but it seems that it affects only one > > job, > > not the whole node -- however, when I send out the job alone it > > works > > fine) > > If a none is truly running out of memory, to the point where the Linux > kernel "out of memory" action is triggered, the entire PBS job will be > killed. I think that would be more visible to you (likely from PBS > errors received by Swift). > > > b) Lack of convergence (algorithm needs to be changed) > > > > > > I am testing my hypothesis right now. > > > > Is it possible to split the pool of nodes into two groups, one where > > I > > run them more packed and one where the more demanding ones are sent? > > Yes; you can create multiple "pool" entries in your sites file with > different JobsPerNode values. Then you can create multiple versions of > your app entry or entries in your tc file, with a different app name > (2nd field) for each site. > > Then in your Swift script you need to create can call multiple app() > function names, using the app() name to determine what site it runs > on. > > Thats a bit crude, but it works. A future Swift enhancement might let > you force an app call to run on a specific site via a settable > parameter. > > Depending on what you are trying to vary between sites, you might be > able to do something clever by varying an environment variable within > a single app function definition. I'll look for that info and post a > pointer. > > - Mike > > > > > Thanks a lot, > > > > Lorenzo > > _______________________________________________ > > Swift-user mailing list > > Swift-user at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory From lpesce at uchicago.edu Mon Mar 26 20:00:59 2012 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Mon, 26 Mar 2012 20:00:59 -0500 Subject: [Swift-user] Question about packing jobs in Cray XE6 nodes In-Reply-To: <2045676666.112703.1332788906582.JavaMail.root@zimbra.anl.gov> References: <2045676666.112703.1332788906582.JavaMail.root@zimbra.anl.gov> Message-ID: <7F2CC05E-B05D-4CBC-959D-52801CD2686F@uchicago.edu> Thanks a lot for your kind reply. Sorry today I am mostly trying to fix the talk for tomorrow. Wednesday (or tomorrow morning). On Mar 26, 2012, at 2:08 PM, Michael Wilde wrote: > Hi Lorenzo, > > ----- Original Message ----- >> From: "Lorenzo Pesce" >> To: swift-user at ci.uchicago.edu >> Sent: Monday, March 26, 2012 1:38:56 PM >> Subject: [Swift-user] Question about packing jobs in Cray XE6 nodes >> Hi all -- >> Thanks a lot for the help so far. >> >> Most jobs work fine, but some of them crash. Crashing appears to be >> caused by either: >> a) Node runs out of memory (but it seems that it affects only one job, >> not the whole node -- however, when I send out the job alone it works >> fine) > > If a none is truly running out of memory, to the point where the Linux kernel "out of memory" action is triggered, the entire PBS job will be killed. I think that would be more visible to you (likely from PBS errors received by Swift). > >> b) Lack of convergence (algorithm needs to be changed) >> >> >> I am testing my hypothesis right now. >> >> Is it possible to split the pool of nodes into two groups, one where I >> run them more packed and one where the more demanding ones are sent? > > Yes; you can create multiple "pool" entries in your sites file with different JobsPerNode values. Then you can create multiple versions of your app entry or entries in your tc file, with a different app name (2nd field) for each site. > > Then in your Swift script you need to create can call multiple app() function names, using the app() name to determine what site it runs on. > > Thats a bit crude, but it works. A future Swift enhancement might let you force an app call to run on a specific site via a settable parameter. > > Depending on what you are trying to vary between sites, you might be able to do something clever by varying an environment variable within a single app function definition. I'll look for that info and post a pointer. > > - Mike > > > >> Thanks a lot, >> >> Lorenzo >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > From lpesce at uchicago.edu Thu Mar 29 21:03:48 2012 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Thu, 29 Mar 2012 21:03:48 -0500 Subject: [Swift-user] Question about mappers Message-ID: Hi all-- Thanks a lot for your help. I have another question. This is what my problem looks like. There is a set of input files, all with the same name (say X.mat), in a set of subdirectories of say : Win1 Win2 Win3 . . . WinN The applications takes the input file as *filename* (string) and not a handle, then some other parameters and produces a couple of output files (see below for app demoreal). For each directory, say Win1, each file X.mat is mapped to many output files result_neuron__ht_.mat, where i and j are a parameter sweep (same for all windows, but the X.mat files are different). My script pokes around, takes some input and then spews up a swift script. My mapping misfires and all the output files end up in , which doesn't work because many should have the same name. If the input file were a handle, I would use a file type and map input and output and I think that I would know how to do that. Here I have no idea. // file to run the Margic square example from the matlab web site type file; string MCRPath = "/soft/matlab/7.13"; string spikefilename = "$infile"; app (file outdata,file result) demoreal (string mcr, int chn, int htin, string datafile) { runDemoReal mcr chn htin datafile stdout=@outdata; } string WinDirs[] = ${swiftWinDirs}; int htvec[] = ${htvec}; int chnvec[] = ${chnvec}; foreach win in WinDirs { string spikefile = @strcat("${currdir}/",win,spikefilename); foreach chn in chnvec { foreach s, i in htvec { file result ; file LogDemoReal ; (LogDemoReal,result) = demoreal (MCRPath, chn, s, spikefile); } } } From wilde at mcs.anl.gov Thu Mar 29 21:46:49 2012 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 29 Mar 2012 21:46:49 -0500 (CDT) Subject: [Swift-user] Question about mappers In-Reply-To: Message-ID: <1495959515.119317.1333075609420.JavaMail.root@zimbra.anl.gov> Lorenzo, I think your outputs are all going to the current working directory in which you run the swift command, because when you map the output files in these statements, you are not specifying any directory in the filename: file result ; file LogDemoReal ; If you want these files to get created in one of the WinDirs directories, you should prepend win,"/" in front: file result From: "Lorenzo Pesce" > To: swift-user at ci.uchicago.edu > Sent: Thursday, March 29, 2012 9:03:48 PM > Subject: [Swift-user] Question about mappers > Hi all-- > Thanks a lot for your help. I have another question. > > This is what my problem looks like. There is a set of input files, all > with the same name (say X.mat), in a set of subdirectories of say > : > Win1 > Win2 > Win3 > . > . > . > WinN > > > > The applications takes the input file as *filename* (string) and not a > handle, then some other parameters and produces a couple of output > files (see below for app demoreal). > > For each directory, say Win1, each file X.mat is mapped to many output > files result_neuron__ht_.mat, where i and j are a parameter > sweep (same for all windows, but the X.mat files are different). > > My script pokes around, takes some input and then spews up a swift > script. My mapping misfires and all the output files end up in > , which doesn't work because many should have the same name. > If the input file were a handle, I would use a file type and map input > and output and I think that I would know how to do that. Here I have > no idea. > > // file to run the Margic square example from the matlab web site > type file; > > string MCRPath = "/soft/matlab/7.13"; > string spikefilename = "$infile"; > > app (file outdata,file result) demoreal (string mcr, int chn, int > htin, string datafile) > { > runDemoReal mcr chn htin datafile stdout=@outdata; > } > > string WinDirs[] = ${swiftWinDirs}; > int htvec[] = ${htvec}; > int chnvec[] = ${chnvec}; > > > foreach win in WinDirs { > string spikefile = @strcat("${currdir}/",win,spikefilename); > foreach chn in chnvec { > foreach s, i in htvec { > file result "_ht_",s,".mat")>; > file LogDemoReal "_ht_",s,".log")>; > (LogDemoReal,result) = demoreal (MCRPath, chn, s, spikefile); > } > } > } > > > > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory From lpesce at uchicago.edu Fri Mar 30 08:56:35 2012 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Fri, 30 Mar 2012 08:56:35 -0500 Subject: [Swift-user] Question about mappers In-Reply-To: <1495959515.119317.1333075609420.JavaMail.root@zimbra.anl.gov> References: <1495959515.119317.1333075609420.JavaMail.root@zimbra.anl.gov> Message-ID: <23510A61-DB69-4F91-B0E3-082BA2ECA44F@uchicago.edu> > file result ; > file LogDemoReal ; > > If you want these files to get created in one of the WinDirs directories, you should prepend win,"/" in front: > > file result > Was that what you intended? It would be, but if I add it, it does this Exception in runDemoReal: Arguments: [/soft/matlab/7.13, 1.0, 3.0, /lustre/beagle/GCNet/grasping/200ms_5windows_new/Win7/X.mat] Host: pbs Directory: demo_real-20120330-1352-71yp4ivb/jobs/x/runDemoReal-xe0j37pk stderr.txt: stdout.txt: ---- Caused by: The following output files were not created by the application: Win7/result_neuron_1_ht_3.mat because (I think) the script generates the files in its own run directory. Originally the script has a cd to Win# and then ran the code. > Also, there should be no need to generate the Swift script with shell variable substitution. You can pass in values from the shell script via swift command line and access them from the Swift script with the @arg() function. > > You can make htvec and chnvec string vectors instead of ints, and pass them like so: > > swift -tc.file etc etc myscript.swift -htvec=1,3,5 -chnvec=a,b,c -currdir=$PWD > > and in the script do: > > string htvec[]=@strsplit(@arg("htvec")); > > You can use the strng htvec[] in your later @strcat() call in the same way you used the int version of that vector. Thanks. > > - Mike > > > > > > ----- Original Message ----- >> From: "Lorenzo Pesce" >> To: swift-user at ci.uchicago.edu >> Sent: Thursday, March 29, 2012 9:03:48 PM >> Subject: [Swift-user] Question about mappers >> Hi all-- >> Thanks a lot for your help. I have another question. >> >> This is what my problem looks like. There is a set of input files, all >> with the same name (say X.mat), in a set of subdirectories of say >> : >> Win1 >> Win2 >> Win3 >> . >> . >> . >> WinN >> >> >> >> The applications takes the input file as *filename* (string) and not a >> handle, then some other parameters and produces a couple of output >> files (see below for app demoreal). >> >> For each directory, say Win1, each file X.mat is mapped to many output >> files result_neuron__ht_.mat, where i and j are a parameter >> sweep (same for all windows, but the X.mat files are different). >> >> My script pokes around, takes some input and then spews up a swift >> script. My mapping misfires and all the output files end up in >> , which doesn't work because many should have the same name. >> If the input file were a handle, I would use a file type and map input >> and output and I think that I would know how to do that. Here I have >> no idea. >> >> // file to run the Margic square example from the matlab web site >> type file; >> >> string MCRPath = "/soft/matlab/7.13"; >> string spikefilename = "$infile"; >> >> app (file outdata,file result) demoreal (string mcr, int chn, int >> htin, string datafile) >> { >> runDemoReal mcr chn htin datafile stdout=@outdata; >> } >> >> string WinDirs[] = ${swiftWinDirs}; >> int htvec[] = ${htvec}; >> int chnvec[] = ${chnvec}; >> >> >> foreach win in WinDirs { >> string spikefile = @strcat("${currdir}/",win,spikefilename); >> foreach chn in chnvec { >> foreach s, i in htvec { >> file result > "_ht_",s,".mat")>; >> file LogDemoReal > "_ht_",s,".log")>; >> (LogDemoReal,result) = demoreal (MCRPath, chn, s, spikefile); >> } >> } >> } >> >> >> >> >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > From wilde at mcs.anl.gov Fri Mar 30 11:14:50 2012 From: wilde at mcs.anl.gov (Michael Wilde) Date: Fri, 30 Mar 2012 11:14:50 -0500 (CDT) Subject: [Swift-user] Question about mappers In-Reply-To: <23510A61-DB69-4F91-B0E3-082BA2ECA44F@uchicago.edu> Message-ID: <902912457.119834.1333124090212.JavaMail.root@zimbra.anl.gov> OK, I think I'll need to look at the code more closely and maybe get more info. I think youre on the right track, in terms of looking at the run directory issues. I suspect this can be further simplified in the process of debugging it. Thats all I can say for now; I will need to spend more time studying your example later. - Mike ----- Original Message ----- > From: "Lorenzo Pesce" > To: "Michael Wilde" > Cc: swift-user at ci.uchicago.edu > Sent: Friday, March 30, 2012 8:56:35 AM > Subject: Re: [Swift-user] Question about mappers > > file result > "_ht_",s,".mat")>; > > file LogDemoReal > "_ht_",s,".log")>; > > > > If you want these files to get created in one of the WinDirs > > directories, you should prepend win,"/" in front: > > > > file result > file=@strcat(win,"/"."result_neuron_",... > > > > Was that what you intended? > > It would be, but if I add it, it does this > > Exception in runDemoReal: > Arguments: [/soft/matlab/7.13, 1.0, 3.0, > /lustre/beagle/GCNet/grasping/200ms_5windows_new/Win7/X.mat] > Host: pbs > Directory: > demo_real-20120330-1352-71yp4ivb/jobs/x/runDemoReal-xe0j37pk > stderr.txt: > stdout.txt: > > ---- > > Caused by: The following output files were not created by the > application: Win7/result_neuron_1_ht_3.mat > > because (I think) the script generates the files in its own run > directory. > Originally the script has a cd to Win# and then ran the code. > > > > > Also, there should be no need to generate the Swift script with > > shell variable substitution. You can pass in values from the shell > > script via swift command line and access them from the Swift script > > with the @arg() function. > > > > You can make htvec and chnvec string vectors instead of ints, and > > pass them like so: > > > > swift -tc.file etc etc myscript.swift -htvec=1,3,5 -chnvec=a,b,c > > -currdir=$PWD > > > > and in the script do: > > > > string htvec[]=@strsplit(@arg("htvec")); > > > > You can use the strng htvec[] in your later @strcat() call in the > > same way you used the int version of that vector. > > Thanks. > > > > > > - Mike > > > > > > > > > > > > ----- Original Message ----- > >> From: "Lorenzo Pesce" > >> To: swift-user at ci.uchicago.edu > >> Sent: Thursday, March 29, 2012 9:03:48 PM > >> Subject: [Swift-user] Question about mappers > >> Hi all-- > >> Thanks a lot for your help. I have another question. > >> > >> This is what my problem looks like. There is a set of input files, > >> all > >> with the same name (say X.mat), in a set of subdirectories of say > >> : > >> Win1 > >> Win2 > >> Win3 > >> . > >> . > >> . > >> WinN > >> > >> > >> > >> The applications takes the input file as *filename* (string) and > >> not a > >> handle, then some other parameters and produces a couple of output > >> files (see below for app demoreal). > >> > >> For each directory, say Win1, each file X.mat is mapped to many > >> output > >> files result_neuron__ht_.mat, where i and j are a parameter > >> sweep (same for all windows, but the X.mat files are different). > >> > >> My script pokes around, takes some input and then spews up a swift > >> script. My mapping misfires and all the output files end up in > >> , which doesn't work because many should have the same > >> name. > >> If the input file were a handle, I would use a file type and map > >> input > >> and output and I think that I would know how to do that. Here I > >> have > >> no idea. > >> > >> // file to run the Margic square example from the matlab web site > >> type file; > >> > >> string MCRPath = "/soft/matlab/7.13"; > >> string spikefilename = "$infile"; > >> > >> app (file outdata,file result) demoreal (string mcr, int chn, int > >> htin, string datafile) > >> { > >> runDemoReal mcr chn htin datafile stdout=@outdata; > >> } > >> > >> string WinDirs[] = ${swiftWinDirs}; > >> int htvec[] = ${htvec}; > >> int chnvec[] = ${chnvec}; > >> > >> > >> foreach win in WinDirs { > >> string spikefile = @strcat("${currdir}/",win,spikefilename); > >> foreach chn in chnvec { > >> foreach s, i in htvec { > >> file result >> chn, > >> "_ht_",s,".mat")>; > >> file LogDemoReal >> chn, > >> "_ht_",s,".log")>; > >> (LogDemoReal,result) = demoreal (MCRPath, chn, s, spikefile); > >> } > >> } > >> } > >> > >> > >> > >> > >> > >> _______________________________________________ > >> Swift-user mailing list > >> Swift-user at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > > > -- > > Michael Wilde > > Computation Institute, University of Chicago > > Mathematics and Computer Science Division > > Argonne National Laboratory > > -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory