From mickelso at mcs.anl.gov Tue Dec 3 07:22:28 2013 From: mickelso at mcs.anl.gov (Sheri Mickelson) Date: Tue, 3 Dec 2013 07:22:28 -0600 Subject: [Swift-user] Absolute Path Question Message-ID: <3034C734-53F3-4777-A5AE-638CA92F4D41@mcs.anl.gov> I modified the swift "hello world" example to include an absolute path: ================================= type messagefile; app (messagefile t) greeting () { echo "Hello, world!" stdout=@filename(t); } message file outfile <"/homes/user/test/my/output/is/here/hello.txt">; outfile = greeting(); ================================= I noticed that if "/homes/user/test/my/output/is/here/" doesn't exist, the directory is created for you. Is there a Swift option to override this and have the swift code fail if "/homes/user/test/my/out/is/here/" doesn't exist? Thanks, Sheri From iraicu at cs.iit.edu Sat Dec 7 13:40:18 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Sat, 07 Dec 2013 13:40:18 -0600 Subject: [Swift-user] CFP: The 23nd International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC) 2014 Message-ID: <52A379A2.9090901@cs.iit.edu> **** CALL FOR PAPERS **** The 23nd International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC-2014) Vancouver, Canada - June 23-27, 2014 http://www.hpdc.org/2014 The ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) is the premier annual conference for presenting the latest research on the design, implementation, evaluation, and application of parallel and distributed systems for high-end computing. In 2014, the 23nd HPDC and affiliated workshops will take place in the beautiful city of Vancouver, Canada during June 23-27, 2014. **** IMPORTANT DATES **** Abstracts Due: January 20, 2014 Papers Due: January 27, 2014 (no extensions) Author Rebuttal: March 14-18, 2014 Author Notifications: March 28, 2014 **** SCOPE AND TOPICS **** Submissions are welcomed on high-performance parallel and distributed computing topics including but not limited to: clouds, clusters, grids, Big Data, massively multicore, and global-scale computing systems. New scholarly research showing empirical and reproducible results in architectures, systems, and networks is strongly encouraged, as are experience reports of operational deployments that can provide insights for future research on HPDC applications and systems. All papers will be evaluated for their originality, technical depth and correctness, potential impact, relevance to the conference, and quality of presentation. Research papers must clearly demonstrate research contributions and novelty, while experience reports must clearly describe lessons learned and demonstrate impact. In the context of high-performance parallel and distributed computing, the topics of interest include, but are not limited to: o Systems, networks, and architectures for high-end computing o Massively multicore systems o Resource virtualization o Programming languages and environments o I/O, storage systems, and data management o Resource management and scheduling, including energy-aware techniques o Performance modeling and analysis o Fault tolerance, reliability, and availability o Data-intensive computing o Applications of parallel and distributed computing **** PAPER SUBMISSION GUIDELINES **** Authors are invited to submit technical papers of at most 12 pages in PDF format, including figures and references. Papers should be formatted in the ACM Proceedings Style (http://www.acm.org/sigs/publications/proceedings-templates) and submitted via the conference web site. No changes to the margins, spacing, or font sizes as specified by the style file are allowed. Accepted papers will appear in the conference proceedings, and will be incorporated into the ACM Digital Library. A limited number of papers will be accepted as posters. Papers must be self-contained and provide the technical substance required for the program committee to evaluate their contributions. Papers should thoughtfully address all related work, particularly work presented at previous HPDC events. Submitted papers must be original work that has not appeared in and is not under consideration for another conference or a journal. See the ACM Prior Publication Policy (http://www.acm.org/publications/policies/sim_submissions) for more details. **** HPDC'14 GENERAL CO-CHAIRS **** Beth Plale, Indiana University Matei Ripeanu, University of British Columbia **** HPDC'14 PROGRAM CO-CHAIRS **** Franck Cappello, Argonne National Lab and INRIA Dongyan Xu, Purdue University **** HPDC'14 PUBLICITY CHAIR **** Ioan Raicu, Illinois Institute of Technology, USA **** HPDC'14 PROGRAM COMMITTEE **** David Abramson, University of Queensland, Australia Gabriel Antoniu, INRIA, France Henri Bal, Vrije Universiteit, the Netherlands Adam Barker, University of St Andrews, UK Michela Becchi, University of Missouri - Columbia, USA Greg Bronevetsky, Lawrence Livermore National Laboratory, USA Ali Butt, Virginia Tech, USA Abhishek Chandra, University of Minnesota, USA Andrew Chien, University of Chicago and Argonne National Laboratory, USA Paolo Costa, Microsoft Research Cambridge, UK Dilma Da Silva, Qualcomm Research Silicon Valley, USA Peter Dinda, Northwestern University, USA Dick Epema, Delft and Eindhoven University of Technology, the Netherlands Gilles Fedak, INRIA, France Wu-chun Feng, Virginia Tech, USA Renato Figueiredo, University of Florida, USA Ian Foster, University of Chicago and Argonne National Laboratory, USA Kartik Gopalan, Binghamton University, USA Haryadi Gunawi, University of Chicago, USA Bingsheng He, Nanyang Technological University, Singapore Alexandru Iosup, Delft University of Technology, the Netherlands Shantenu Jha, Rutgers University, USA Hai Jin Huazhong, University of Science and Technology, China Larry Kaplan, Cray, Inc., USA Kate Keahey, Argonne National Laboratory, USA Thilo Kielmann, Vrije Universiteit, the Netherlands Charles Killian, Google, USA Zhiling Lan, Illinois Institute of Technology, USA John (Jack) Lange, University of Pittsburgh, USA Minglu Li, Shanghai Jiaotong University, China Barney Maccabe, Oak Ridge National Laboratory, USA Naoya Maruyama, RIKEN Advanced Institute for Computational Science, Japan Satoshi Matsuoka, Tokyo Institute of Technology, Japan Dejan Milojicic, HP Labs, USA Bogdan Nicolae, IBM Research, Ireland Sangmi Pallickara, Colorado State University, USA Manish Parashar, Rutgers University, USA Ioan Raicu, Illinois Institute of Technology, USA Prasenjit Sarkar, IBM Research, USA Daniele Scarpazza, D.E. Shaw Research, USA Karsten Schwan, Georgia Institute of Technology, USA Martin Swany, Indiana University, USA Michela Taufer, University of Delaware, USA Kenjiro Taura, University of Tokyo, Japan Douglas Thain, University of Notre Dame, USA Bhuvan Urgaonkar, Pennsylvania State University, USA Mustafa Uysal, VMware, Inc., USA Jon Weissman, University of Minnesota, USA Ming Zhao, Florida International University, USA **** HPDC STEERING COMMITTEE **** Henri Bal, Vrije Universiteit Andrew A. Chien, University of Chicago Peter Dinda, Northwestern University Dick Epema, Delft University of Technology Renato Figueiredo, University of Florida Ian Foster, Univ. of Chicago and Argonne National Laboratory Salim Hariri, University of Arizona Thilo Kielmann, Vrije Universiteit Amsterdam Arthur "Barney" Maccabe, Oak Ridge National Laboratory Manish Parashar, Rutgers University Matei Ripeanu, University of British Columbia Karsten Schwan, Georgia Tech Doug Thain, University of Notre Dame Jon Weissman, University of Minnesota (Chair) -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer Cluster, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= From iraicu at cs.iit.edu Sat Dec 7 13:47:11 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Sat, 07 Dec 2013 13:47:11 -0600 Subject: [Swift-user] Call for Workshops: ACM HPDC 2014 -- Proposals due December 9th, 2013 Message-ID: <52A37B3F.1010004@cs.iit.edu> The organizers of the /23rd International ACM Symposium on High-Performance Parallel and Distributed Computing/ (HPDC'14) *call for proposals for workshops* to be held with HPDC'14. The workshops will be held on June 23-27, 2014. Workshops should provide forums for discussion among researchers and practitioners on focused topics or emerging research areas relevant to the HPDC community. Organizers may structure workshops as they see fit, including invited talks, panel discussions, presentations of work in progress, fully peer-reviewed papers, or some combination. Workshops could be scheduled for half a day or a full day, depending on interest, space constraints, and organizer preference. Organizers should design workshops for approximately 20-40 participants, to balance impact and effective discussion. *Workshop proposal* will be uploaded in PDF format to the submission site by the deadline, and should include: * The name and acronym of the workshop * A description (0.5-1 page) of the theme of the workshop * A description (one paragraph) of the relation between the theme of the workshop and of HPDC * A list of topics of interest * The names and affiliations of the workshop organizers, and if applicable, of a significant portion of the program committee * A description of the expected structure of the workshop (papers, invited talks, panel discussions, etc.) * Data about previous offerings of the workshop (if any), including the attendance, the numbers of papers or presentations submitted and accepted, and the links to the corresponding websites * A publicity plan for attracting submissions and attendees. Please also include expected number of submissions, accepted papers, and attendees that you anticipate for a successful workshop. Due to publication deadlines, workshops must operate within roughly the following timeline: papers due mid February (2-3 weeks after the HPDC deadline), and selected and sent to the publisher by mid April. Workshop Chair: Martin Swany, Indiana University, USA * Questions: * For any questions regarding your submission, please contact the workshop chair at: hpdc-workshop-l [at] list [dot] indiana [dot] edu Important dates: *Workshop Proposals Due: * *December 9, 2013* Notifications: December 20, 2014 Workshop CFPs Online and Distributed: January 13, 2014 -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer Cluster, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= -------------- next part -------------- An HTML attachment was scrubbed... URL: From marialemos72 at gmail.com Fri Dec 13 19:11:38 2013 From: marialemos72 at gmail.com (WorldCIST) Date: Sat, 14 Dec 2013 01:11:38 +0000 Subject: [Swift-user] Last Call - 2sd World Conference on IST; Submission: December 29 Message-ID: <20131214011151.781DD7CC0A4@mailrelay.anl.gov> ============================================================================ 2sd World Conference on Information Systems and Technologies - WorldCIST'14 April 15-18, 2014, Madeira Island, Portugal http://www.aisti.eu/worldcist14/ ================================ Submission deadline: December 29 * Proceedings published by Springer. ** Papers submitted for indexation by ISI, SCOPUS, DBLP, etc. *** Extended versions of best papers published in ISI/SCI/SSCI/JCR journals. Regards, Maria Lemos WorldCIST'14 http://www.aisti.eu/worldcist14/ From madduri at mcs.anl.gov Wed Dec 18 16:26:58 2013 From: madduri at mcs.anl.gov (Ravi K Madduri) Date: Wed, 18 Dec 2013 16:26:58 -0600 Subject: [Swift-user] Swift log parser Message-ID: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> Hi I wonder if you guys have any scripts that help me parse swift logs and provide me with information about how many jobs launched, how long they ran etc. Thanks -- Ravi K Madduri MCS, Argonne National Laboratory Computation Institute, University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidk at ci.uchicago.edu Wed Dec 18 16:58:05 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Wed, 18 Dec 2013 16:58:05 -0600 Subject: [Swift-user] Swift log parser In-Reply-To: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> References: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> Message-ID: Hey Ravi, Which version of Swift are you using? I just wrote something like this yesterday that parses a Swift log and prints information about jobs in a clear format (start time, stop time, files staged in and out, etc). But it was written for 0.95 log files.. not sure it will work with 0.94 or older logs. There are other scripts available to get generate of # of active tasks and # of tasks completed if that would be helpful? On Wed, Dec 18, 2013 at 4:26 PM, Ravi K Madduri wrote: > Hi > I wonder if you guys have any scripts that help me parse swift logs and > provide me with information about how many jobs launched, how long they ran > etc. > > Thanks > -- > Ravi K Madduri > MCS, Argonne National Laboratory > Computation Institute, University of Chicago > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From madduri at mcs.anl.gov Wed Dec 18 17:40:00 2013 From: madduri at mcs.anl.gov (Ravi K Madduri) Date: Wed, 18 Dec 2013 17:40:00 -0600 Subject: [Swift-user] Swift log parser In-Reply-To: References: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> Message-ID: <4B59F22F-335E-43DA-B6F0-2149D27AE6D6@mcs.anl.gov> David This is the swift version on cox globus genomics instance: madduri at cox2:/nfs/software/galaxy/tools$ sudo /mnt/galaxyTools/tools/swift/swift-trunk/bin/swift -version Swift trunk swift-r6209 (swift modified locally) cog-r3574 (cog modified locally) I am just trying to get some aggregate details on the number of jobs run and how long they took, number of nodes spawned etc. On Dec 18, 2013, at 4:58 PM, David Kelly wrote: > Hey Ravi, > > Which version of Swift are you using? I just wrote something like this yesterday that parses a Swift log and prints information about jobs in a clear format (start time, stop time, files staged in and out, etc). But it was written for 0.95 log files.. not sure it will work with 0.94 or older logs. > > There are other scripts available to get generate of # of active tasks and # of tasks completed if that would be helpful? > > > On Wed, Dec 18, 2013 at 4:26 PM, Ravi K Madduri wrote: > Hi > I wonder if you guys have any scripts that help me parse swift logs and provide me with information about how many jobs launched, how long they ran etc. > > Thanks > -- > Ravi K Madduri > MCS, Argonne National Laboratory > Computation Institute, University of Chicago > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -- Ravi K Madduri MCS, Argonne National Laboratory Computation Institute, University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidk at ci.uchicago.edu Thu Dec 19 09:41:17 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Thu, 19 Dec 2013 09:41:17 -0600 Subject: [Swift-user] Swift log parser In-Reply-To: <4B59F22F-335E-43DA-B6F0-2149D27AE6D6@mcs.anl.gov> References: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> <4B59F22F-335E-43DA-B6F0-2149D27AE6D6@mcs.anl.gov> Message-ID: Hey Ravi, Lorenzo, I modified the script so it should work now with logs generated by 0.94. You can run the attached script as: $ ./swiftlog /path/to/ A version of this will be included with 0.95 when it's released, I'll send some more details about that soon. Hope this is helpful - let me know if you run into any problems. On Wed, Dec 18, 2013 at 5:40 PM, Ravi K Madduri wrote: > David > This is the swift version on cox globus genomics instance: > > madduri at cox2:/nfs/software/galaxy/tools$ sudo > /mnt/galaxyTools/tools/swift/swift-trunk/bin/swift -version > Swift trunk swift-r6209 (swift modified locally) cog-r3574 (cog modified > locally) > > I am just trying to get some aggregate details on the number of jobs run > and how long they took, number of nodes spawned etc. > > > On Dec 18, 2013, at 4:58 PM, David Kelly wrote: > > Hey Ravi, > > Which version of Swift are you using? I just wrote something like this > yesterday that parses a Swift log and prints information about jobs in a > clear format (start time, stop time, files staged in and out, etc). But it > was written for 0.95 log files.. not sure it will work with 0.94 or older > logs. > > There are other scripts available to get generate of # of active tasks and > # of tasks completed if that would be helpful? > > > On Wed, Dec 18, 2013 at 4:26 PM, Ravi K Madduri wrote: > >> Hi >> I wonder if you guys have any scripts that help me parse swift logs and >> provide me with information about how many jobs launched, how long they ran >> etc. >> >> Thanks >> -- >> Ravi K Madduri >> MCS, Argonne National Laboratory >> Computation Institute, University of Chicago >> >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >> > > > -- > Ravi K Madduri > MCS, Argonne National Laboratory > Computation Institute, University of Chicago > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: swiftlog Type: application/octet-stream Size: 2979 bytes Desc: not available URL: From ketan at mcs.anl.gov Thu Dec 19 14:55:10 2013 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Thu, 19 Dec 2013 14:55:10 -0600 Subject: [Swift-user] Swift log parser In-Reply-To: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> References: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> Message-ID: Hi Ravi, As we discussed, please find a plotting script I wrote a while ago. It is tested on linux platform and uses gnuplot to plot from the swift run log. To run, do: $ plotswiftlogs.ketan It should produce two plots as png files which are active and cumulative tasks plots over the duration of Swift run. Please note that since the plot uses the standard Swift progress log, the accuracy might be off by a second or two. We are currently working on a high fidelity plotting routine for Swift runs. Best, Ketan On Wed, Dec 18, 2013 at 4:26 PM, Ravi K Madduri wrote: > Hi > I wonder if you guys have any scripts that help me parse swift logs and > provide me with information about how many jobs launched, how long they ran > etc. > > Thanks > -- > Ravi K Madduri > MCS, Argonne National Laboratory > Computation Institute, University of Chicago > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: plotswiftlogs.ketan Type: application/octet-stream Size: 1837 bytes Desc: not available URL: From madduri at mcs.anl.gov Thu Dec 19 15:00:33 2013 From: madduri at mcs.anl.gov (Ravi K Madduri) Date: Thu, 19 Dec 2013 15:00:33 -0600 Subject: [Swift-user] Swift log parser In-Reply-To: References: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> Message-ID: When I run it, I get a png thats 0bytes in size. On Dec 19, 2013, at 2:55 PM, Ketan Maheshwari wrote: > Hi Ravi, > > As we discussed, please find a plotting script I wrote a while ago. It is tested on linux platform and uses gnuplot to plot from the swift run log. > > To run, do: > > $ plotswiftlogs.ketan > > It should produce two plots as png files which are active and cumulative tasks plots over the duration of Swift run. Please note that since the plot uses the standard Swift progress log, the accuracy might be off by a second or two. We are currently working on a high fidelity plotting routine for Swift runs. > > Best, > Ketan > > > On Wed, Dec 18, 2013 at 4:26 PM, Ravi K Madduri wrote: > Hi > I wonder if you guys have any scripts that help me parse swift logs and provide me with information about how many jobs launched, how long they ran etc. > > Thanks > -- > Ravi K Madduri > MCS, Argonne National Laboratory > Computation Institute, University of Chicago > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > -- Ravi K Madduri MCS, Argonne National Laboratory Computation Institute, University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From madduri at mcs.anl.gov Thu Dec 19 15:33:17 2013 From: madduri at mcs.anl.gov (Ravi K Madduri) Date: Thu, 19 Dec 2013 15:33:17 -0600 Subject: [Swift-user] Swift log parser In-Reply-To: References: <9734F12B-D4D6-41C7-924B-BB58B0025B34@mcs.anl.gov> Message-ID: Ketan helped me with this and it works now. Find attached the plots we were able to generate On Dec 19, 2013, at 3:00 PM, Ravi K Madduri wrote: > When I run it, I get a png thats 0bytes in size. > On Dec 19, 2013, at 2:55 PM, Ketan Maheshwari wrote: > >> Hi Ravi, >> >> As we discussed, please find a plotting script I wrote a while ago. It is tested on linux platform and uses gnuplot to plot from the swift run log. >> >> To run, do: >> >> $ plotswiftlogs.ketan >> >> It should produce two plots as png files which are active and cumulative tasks plots over the duration of Swift run. Please note that since the plot uses the standard Swift progress log, the accuracy might be off by a second or two. We are currently working on a high fidelity plotting routine for Swift runs. >> >> Best, >> Ketan >> >> >> On Wed, Dec 18, 2013 at 4:26 PM, Ravi K Madduri wrote: >> Hi >> I wonder if you guys have any scripts that help me parse swift logs and provide me with information about how many jobs launched, how long they ran etc. >> >> Thanks >> -- >> Ravi K Madduri >> MCS, Argonne National Laboratory >> Computation Institute, University of Chicago >> >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >> >> > > -- > Ravi K Madduri > MCS, Argonne National Laboratory > Computation Institute, University of Chicago > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -- Ravi K Madduri MCS, Argonne National Laboratory Computation Institute, University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cumulativeplot.png Type: image/png Size: 4784 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: activeplot.png Type: image/png Size: 5771 bytes Desc: not available URL: