From marialemos72 at gmail.com Sun Feb 3 07:21:18 2013 From: marialemos72 at gmail.com (Maria Lemos) Date: Sun, 3 Feb 2013 13:21:18 +0000 Subject: [Swift-devel] CISTI'2013 Doctoral Symposium - CFP, Lisbon; Deadline: February 15 Message-ID: <20130203132010.0F9F67CC08E@mailrelay.anl.gov> *************************************************************************************************** CISTI'2013 DOCTORAL SYMPOSIUM 8th Iberian Conference on Information Systems and Technologies Lisbon, Portugal, June 19 - 23, 2013 http://www.aisti.eu/cisti2013/index.php?option=com_content&view=article&id=64&Itemid=68&lang=en *************************************************************************************************** INTRODUCTION The purpose of CISTI'2013?s Doctoral Symposium (http://www.aisti.eu/cisti2013/index.php?option=com_content&view=article&id=64&Itemid=68&lang=en) is to provide graduate students a setting where they can, informally, expose and discuss their work, collecting valuable expert opinions and sharing new ideas, methods and applications. The Doctoral Symposium is an excellent opportunity for PhD students to present and discuss their work in a Workshop format. Each presentation will be evaluated by a panel composed by at least three Information Systems and Technologies experts. CONTRIBUTIONS SUBMISSION The Doctoral Symposium is opened to PhD students whose research area includes the themes proposed for this Conference. Submissions must include an extended abstract (maximum 4 pages), following the Conference style guide. All selected contributions will be handed out along with the Conference Proceedings, in CD with an ISBN. These contributions will be send for indexation by EBSCO, and EI-Compendex. Submissions must include the field, the PhD institution and the number of months devoted to the development of the work. Additionally, they should include in a clear and succinct manner: ? The problem approached and its significance or relevance ? The research objectives and related investigation topics ? A brief display of what is already known ? A proposed solution methodology for the problem ? Expected results IMPORTANT DATES ? Proposal submission deadline: 15th of February 2013 ? Acceptance notification: 29th of March 2013 ? Final submission deadline: 12th of April 2013 ? Registration payment, to ensure the inclusion of the accepted contribution in the Conference Proceedings: 12th of April 2013 SCIENTIFIC AND ORGANIZING COMMITTEE Manuel P?rez Cota, Universidad de Vigo (Chair) Adolfo Lozano Tello, Universidad de Extremadura Alberto J. Bugar?n Diz, Universidad de Santiago de Compostela ?lvaro Rocha, Universidade do Porto, LIACC Ana Maria Ramalho Correia, Universidade Nova de Lisboa, ISEGI Ant?nio Palma dos Reis, Universidade T?cnica de Lisboa, ISEG Ant?nio Pereira, Instituto Polit?cnico de Leiria Arturo Mendez Pen?n, Universidade de Vigo Carlos Ferr?s Sexto, Universidad de Santiago de Compostela David Fonseca, Universidad Ram?n Llul Ernesto Redondo, Universidad Polit?cnica de Catalu?a Feliz Gouveia, Universidade Fernando Pessoa Francisco Restivo, Universidade Cat?lica Portuguesa - Braga Guilhermina Miranda, Universidade de Lisboa Gonzalo Cuevas Agust?n, Universidad Polit?cnica de Madrid H?ctor Jorge Garc?a Neder, Universidad Tecnol?gica NacioNal Jo?o ?lvaro Carvalho, Universidade do Minho Jo?o Barroso, Universidade de Tr?s-os-Montes e Alto Douro J?rg Thomaschewski, University of Applied Sciences of Emden-Leer Jos? Antonio Calvo-Manzano Villal?n, Universidad Polit?cnica de Madrid Jos? Bulas Cruz, Universidade de Tr?s-os-Montes e Alto Douro Jos? Tribolet, Universidade T?cnica de Lisboa, IST Leandro Rodr?guez Li?ares, Universidade de Vigo Lu?s Paulo Reis, Universidade do Minho Mar?a Jos? Lado Touri?o, Universidade de Vigo Maria Manuela Cruz Cunha, Instituto Polit?cnico do C?vado e do Ave Marco Painho, Universidade Nova de Lisboa, ISEGI Mario Alberto Groppo, Universidad Tecnol?gica Nacional Nuno Ribeiro, Universidade Fernando Pessoa Pilar Mareca, Universidade Polit?cnica de Madrid Ramiro Gon?alves, Universidade de Tr?s-os-Montes e Alto Douro Tomas San Feliu Gilabert, Universidad Polit?cnica de Madrid Vicente Alcober, Universidad Polit?cnica de Madrid - CISTI'2013 Team http://www.aisti.eu/cisti2013 cistimail at gmail.com ------------------- From davidk at ci.uchicago.edu Thu Feb 7 20:10:43 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Thu, 7 Feb 2013 20:10:43 -0600 (CST) Subject: [Swift-devel] Work directory on a non-shared filesystem In-Reply-To: <1011613791.3988385.1360287513115.JavaMail.root@ci.uchicago.edu> Message-ID: <147907730.3992522.1360289443800.JavaMail.root@ci.uchicago.edu> Is it possible to have a work directory on a non-shared filesystem (/tmp) without using provider staging? I'm not staging any files in or out with swift, just passing strings. When I try this, I get an 127 app failure for some reason (the app is /bin/bash). I've been able to set the work directory to a shared filesystem, and set the scratch tag to /tmp. That combination seems to work fine. But ideally I'd like to avoid the shared filesystem as much as possible. It also seems to work fine with /tmp as the work directory and with provider staging enabled. Just curious if this configuration is possible somehow. Thanks! David Michael Kelly Systems Programmer University of Chicago Computation Institute 5735 S. Ellis Ave. Chicago, IL 60637 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Thu Feb 7 21:43:02 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 07 Feb 2013 19:43:02 -0800 Subject: [Swift-devel] Work directory on a non-shared filesystem In-Reply-To: <147907730.3992522.1360289443800.JavaMail.root@ci.uchicago.edu> References: <147907730.3992522.1360289443800.JavaMail.root@ci.uchicago.edu> Message-ID: <1360294982.30249.10.camel@echo> _swiftwrap needs to make it to the compute nodes somehow. The scenarios you list below make sense: 1. work dir on shared fs, scratch on /tmp: _swiftwrap is accessed through the shared fs so it works. 2. provider staging with workdir and scratch on /tmp: _swiftwrap is staged in by coasters, so that works too. 3. no provider staging, workdir and scratch on /tmp: _swiftwrap has no way of making it to the compute node, so 127 (command/function not found - see COMMAND EXECUTION section of man bash). That's standard stuff. A hack would be to install _swiftwrap along with the app and convince swift to run that instead of its own swiftwrap. That might require hacking vdl-int.k, around line 502. Let me know if you want to go that route, and we can chat on how to get it done. Mihael On Thu, 2013-02-07 at 20:10 -0600, David Kelly wrote: > Is it possible to have a work directory on a non-shared filesystem > (/tmp) without using provider staging? I'm not staging any files in or > out with swift, just passing strings. When I try this, I get an 127 > app failure for some reason (the app is /bin/bash). > > > I've been able to set the work directory to a shared filesystem, and > set the scratch tag to /tmp. That combination seems to work fine. But > ideally I'd like to avoid the shared filesystem as much as possible. > > > It also seems to work fine with /tmp as the work directory and with provider staging enabled. > > > Just curious if this configuration is possible somehow. Thanks! > > > > > > > > > > > David Michael Kelly > Systems Programmer > University of Chicago Computation Institute > 5735 S. Ellis Ave. Chicago, IL 60637 > Is it possible to have a work directory on a non-shared filesystem > (/tmp) without using provider staging? I'm not staging any files in or > out with swift, just passing strings. When I try this, I get an 127 > app failure for some reason (the app is /bin/bash). > > > I've been able to set the work directory to a shared filesystem, and > set the scratch tag to /tmp. That combination seems to work fine. But > ideally I'd like to avoid the shared filesystem as much as possible. > > > It also seems to work fine with /tmp as the work directory and with > provider staging enabled. > > > Just curious if this configuration is possible somehow. Thanks! > > > > > > David Michael Kelly > Systems Programmer > University of Chicago Computation Institute > 5735 S. Ellis Ave. Chicago, IL 60637 > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From davidk at ci.uchicago.edu Thu Feb 7 22:20:50 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Thu, 7 Feb 2013 22:20:50 -0600 (CST) Subject: [Swift-devel] Work directory on a non-shared filesystem In-Reply-To: <1360294982.30249.10.camel@echo> Message-ID: <213026860.4015856.1360297250901.JavaMail.root@ci.uchicago.edu> Thanks for the info Mihael. Right now, I am seeing really good performance using /tmp and provider staging, so I'm happy to stick with that. It's been pretty consistent as I scale up in my tests. I'll send you all the plots/benchmarks when I am finished. ----- Original Message ----- > From: "Mihael Hategan" > To: "David Kelly" > Cc: "swift-devel" > Sent: Thursday, February 7, 2013 9:43:02 PM > Subject: Re: [Swift-devel] Work directory on a non-shared filesystem > _swiftwrap needs to make it to the compute nodes somehow. > The scenarios you list below make sense: > 1. work dir on shared fs, scratch on /tmp: _swiftwrap is accessed > through the shared fs so it works. > 2. provider staging with workdir and scratch on /tmp: _swiftwrap is > staged in by coasters, so that works too. > 3. no provider staging, workdir and scratch on /tmp: _swiftwrap has > no > way of making it to the compute node, so 127 (command/function not > found > - see COMMAND EXECUTION section of man bash). > That's standard stuff. A hack would be to install _swiftwrap along > with > the app and convince swift to run that instead of its own swiftwrap. > That might require hacking vdl-int.k, around line 502. Let me know if > you want to go that route, and we can chat on how to get it done. > Mihael > On Thu, 2013-02-07 at 20:10 -0600, David Kelly wrote: > > Is it possible to have a work directory on a non-shared filesystem > > (/tmp) without using provider staging? I'm not staging any files in > > or > > out with swift, just passing strings. When I try this, I get an 127 > > app failure for some reason (the app is /bin/bash). > > > > > > I've been able to set the work directory to a shared filesystem, > > and > > set the scratch tag to /tmp. That combination seems to work fine. > > But > > ideally I'd like to avoid the shared filesystem as much as > > possible. > > > > > > It also seems to work fine with /tmp as the work directory and with > > provider staging enabled. > > > > > > Just curious if this configuration is possible somehow. Thanks! > > > > > > > > > > > > > > > > > > > > > > David Michael Kelly > > Systems Programmer > > University of Chicago Computation Institute > > 5735 S. Ellis Ave. Chicago, IL 60637 > > Is it possible to have a work directory on a non-shared filesystem > > (/tmp) without using provider staging? I'm not staging any files in > > or > > out with swift, just passing strings. When I try this, I get an 127 > > app failure for some reason (the app is /bin/bash). > > > > > > I've been able to set the work directory to a shared filesystem, > > and > > set the scratch tag to /tmp. That combination seems to work fine. > > But > > ideally I'd like to avoid the shared filesystem as much as > > possible. > > > > > > It also seems to work fine with /tmp as the work directory and with > > provider staging enabled. > > > > > > Just curious if this configuration is possible somehow. Thanks! > > > > > > > > > > > > David Michael Kelly > > Systems Programmer > > University of Chicago Computation Institute > > 5735 S. Ellis Ave. Chicago, IL 60637 > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Fri Feb 8 09:35:04 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Fri, 8 Feb 2013 09:35:04 -0600 (CST) Subject: [Swift-devel] Work directory on a non-shared filesystem In-Reply-To: <213026860.4015856.1360297250901.JavaMail.root@ci.uchicago.edu> Message-ID: <1226888249.79421.1360337704473.JavaMail.root@mcs.anl.gov> This sounds great. Provide staging with no actual data staging at least allows local fs to be used on both the submit-node and worker-node, and gets _swiftwrap transported with node code changes. David, how does the scaling look? Do you have any plots for full 55K-cell DSSAT runs? - Mike ----- Original Message ----- > From: "David Kelly" > To: "Mihael Hategan" > Cc: "swift-devel" > Sent: Thursday, February 7, 2013 10:20:50 PM > Subject: Re: [Swift-devel] Work directory on a non-shared filesystem > > > > Thanks for the info Mihael. Right now, I am seeing really good > performance using /tmp and provider staging, so I'm happy to stick > with that. It's been pretty consistent as I scale up in my tests. > I'll send you all the plots/benchmarks when I am finished. > > > > > > From: "Mihael Hategan" > To: "David Kelly" > Cc: "swift-devel" > Sent: Thursday, February 7, 2013 9:43:02 PM > Subject: Re: [Swift-devel] Work directory on a non-shared filesystem > > _swiftwrap needs to make it to the compute nodes somehow. > > The scenarios you list below make sense: > 1. work dir on shared fs, scratch on /tmp: _swiftwrap is accessed > through the shared fs so it works. > 2. provider staging with workdir and scratch on /tmp: _swiftwrap is > staged in by coasters, so that works too. > 3. no provider staging, workdir and scratch on /tmp: _swiftwrap has > no > way of making it to the compute node, so 127 (command/function not > found > - see COMMAND EXECUTION section of man bash). > > That's standard stuff. A hack would be to install _swiftwrap along > with > the app and convince swift to run that instead of its own swiftwrap. > That might require hacking vdl-int.k, around line 502. Let me know if > you want to go that route, and we can chat on how to get it done. > > Mihael > > On Thu, 2013-02-07 at 20:10 -0600, David Kelly wrote: > > Is it possible to have a work directory on a non-shared filesystem > > (/tmp) without using provider staging? I'm not staging any files in > > or > > out with swift, just passing strings. When I try this, I get an 127 > > app failure for some reason (the app is /bin/bash). > > > > > > I've been able to set the work directory to a shared filesystem, > > and > > set the scratch tag to /tmp. That combination seems to work fine. > > But > > ideally I'd like to avoid the shared filesystem as much as > > possible. > > > > > > It also seems to work fine with /tmp as the work directory and with > > provider staging enabled. > > > > > > Just curious if this configuration is possible somehow. Thanks! > > > > > > > > > > > > > > > > > > > > > > David Michael Kelly > > Systems Programmer > > University of Chicago Computation Institute > > 5735 S. Ellis Ave. Chicago, IL 60637 > > Is it possible to have a work directory on a non-shared filesystem > > (/tmp) without using provider staging? I'm not staging any files in > > or > > out with swift, just passing strings. When I try this, I get an 127 > > app failure for some reason (the app is /bin/bash). > > > > > > I've been able to set the work directory to a shared filesystem, > > and > > set the scratch tag to /tmp. That combination seems to work fine. > > But > > ideally I'd like to avoid the shared filesystem as much as > > possible. > > > > > > It also seems to work fine with /tmp as the work directory and with > > provider staging enabled. > > > > > > Just curious if this configuration is possible somehow. Thanks! > > > > > > > > > > > > David Michael Kelly > > Systems Programmer > > University of Chicago Computation Institute > > 5735 S. Ellis Ave. Chicago, IL 60637 > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From marialemos72 at gmail.com Mon Feb 11 10:29:56 2013 From: marialemos72 at gmail.com (Maria Lemos) Date: Mon, 11 Feb 2013 16:29:56 +0000 Subject: [Swift-devel] CFP: Workshops in CISTI'2013 - 8th Iberian Conference on IST, Lisbon, June 19-22 Message-ID: <51191c3a.8665b40a.3765.4e65@mx.google.com> *************************************************************************************************** WORKSHOPS in CISTI'2013 8th Iberian Conference on Information Systems and Technologies Lisbon, Portugal, June 19 - 22, 2013 http://www.aisti.eu/cisti2013/index.php?option=com_content&view=article&id=82&Itemid=67&lang=en *************************************************************************************************** Workshops in CISTI'2013 (8th Iberian Conference on Information Systems and Technologies) with CFP open: > IAwDQ 2013 - Fourth Ibero-American Workshop on Data Quality > SGaMePlay 2013 - Third Iberian Workshop on Serious Games and Meaningful Play > TICAMES 2013 - First Workshop on Information and Communication Technology in Higher Education: Learning Mathematics > WIA 2013 - Primero Workshop en Innovaci?n Abierta > WISA 2013 - Fifth Workshop on Intelligent Systems and Applications > WISIS 2013 - Third Workshop on Information Systems for Interactive Spaces > WSEQP 2013 - First Workshop in Software Engineering and Quality Process URL: http://www.aisti.eu/cisti2013/index.php?option=com_content&view=article&id=82&Itemid=67&lang=en Best regards, Maria Lemos AISTI / CISTI'2013 http://www.aisti.eu/cisti2013 From wilde at mcs.anl.gov Mon Feb 11 11:29:33 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Mon, 11 Feb 2013 11:29:33 -0600 (CST) Subject: [Swift-devel] Welcome Yadu Nand - and initial work plan Message-ID: <1049627788.620455.1360603773784.JavaMail.root@mcs.anl.gov> Hi All, Yadu Nand, who you all know, starts today as temporary professional from India. Please welcome him to this new role. I'd like to use this opportunity to re-focus more of our group communication back on the Swift-Devel list. I've assigned Yadu the following initial plan/roadmap: - learn how to run Swift 0.94 - learn coasters - get access to the 0.94 test platforms (Midway, Fusion, MCS, UC3, Beagle, Raven, Hera, Stampede, Kraken, Hopper) - help David add and integrate app tests into 0.94 (doubling as the initial "stress" tests) - work with David on Swift 0.94 testing and release - learn Bugzilla - set up irc for us - add stress and performance tests (and capture and document performance plots) - learn and improve Swift logging - help improve docs on the way Then for longer term: - unify testing between Swift K (1.0) and T (2.0) - adapt and support Coasters for Swift 2.0 - unify and improve logging/tracing between 1.0 and 2.0 David, can you help Yadu this week to get on the necessary test platforms, to fid the apps to integrate into the test suite, and to run (and improve the documentation of) the Swift/K test suite? Thanks, - Mike -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory From calendarsites at insticc.org Mon Feb 11 12:53:51 2013 From: calendarsites at insticc.org (CalendarSites) Date: Mon, 11 Feb 2013 18:53:51 -0000 Subject: [Swift-devel] CFP IC3K 2013 - 5th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management Message-ID: <049101ce0889$25c592d0$7150b870$@insticc.org> CALL FOR PAPERS 5th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management ? IC3K 2013 Website: http://www.ic3k.org September 19 - 22, 2013 Vilamoura, Algarve, Portugal Important Deadlines: Regular Papers Paper Submission: March 12, 2013 Authors Notification: May 13, 2013 Final Paper Submission and Registration: June 4, 2013 Sponsored by: INSTICC-Institute for Systems and Technologies of Information, Control and Communication INSTICC is Member of: FIPA ? Foundation for Intelligent Physical Agents WfMC - Workflow Management Coaliton OMG - Object Management Group The purpose of the IC3K is to bring together researchers, engineers and practitioners on the areas of Knowledge Discovery, Knowledge Engineering and Knowledge Management. IC3K is composed of three co-located conferences, each specialized in at least one of the aforementioned main knowledge areas. IC3K is composed of three co-located conferences, each specialized in at least one of the aforementioned main knowledge areas. - KDIR: International Conference on Knowledge Discovery and Information Retrieval CONFERENCE TOPICS Web mining Machine Learning Foundations of knowledge discovery in databases Data Analytics Data mining in electronic commerce Interactive and online data mining Process mining Integration of data warehousing and data mining Data reduction and quality assessment Mining high-dimensional data Mining text and semi-structured data Mining multimedia data Structured data analysis and statistical methods BioInformatics & pattern discovery Clustering and classification methods Pre-processing and post-processing for data mining Visual data mining and data visualization Software development Business intelligence applications Information extraction Concept Mining Context Discovery Optimization Information Extraction from Emails User Profiling and Recommender Systems Collaborative Filtering - KEOD: International Conference on Knowledge Engineering and Ontology Development CONFERENCE TOPICS Ontology Engineering Decision Support Systems Semantic Web Natural language processing Intelligent Problem Solving Knowledge Acquisition Knowledge Representation Information Integration Expert Systems Metamodelling Human-Machine Cooperation Process knowledge and semantic services Knowledge reengineering AI programming Applications and case-studies Ontology matching and alignment Ontology sharing and reuse Context Analysis and Modeling Domain Analysis and Modeling Education and Training Issues Enterprise Ontology - KMIS: International Conference on Knowledge Management and Information Sharing CONFERENCE TOPICS E-Learning Organizational semiotics e-Government Digital libraries Interoperability Enterprise Information Systems Metadata and structured documents Best practices & communities of practice Business intelligence & CRM Communication, collaboration and information sharing Impact measurement of knowledge management Tools and technology for knowledge management Knowledge management projects Organizational memories Learning organization & organizational learning Intelligent information systems Social networks and the psychological dimension Intellectual capital Studies, metrics & benchmarks Innovation facilitation KM strategies and implementations Information security Business Process Management Requirements Engineering These three concurrent conferences are held in parallel and registration to one warrants delegates to attend all three. IC3K Keynote Speakers Alan Eardley, Staffordshire University, United Kingdom (List not yet complete) PUBLICATIONS All accepted papers (full and short) will be published in the conference proceedings, under an ISBN reference, on paper and on CD-ROM support. All papers presented at the conference venue will be available at the SCITEPRESS Digital Library (http://www.scitepress.org/DigitalLibrary/). SCITEPRESS is member of CrossRef (http://www.crossref.org/). A short list of presented papers will be selected so that revised and extended versions of these papers will be published by Springer-Verlag in a CCIS Series book. The proceedings will be submitted for indexation by Thomson Reuters Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI (Elsevier Index). AWARDS The awards will be announced and bestowed at the conference closing session. IC3K Conference Chair Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, Portugal Please check further details at the conference website (http://ic3k.org/). -------------- next part -------------- An HTML attachment was scrubbed... URL: From calendarsites at insticc.org Mon Feb 11 13:05:28 2013 From: calendarsites at insticc.org (CalendarSites) Date: Mon, 11 Feb 2013 19:05:28 -0000 Subject: [Swift-devel] CFP IJCCI 2013 - 5th International Joint Conference on Computational Intelligence Message-ID: <05d101ce088a$cc377a60$64a66f20$@insticc.org> CALL FOR PAPERS 5th International Joint Conference on Computational Intelligence - IJCCI 2013 Website: http://www.ijcci.org September 20 - 22, 2013 Vilamoura, Algarve, Portugal Important Deadlines: Regular Papers Paper Submission: March 13, 2013 Authors Notification: May 15, 2013 Final Paper Submission and Registration: June 5, 2013 Sponsored by: INSTICC-Institute for Systems and Technologies of Information, Control and Communication INSTICC is Member of: wfMC- Workflow Management Coaliton The purpose of IJCCI is to bring together researchers, engineers and practitioners on the areas of Fuzzy Computation, Evolutionary Computation and Neural Computation. IJCCI is composed of three co-located conferences, each specialized in at least one of the aforementioned main knowledge areas. - ECTA: International Conference on Evolutionary Computation Theory and Applications (http://www.ecta.ijcci.org) CONFERENCE TOPICS Genetic Algorithms Machine Learning Cognitive Systems Artificial Life Representation techniques Software engineering issues; Metamodelling Evolutionary multiobjective optimization Game Theory and applications Evolution strategies Evolutionary robotics and intelligent agents Society and cultural aspects of evolution Concurrent co-operation Co-evolution and collective behavior Biocomputing and complex adaptive systems Bio-inspired hardware and networks Swarm/collective intelligence Evolutionary art and design Hybrid Systems - FCTA: International Conference on Fuzzy Computation Theory and Applications (http://www.fcta.ijcci.org) CONFERENCE TOPICS Fuzzy hardware, fuzzy architectures Soft computing and intelligent agents Mathematical foundations: Fuzzy set theory and fuzzy logic Approximate reasoning and fuzzy inference System identification and fault detection Fuzzy information retrieval and data mining Fuzzy information processing, fusion, text mining Learning and adaptive fuzzy systems Complex fuzzy systems Pattern recognition: Fuzzy clustering and classifiers Fuzzy image, speech and signal processing, vision and multimedia Industrial, financial and medical applications Type-2 Fuzzy Logic Neuro-fuzzy systems Fuzzy Systems Design, Modeling and Control Real-time Learning of Fuzzy and Neuro-fuzzy Systems Fuzzy Control Fuzzy Systems in Robotics: Sensors, Navigation and Coordination - NCTA: International Conference on Neural Computation Theory and Applications (http://www.ncta.ijcci.org) CONFERENCE TOPICS Pattern Recognition Industrial, financial and medical applications Computational neuroscience Neural network software and applications Complex-valued neural networks Neuroinformatics and bioinformatics Learning paradigms and algorithms Supervised and unsupervised learning Adaptive architectures and mechanisms Support Vector Machines and Applications Complex artificial neural network based systems and dynamics Higher level artificial neural network based intelligent systems Bio-inspired and humanoid robotics Artificial Emotions and Emotional Intelligence Collective & Distributed Intelligent Systems and Dynamics Image Processing and Artificial Vision Applications Intelligent Artificial Perception and Neural Sensors Modular Implementation of Artificial Neural Networks Neural based Data Mining and Complex Information Processing Neural Multi-agent Intelligent Systems and Applications Self-organization and Emergence Stability and Instability in Artificial Neural Networks Neural Network Hardware Implementation and Applications Neural Computation issues in Social Behaviour Emergence These three concurrent conferences are held in parallel and registration to one warrants delegates to attend all three. IJCCI Keynote Speakers Kevin Warwick, University of Reading, United Kingdom Leslie Smith, University of Stirling, United Kingdom (List Not Complete) PUBLICATIONS All accepted papers (full and short) will be published in the conference proceedings, under an ISBN reference, on paper and on CD-ROM support. All papers presented at the conference venue will be available at the SCITEPRESS Digital Library (http://www.scitepress.org/DigitalLibrary/). SCITEPRESS is member of CrossRef (http://www.crossref.org/). A short list of presented papers will be selected so that revised and extended versions of these papers will be published by Springer-Verlag in a SCI Series book. The proceedings will be submitted for indexation by Thomson Reuters Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI (Elsevier Index) AWARDS Best paper awards will be distributed during the conference closing session. IJCCI Conference Co-chairs Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, Portugal Janusz Kacprzyk, Systems Research Institute - Polish Academy of Sciences, Poland Please check further details at the conference website (http://www.ijcci.org). -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Mon Feb 11 13:11:11 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Mon, 11 Feb 2013 11:11:11 -0800 Subject: [Swift-devel] CFP IJCCI 2013 - 5th International Joint Conference on Computational Intelligence In-Reply-To: <05d101ce088a$cc377a60$64a66f20$@insticc.org> References: <05d101ce088a$cc377a60$64a66f20$@insticc.org> Message-ID: <1360609871.2291.1.camel@echo> I'm beginning to doubt the appropriateness of these emails on our development list. Mihael On Mon, 2013-02-11 at 19:05 +0000, CalendarSites wrote: > CALL FOR PAPERS > > 5th International Joint Conference on Computational Intelligence - IJCCI > 2013 > > Website: http://www.ijcci.org > > September 20 - 22, 2013 > > Vilamoura, Algarve, Portugal > > > > Important Deadlines: > > Regular Papers > > Paper Submission: March 13, 2013 > > Authors Notification: May 15, 2013 > > Final Paper Submission and Registration: June 5, 2013 > > > > Sponsored by: > > INSTICC-Institute for Systems and Technologies of Information, Control and > Communication > > INSTICC is Member of: > > wfMC- Workflow Management Coaliton > > The purpose of IJCCI is to bring together researchers, engineers and > practitioners on the areas of Fuzzy Computation, Evolutionary Computation > and Neural Computation. > > IJCCI is composed of three co-located conferences, each specialized in at > least one of the aforementioned main knowledge areas. > > - ECTA: International Conference on Evolutionary Computation Theory and > Applications (http://www.ecta.ijcci.org) > > CONFERENCE TOPICS > > Genetic Algorithms > > Machine Learning > > Cognitive Systems > > Artificial Life > > Representation techniques > > Software engineering issues; Metamodelling > > Evolutionary multiobjective optimization > > Game Theory and applications > > Evolution strategies > > Evolutionary robotics and intelligent agents > > Society and cultural aspects of evolution > > Concurrent co-operation > > Co-evolution and collective behavior > > Biocomputing and complex adaptive systems > > Bio-inspired hardware and networks > > Swarm/collective intelligence > > Evolutionary art and design > > Hybrid Systems > > > > - FCTA: International Conference on Fuzzy Computation Theory and > Applications (http://www.fcta.ijcci.org) > > CONFERENCE TOPICS > > Fuzzy hardware, fuzzy architectures > > Soft computing and intelligent agents > > Mathematical foundations: Fuzzy set theory and fuzzy logic > > Approximate reasoning and fuzzy inference > > System identification and fault detection > > Fuzzy information retrieval and data mining > > Fuzzy information processing, fusion, text mining > > Learning and adaptive fuzzy systems > > Complex fuzzy systems > > Pattern recognition: Fuzzy clustering and classifiers > > Fuzzy image, speech and signal processing, vision and multimedia > > Industrial, financial and medical applications > > Type-2 Fuzzy Logic > > Neuro-fuzzy systems > > Fuzzy Systems Design, Modeling and Control > > Real-time Learning of Fuzzy and Neuro-fuzzy Systems > > Fuzzy Control > > Fuzzy Systems in Robotics: Sensors, Navigation and Coordination > > > > - NCTA: International Conference on Neural Computation Theory and > Applications (http://www.ncta.ijcci.org) > > CONFERENCE TOPICS > > Pattern Recognition > > Industrial, financial and medical applications > > Computational neuroscience > > Neural network software and applications > > Complex-valued neural networks > > Neuroinformatics and bioinformatics > > Learning paradigms and algorithms > > Supervised and unsupervised learning > > Adaptive architectures and mechanisms > > Support Vector Machines and Applications > > Complex artificial neural network based systems and dynamics > > Higher level artificial neural network based intelligent systems > > Bio-inspired and humanoid robotics > > Artificial Emotions and Emotional Intelligence > > Collective & Distributed Intelligent Systems and Dynamics > > Image Processing and Artificial Vision Applications > > Intelligent Artificial Perception and Neural Sensors > > Modular Implementation of Artificial Neural Networks > > Neural based Data Mining and Complex Information Processing > > Neural Multi-agent Intelligent Systems and Applications > > Self-organization and Emergence > > Stability and Instability in Artificial Neural Networks > > Neural Network Hardware Implementation and Applications > > Neural Computation issues in Social Behaviour Emergence > > > > These three concurrent conferences are held in parallel and registration to > one warrants delegates to attend all three. > > > > IJCCI Keynote Speakers > > Kevin Warwick, University of Reading, United Kingdom > > Leslie Smith, University of Stirling, United Kingdom > > (List Not Complete) > > > > PUBLICATIONS > > All accepted papers (full and short) will be published in the conference > proceedings, under an ISBN reference, on paper and on CD-ROM support. > > All papers presented at the conference venue will be available at the > SCITEPRESS Digital Library (http://www.scitepress.org/DigitalLibrary/). > > SCITEPRESS is member of CrossRef (http://www.crossref.org/). > > A short list of presented papers will be selected so that revised and > extended versions of these papers will be published by Springer-Verlag in > a SCI Series book. > > The proceedings will be submitted for indexation by Thomson Reuters > Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI (Elsevier > Index) > > > > AWARDS > > Best paper awards will be distributed during the conference closing session. > > IJCCI Conference Co-chairs > > Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, Portugal > > Janusz Kacprzyk, Systems Research Institute - Polish Academy of Sciences, > Poland > > Please check further details at the conference website > (http://www.ijcci.org). > > CALL FOR PAPERS > > 5th International Joint Conference on Computational Intelligence - > IJCCI 2013 > > Website: http://www.ijcci.org > > September 20 - 22, 2013 > > Vilamoura, Algarve, Portugal > > > > Important Deadlines: > > Regular Papers > > Paper Submission: March 13, 2013 > > Authors Notification: May 15, 2013 > > Final Paper Submission and Registration: June 5, 2013 > > > > Sponsored by: > > INSTICC-Institute for Systems and Technologies of Information, Control > and Communication > > INSTICC is Member of: > > wfMC- Workflow Management Coaliton > > The purpose of IJCCI is to bring together researchers, engineers and > practitioners on the areas of Fuzzy Computation, Evolutionary > Computation and Neural Computation. > > IJCCI is composed of three co-located conferences, each specialized in > at least one of the aforementioned main knowledge areas. > > - ECTA: International Conference on Evolutionary Computation Theory > and Applications (http://www.ecta.ijcci.org) > > CONFERENCE TOPICS > > Genetic Algorithms > > Machine Learning > > Cognitive Systems > > Artificial Life > > Representation techniques > > Software engineering issues; Metamodelling > > Evolutionary multiobjective optimization > > Game Theory and applications > > Evolution strategies > > Evolutionary robotics and intelligent agents > > Society and cultural aspects of evolution > > Concurrent co-operation > > Co-evolution and collective behavior > > Biocomputing and complex adaptive systems > > Bio-inspired hardware and networks > > Swarm/collective intelligence > > Evolutionary art and design > > Hybrid Systems > > > > - FCTA: International Conference on Fuzzy Computation Theory and > Applications (http://www.fcta.ijcci.org) > > CONFERENCE TOPICS > > Fuzzy hardware, fuzzy architectures > > Soft computing and intelligent agents > > Mathematical foundations: Fuzzy set theory and fuzzy logic > > Approximate reasoning and fuzzy inference > > System identification and fault detection > > Fuzzy information retrieval and data mining > > Fuzzy information processing, fusion, text mining > > Learning and adaptive fuzzy systems > > Complex fuzzy systems > > Pattern recognition: Fuzzy clustering and classifiers > > Fuzzy image, speech and signal processing, vision and multimedia > > Industrial, financial and medical applications > > Type-2 Fuzzy Logic > > Neuro-fuzzy systems > > Fuzzy Systems Design, Modeling and Control > > Real-time Learning of Fuzzy and Neuro-fuzzy Systems > > Fuzzy Control > > Fuzzy Systems in Robotics: Sensors, Navigation and Coordination > > > > - NCTA: International Conference on Neural Computation Theory and > Applications (http://www.ncta.ijcci.org) > > CONFERENCE TOPICS > > Pattern Recognition > > Industrial, financial and medical applications > > Computational neuroscience > > Neural network software and applications > > Complex-valued neural networks > > Neuroinformatics and bioinformatics > > Learning paradigms and algorithms > > Supervised and unsupervised learning > > Adaptive architectures and mechanisms > > Support Vector Machines and Applications > > Complex artificial neural network based systems and dynamics > > Higher level artificial neural network based intelligent systems > > Bio-inspired and humanoid robotics > > Artificial Emotions and Emotional Intelligence > > Collective & Distributed Intelligent Systems and Dynamics > > Image Processing and Artificial Vision Applications > > Intelligent Artificial Perception and Neural Sensors > > Modular Implementation of Artificial Neural Networks > > Neural based Data Mining and Complex Information Processing > > Neural Multi-agent Intelligent Systems and Applications > > Self-organization and Emergence > > Stability and Instability in Artificial Neural Networks > > Neural Network Hardware Implementation and Applications > > Neural Computation issues in Social Behaviour Emergence > > > > These three concurrent conferences are held in parallel and > registration to one warrants delegates to attend all three. > > > > IJCCI Keynote Speakers > > Kevin Warwick, University of Reading, United Kingdom > > Leslie Smith, University of Stirling, United Kingdom > > (List Not Complete) > > > > PUBLICATIONS > > All accepted papers (full and short) will be published in the > conference proceedings, under an ISBN reference, on paper and on > CD-ROM support. > > All papers presented at the conference venue will be available at the > SCITEPRESS Digital Library > (http://www.scitepress.org/DigitalLibrary/). > > SCITEPRESS is member of CrossRef (http://www.crossref.org/). > > A short list of presented papers will be selected so that revised and > extended versions of these papers will be published by > Springer-Verlag in a SCI Series book. > > The proceedings will be submitted for indexation by Thomson Reuters > Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI > (Elsevier Index) > > > > AWARDS > > Best paper awards will be distributed during the conference closing > session. > > IJCCI Conference Co-chairs > > Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, Portugal > > Janusz Kacprzyk, Systems Research Institute - Polish Academy of > Sciences, Poland > > Please check further details at the conference website > (http://www.ijcci.org). > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From iraicu at cs.iit.edu Mon Feb 11 13:55:08 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Mon, 11 Feb 2013 13:55:08 -0600 Subject: [Swift-devel] CFP IJCCI 2013 - 5th International Joint Conference on Computational Intelligence In-Reply-To: <1360609871.2291.1.camel@echo> References: <05d101ce088a$cc377a60$64a66f20$@insticc.org> <1360609871.2291.1.camel@echo> Message-ID: <51194C9C.5010101@cs.iit.edu> How did calendarsites at insticc.org get to be a member of the dev list? I thought this was a member only list? Ioan On 2/11/2013 1:11 PM, Mihael Hategan wrote: > I'm beginning to doubt the appropriateness of these emails on our > development list. > > Mihael > > On Mon, 2013-02-11 at 19:05 +0000, CalendarSites wrote: >> CALL FOR PAPERS >> >> 5th International Joint Conference on Computational Intelligence - IJCCI >> 2013 >> >> Website: http://www.ijcci.org >> >> September 20 - 22, 2013 >> >> Vilamoura, Algarve, Portugal >> >> >> >> Important Deadlines: >> >> Regular Papers >> >> Paper Submission: March 13, 2013 >> >> Authors Notification: May 15, 2013 >> >> Final Paper Submission and Registration: June 5, 2013 >> >> >> >> Sponsored by: >> >> INSTICC-Institute for Systems and Technologies of Information, Control and >> Communication >> >> INSTICC is Member of: >> >> wfMC- Workflow Management Coaliton >> >> The purpose of IJCCI is to bring together researchers, engineers and >> practitioners on the areas of Fuzzy Computation, Evolutionary Computation >> and Neural Computation. >> >> IJCCI is composed of three co-located conferences, each specialized in at >> least one of the aforementioned main knowledge areas. >> >> - ECTA: International Conference on Evolutionary Computation Theory and >> Applications (http://www.ecta.ijcci.org) >> >> CONFERENCE TOPICS >> >> Genetic Algorithms >> >> Machine Learning >> >> Cognitive Systems >> >> Artificial Life >> >> Representation techniques >> >> Software engineering issues; Metamodelling >> >> Evolutionary multiobjective optimization >> >> Game Theory and applications >> >> Evolution strategies >> >> Evolutionary robotics and intelligent agents >> >> Society and cultural aspects of evolution >> >> Concurrent co-operation >> >> Co-evolution and collective behavior >> >> Biocomputing and complex adaptive systems >> >> Bio-inspired hardware and networks >> >> Swarm/collective intelligence >> >> Evolutionary art and design >> >> Hybrid Systems >> >> >> >> - FCTA: International Conference on Fuzzy Computation Theory and >> Applications (http://www.fcta.ijcci.org) >> >> CONFERENCE TOPICS >> >> Fuzzy hardware, fuzzy architectures >> >> Soft computing and intelligent agents >> >> Mathematical foundations: Fuzzy set theory and fuzzy logic >> >> Approximate reasoning and fuzzy inference >> >> System identification and fault detection >> >> Fuzzy information retrieval and data mining >> >> Fuzzy information processing, fusion, text mining >> >> Learning and adaptive fuzzy systems >> >> Complex fuzzy systems >> >> Pattern recognition: Fuzzy clustering and classifiers >> >> Fuzzy image, speech and signal processing, vision and multimedia >> >> Industrial, financial and medical applications >> >> Type-2 Fuzzy Logic >> >> Neuro-fuzzy systems >> >> Fuzzy Systems Design, Modeling and Control >> >> Real-time Learning of Fuzzy and Neuro-fuzzy Systems >> >> Fuzzy Control >> >> Fuzzy Systems in Robotics: Sensors, Navigation and Coordination >> >> >> >> - NCTA: International Conference on Neural Computation Theory and >> Applications (http://www.ncta.ijcci.org) >> >> CONFERENCE TOPICS >> >> Pattern Recognition >> >> Industrial, financial and medical applications >> >> Computational neuroscience >> >> Neural network software and applications >> >> Complex-valued neural networks >> >> Neuroinformatics and bioinformatics >> >> Learning paradigms and algorithms >> >> Supervised and unsupervised learning >> >> Adaptive architectures and mechanisms >> >> Support Vector Machines and Applications >> >> Complex artificial neural network based systems and dynamics >> >> Higher level artificial neural network based intelligent systems >> >> Bio-inspired and humanoid robotics >> >> Artificial Emotions and Emotional Intelligence >> >> Collective & Distributed Intelligent Systems and Dynamics >> >> Image Processing and Artificial Vision Applications >> >> Intelligent Artificial Perception and Neural Sensors >> >> Modular Implementation of Artificial Neural Networks >> >> Neural based Data Mining and Complex Information Processing >> >> Neural Multi-agent Intelligent Systems and Applications >> >> Self-organization and Emergence >> >> Stability and Instability in Artificial Neural Networks >> >> Neural Network Hardware Implementation and Applications >> >> Neural Computation issues in Social Behaviour Emergence >> >> >> >> These three concurrent conferences are held in parallel and registration to >> one warrants delegates to attend all three. >> >> >> >> IJCCI Keynote Speakers >> >> Kevin Warwick, University of Reading, United Kingdom >> >> Leslie Smith, University of Stirling, United Kingdom >> >> (List Not Complete) >> >> >> >> PUBLICATIONS >> >> All accepted papers (full and short) will be published in the conference >> proceedings, under an ISBN reference, on paper and on CD-ROM support. >> >> All papers presented at the conference venue will be available at the >> SCITEPRESS Digital Library (http://www.scitepress.org/DigitalLibrary/). >> >> SCITEPRESS is member of CrossRef (http://www.crossref.org/). >> >> A short list of presented papers will be selected so that revised and >> extended versions of these papers will be published by Springer-Verlag in >> a SCI Series book. >> >> The proceedings will be submitted for indexation by Thomson Reuters >> Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI (Elsevier >> Index) >> >> >> >> AWARDS >> >> Best paper awards will be distributed during the conference closing session. >> >> IJCCI Conference Co-chairs >> >> Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, Portugal >> >> Janusz Kacprzyk, Systems Research Institute - Polish Academy of Sciences, >> Poland >> >> Please check further details at the conference website >> (http://www.ijcci.org). >> >> CALL FOR PAPERS >> >> 5th International Joint Conference on Computational Intelligence - >> IJCCI 2013 >> >> Website: http://www.ijcci.org >> >> September 20 - 22, 2013 >> >> Vilamoura, Algarve, Portugal >> >> >> >> Important Deadlines: >> >> Regular Papers >> >> Paper Submission: March 13, 2013 >> >> Authors Notification: May 15, 2013 >> >> Final Paper Submission and Registration: June 5, 2013 >> >> >> >> Sponsored by: >> >> INSTICC-Institute for Systems and Technologies of Information, Control >> and Communication >> >> INSTICC is Member of: >> >> wfMC- Workflow Management Coaliton >> >> The purpose of IJCCI is to bring together researchers, engineers and >> practitioners on the areas of Fuzzy Computation, Evolutionary >> Computation and Neural Computation. >> >> IJCCI is composed of three co-located conferences, each specialized in >> at least one of the aforementioned main knowledge areas. >> >> - ECTA: International Conference on Evolutionary Computation Theory >> and Applications (http://www.ecta.ijcci.org) >> >> CONFERENCE TOPICS >> >> Genetic Algorithms >> >> Machine Learning >> >> Cognitive Systems >> >> Artificial Life >> >> Representation techniques >> >> Software engineering issues; Metamodelling >> >> Evolutionary multiobjective optimization >> >> Game Theory and applications >> >> Evolution strategies >> >> Evolutionary robotics and intelligent agents >> >> Society and cultural aspects of evolution >> >> Concurrent co-operation >> >> Co-evolution and collective behavior >> >> Biocomputing and complex adaptive systems >> >> Bio-inspired hardware and networks >> >> Swarm/collective intelligence >> >> Evolutionary art and design >> >> Hybrid Systems >> >> >> >> - FCTA: International Conference on Fuzzy Computation Theory and >> Applications (http://www.fcta.ijcci.org) >> >> CONFERENCE TOPICS >> >> Fuzzy hardware, fuzzy architectures >> >> Soft computing and intelligent agents >> >> Mathematical foundations: Fuzzy set theory and fuzzy logic >> >> Approximate reasoning and fuzzy inference >> >> System identification and fault detection >> >> Fuzzy information retrieval and data mining >> >> Fuzzy information processing, fusion, text mining >> >> Learning and adaptive fuzzy systems >> >> Complex fuzzy systems >> >> Pattern recognition: Fuzzy clustering and classifiers >> >> Fuzzy image, speech and signal processing, vision and multimedia >> >> Industrial, financial and medical applications >> >> Type-2 Fuzzy Logic >> >> Neuro-fuzzy systems >> >> Fuzzy Systems Design, Modeling and Control >> >> Real-time Learning of Fuzzy and Neuro-fuzzy Systems >> >> Fuzzy Control >> >> Fuzzy Systems in Robotics: Sensors, Navigation and Coordination >> >> >> >> - NCTA: International Conference on Neural Computation Theory and >> Applications (http://www.ncta.ijcci.org) >> >> CONFERENCE TOPICS >> >> Pattern Recognition >> >> Industrial, financial and medical applications >> >> Computational neuroscience >> >> Neural network software and applications >> >> Complex-valued neural networks >> >> Neuroinformatics and bioinformatics >> >> Learning paradigms and algorithms >> >> Supervised and unsupervised learning >> >> Adaptive architectures and mechanisms >> >> Support Vector Machines and Applications >> >> Complex artificial neural network based systems and dynamics >> >> Higher level artificial neural network based intelligent systems >> >> Bio-inspired and humanoid robotics >> >> Artificial Emotions and Emotional Intelligence >> >> Collective & Distributed Intelligent Systems and Dynamics >> >> Image Processing and Artificial Vision Applications >> >> Intelligent Artificial Perception and Neural Sensors >> >> Modular Implementation of Artificial Neural Networks >> >> Neural based Data Mining and Complex Information Processing >> >> Neural Multi-agent Intelligent Systems and Applications >> >> Self-organization and Emergence >> >> Stability and Instability in Artificial Neural Networks >> >> Neural Network Hardware Implementation and Applications >> >> Neural Computation issues in Social Behaviour Emergence >> >> >> >> These three concurrent conferences are held in parallel and >> registration to one warrants delegates to attend all three. >> >> >> >> IJCCI Keynote Speakers >> >> Kevin Warwick, University of Reading, United Kingdom >> >> Leslie Smith, University of Stirling, United Kingdom >> >> (List Not Complete) >> >> >> >> PUBLICATIONS >> >> All accepted papers (full and short) will be published in the >> conference proceedings, under an ISBN reference, on paper and on >> CD-ROM support. >> >> All papers presented at the conference venue will be available at the >> SCITEPRESS Digital Library >> (http://www.scitepress.org/DigitalLibrary/). >> >> SCITEPRESS is member of CrossRef (http://www.crossref.org/). >> >> A short list of presented papers will be selected so that revised and >> extended versions of these papers will be published by >> Springer-Verlag in a SCI Series book. >> >> The proceedings will be submitted for indexation by Thomson Reuters >> Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI >> (Elsevier Index) >> >> >> >> AWARDS >> >> Best paper awards will be distributed during the conference closing >> session. >> >> IJCCI Conference Co-chairs >> >> Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, Portugal >> >> Janusz Kacprzyk, Systems Research Institute - Polish Academy of >> Sciences, Poland >> >> Please check further details at the conference website >> (http://www.ijcci.org). >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud, IEEE/ACM DataCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= From wilde at mcs.anl.gov Mon Feb 11 15:28:11 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Mon, 11 Feb 2013 15:28:11 -0600 (CST) Subject: [Swift-devel] CFP IJCCI 2013 - 5th International Joint Conference on Computational Intelligence In-Reply-To: <1360609871.2291.1.camel@echo> Message-ID: <1892918572.722626.1360618091921.JavaMail.root@mcs.anl.gov> Maybe time to set swift-devel to members-only submission. - Mike ----- Original Message ----- > From: "Mihael Hategan" > To: "CalendarSites" > Cc: swift-devel at ci.uchicago.edu > Sent: Monday, February 11, 2013 1:11:11 PM > Subject: Re: [Swift-devel] CFP IJCCI 2013 - 5th International Joint Conference on Computational Intelligence > > I'm beginning to doubt the appropriateness of these emails on our > development list. > > Mihael > > On Mon, 2013-02-11 at 19:05 +0000, CalendarSites wrote: > > CALL FOR PAPERS > > > > 5th International Joint Conference on Computational Intelligence - > > IJCCI > > 2013 > > > > Website: http://www.ijcci.org > > > > September 20 - 22, 2013 > > > > Vilamoura, Algarve, Portugal > > > > > > > > Important Deadlines: > > > > Regular Papers > > > > Paper Submission: March 13, 2013 > > > > Authors Notification: May 15, 2013 > > > > Final Paper Submission and Registration: June 5, 2013 > > > > > > > > Sponsored by: > > > > INSTICC-Institute for Systems and Technologies of Information, > > Control and > > Communication > > > > INSTICC is Member of: > > > > wfMC- Workflow Management Coaliton > > > > The purpose of IJCCI is to bring together researchers, engineers > > and > > practitioners on the areas of Fuzzy Computation, Evolutionary > > Computation > > and Neural Computation. > > > > IJCCI is composed of three co-located conferences, each specialized > > in at > > least one of the aforementioned main knowledge areas. > > > > - ECTA: International Conference on Evolutionary Computation Theory > > and > > Applications (http://www.ecta.ijcci.org) > > > > CONFERENCE TOPICS > > > > Genetic Algorithms > > > > Machine Learning > > > > Cognitive Systems > > > > Artificial Life > > > > Representation techniques > > > > Software engineering issues; Metamodelling > > > > Evolutionary multiobjective optimization > > > > Game Theory and applications > > > > Evolution strategies > > > > Evolutionary robotics and intelligent agents > > > > Society and cultural aspects of evolution > > > > Concurrent co-operation > > > > Co-evolution and collective behavior > > > > Biocomputing and complex adaptive systems > > > > Bio-inspired hardware and networks > > > > Swarm/collective intelligence > > > > Evolutionary art and design > > > > Hybrid Systems > > > > > > > > - FCTA: International Conference on Fuzzy Computation Theory and > > Applications (http://www.fcta.ijcci.org) > > > > CONFERENCE TOPICS > > > > Fuzzy hardware, fuzzy architectures > > > > Soft computing and intelligent agents > > > > Mathematical foundations: Fuzzy set theory and fuzzy logic > > > > Approximate reasoning and fuzzy inference > > > > System identification and fault detection > > > > Fuzzy information retrieval and data mining > > > > Fuzzy information processing, fusion, text mining > > > > Learning and adaptive fuzzy systems > > > > Complex fuzzy systems > > > > Pattern recognition: Fuzzy clustering and classifiers > > > > Fuzzy image, speech and signal processing, vision and multimedia > > > > Industrial, financial and medical applications > > > > Type-2 Fuzzy Logic > > > > Neuro-fuzzy systems > > > > Fuzzy Systems Design, Modeling and Control > > > > Real-time Learning of Fuzzy and Neuro-fuzzy Systems > > > > Fuzzy Control > > > > Fuzzy Systems in Robotics: Sensors, Navigation and Coordination > > > > > > > > - NCTA: International Conference on Neural Computation Theory and > > Applications (http://www.ncta.ijcci.org) > > > > CONFERENCE TOPICS > > > > Pattern Recognition > > > > Industrial, financial and medical applications > > > > Computational neuroscience > > > > Neural network software and applications > > > > Complex-valued neural networks > > > > Neuroinformatics and bioinformatics > > > > Learning paradigms and algorithms > > > > Supervised and unsupervised learning > > > > Adaptive architectures and mechanisms > > > > Support Vector Machines and Applications > > > > Complex artificial neural network based systems and dynamics > > > > Higher level artificial neural network based intelligent systems > > > > Bio-inspired and humanoid robotics > > > > Artificial Emotions and Emotional Intelligence > > > > Collective & Distributed Intelligent Systems and Dynamics > > > > Image Processing and Artificial Vision Applications > > > > Intelligent Artificial Perception and Neural Sensors > > > > Modular Implementation of Artificial Neural Networks > > > > Neural based Data Mining and Complex Information Processing > > > > Neural Multi-agent Intelligent Systems and Applications > > > > Self-organization and Emergence > > > > Stability and Instability in Artificial Neural Networks > > > > Neural Network Hardware Implementation and Applications > > > > Neural Computation issues in Social Behaviour Emergence > > > > > > > > These three concurrent conferences are held in parallel and > > registration to > > one warrants delegates to attend all three. > > > > > > > > IJCCI Keynote Speakers > > > > Kevin Warwick, University of Reading, United Kingdom > > > > Leslie Smith, University of Stirling, United Kingdom > > > > (List Not Complete) > > > > > > > > PUBLICATIONS > > > > All accepted papers (full and short) will be published in the > > conference > > proceedings, under an ISBN reference, on paper and on CD-ROM > > support. > > > > All papers presented at the conference venue will be available at > > the > > SCITEPRESS Digital Library > > (http://www.scitepress.org/DigitalLibrary/). > > > > SCITEPRESS is member of CrossRef (http://www.crossref.org/). > > > > A short list of presented papers will be selected so that revised > > and > > extended versions of these papers will be published by > > Springer-Verlag in > > a SCI Series book. > > > > The proceedings will be submitted for indexation by Thomson Reuters > > Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI > > (Elsevier > > Index) > > > > > > > > AWARDS > > > > Best paper awards will be distributed during the conference closing > > session. > > > > IJCCI Conference Co-chairs > > > > Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, > > Portugal > > > > Janusz Kacprzyk, Systems Research Institute - Polish Academy of > > Sciences, > > Poland > > > > Please check further details at the conference website > > (http://www.ijcci.org). > > > > CALL FOR PAPERS > > > > 5th International Joint Conference on Computational Intelligence > > - > > IJCCI 2013 > > > > Website: http://www.ijcci.org > > > > September 20 - 22, 2013 > > > > Vilamoura, Algarve, Portugal > > > > > > > > Important Deadlines: > > > > Regular Papers > > > > Paper Submission: March 13, 2013 > > > > Authors Notification: May 15, 2013 > > > > Final Paper Submission and Registration: June 5, 2013 > > > > > > > > Sponsored by: > > > > INSTICC-Institute for Systems and Technologies of Information, > > Control > > and Communication > > > > INSTICC is Member of: > > > > wfMC- Workflow Management Coaliton > > > > The purpose of IJCCI is to bring together researchers, engineers > > and > > practitioners on the areas of Fuzzy Computation, Evolutionary > > Computation and Neural Computation. > > > > IJCCI is composed of three co-located conferences, each specialized > > in > > at least one of the aforementioned main knowledge areas. > > > > - ECTA: International Conference on Evolutionary Computation Theory > > and Applications (http://www.ecta.ijcci.org) > > > > CONFERENCE TOPICS > > > > Genetic Algorithms > > > > Machine Learning > > > > Cognitive Systems > > > > Artificial Life > > > > Representation techniques > > > > Software engineering issues; Metamodelling > > > > Evolutionary multiobjective optimization > > > > Game Theory and applications > > > > Evolution strategies > > > > Evolutionary robotics and intelligent agents > > > > Society and cultural aspects of evolution > > > > Concurrent co-operation > > > > Co-evolution and collective behavior > > > > Biocomputing and complex adaptive systems > > > > Bio-inspired hardware and networks > > > > Swarm/collective intelligence > > > > Evolutionary art and design > > > > Hybrid Systems > > > > > > > > - FCTA: International Conference on Fuzzy Computation Theory and > > Applications (http://www.fcta.ijcci.org) > > > > CONFERENCE TOPICS > > > > Fuzzy hardware, fuzzy architectures > > > > Soft computing and intelligent agents > > > > Mathematical foundations: Fuzzy set theory and fuzzy logic > > > > Approximate reasoning and fuzzy inference > > > > System identification and fault detection > > > > Fuzzy information retrieval and data mining > > > > Fuzzy information processing, fusion, text mining > > > > Learning and adaptive fuzzy systems > > > > Complex fuzzy systems > > > > Pattern recognition: Fuzzy clustering and classifiers > > > > Fuzzy image, speech and signal processing, vision and multimedia > > > > Industrial, financial and medical applications > > > > Type-2 Fuzzy Logic > > > > Neuro-fuzzy systems > > > > Fuzzy Systems Design, Modeling and Control > > > > Real-time Learning of Fuzzy and Neuro-fuzzy Systems > > > > Fuzzy Control > > > > Fuzzy Systems in Robotics: Sensors, Navigation and Coordination > > > > > > > > - NCTA: International Conference on Neural Computation Theory and > > Applications (http://www.ncta.ijcci.org) > > > > CONFERENCE TOPICS > > > > Pattern Recognition > > > > Industrial, financial and medical applications > > > > Computational neuroscience > > > > Neural network software and applications > > > > Complex-valued neural networks > > > > Neuroinformatics and bioinformatics > > > > Learning paradigms and algorithms > > > > Supervised and unsupervised learning > > > > Adaptive architectures and mechanisms > > > > Support Vector Machines and Applications > > > > Complex artificial neural network based systems and dynamics > > > > Higher level artificial neural network based intelligent systems > > > > Bio-inspired and humanoid robotics > > > > Artificial Emotions and Emotional Intelligence > > > > Collective & Distributed Intelligent Systems and Dynamics > > > > Image Processing and Artificial Vision Applications > > > > Intelligent Artificial Perception and Neural Sensors > > > > Modular Implementation of Artificial Neural Networks > > > > Neural based Data Mining and Complex Information Processing > > > > Neural Multi-agent Intelligent Systems and Applications > > > > Self-organization and Emergence > > > > Stability and Instability in Artificial Neural Networks > > > > Neural Network Hardware Implementation and Applications > > > > Neural Computation issues in Social Behaviour Emergence > > > > > > > > These three concurrent conferences are held in parallel and > > registration to one warrants delegates to attend all three. > > > > > > > > IJCCI Keynote Speakers > > > > Kevin Warwick, University of Reading, United Kingdom > > > > Leslie Smith, University of Stirling, United Kingdom > > > > (List Not Complete) > > > > > > > > PUBLICATIONS > > > > All accepted papers (full and short) will be published in the > > conference proceedings, under an ISBN reference, on paper and on > > CD-ROM support. > > > > All papers presented at the conference venue will be available at > > the > > SCITEPRESS Digital Library > > (http://www.scitepress.org/DigitalLibrary/). > > > > SCITEPRESS is member of CrossRef (http://www.crossref.org/). > > > > A short list of presented papers will be selected so that revised > > and > > extended versions of these papers will be published by > > Springer-Verlag in a SCI Series book. > > > > The proceedings will be submitted for indexation by Thomson Reuters > > Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI > > (Elsevier Index) > > > > > > > > AWARDS > > > > Best paper awards will be distributed during the conference closing > > session. > > > > IJCCI Conference Co-chairs > > > > Joaquim Filipe, Polytechnic Institute of Set?bal / INSTICC, > > Portugal > > > > Janusz Kacprzyk, Systems Research Institute - Polish Academy of > > Sciences, Poland > > > > Please check further details at the conference website > > (http://www.ijcci.org). > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From tim.g.armstrong at gmail.com Mon Feb 11 21:54:57 2013 From: tim.g.armstrong at gmail.com (Tim Armstrong) Date: Mon, 11 Feb 2013 21:54:57 -0600 Subject: [Swift-devel] Welcome Yadu Nand - and initial work plan In-Reply-To: <1049627788.620455.1360603773784.JavaMail.root@mcs.anl.gov> References: <1049627788.620455.1360603773784.JavaMail.root@mcs.anl.gov> Message-ID: Welcome Yadu! On Mon, Feb 11, 2013 at 11:29 AM, Michael Wilde wrote: > Hi All, > > Yadu Nand, who you all know, starts today as temporary professional from > India. Please welcome him to this new role. > > I'd like to use this opportunity to re-focus more of our group > communication back on the Swift-Devel list. > > I've assigned Yadu the following initial plan/roadmap: > > - learn how to run Swift 0.94 > - learn coasters > - get access to the 0.94 test platforms (Midway, Fusion, MCS, UC3, Beagle, > Raven, Hera, Stampede, Kraken, Hopper) > - help David add and integrate app tests into 0.94 (doubling as the > initial "stress" tests) > - work with David on Swift 0.94 testing and release > - learn Bugzilla > - set up irc for us > - add stress and performance tests (and capture and document performance > plots) > - learn and improve Swift logging > - help improve docs on the way > > Then for longer term: > > - unify testing between Swift K (1.0) and T (2.0) > - adapt and support Coasters for Swift 2.0 > - unify and improve logging/tracing between 1.0 and 2.0 > > David, can you help Yadu this week to get on the necessary test platforms, > to fid the apps to integrate into the test suite, and to run (and improve > the documentation of) the Swift/K test suite? > > Thanks, > > - Mike > > > > > > > > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadudoc1729 at gmail.com Tue Feb 12 06:21:45 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Tue, 12 Feb 2013 17:51:45 +0530 Subject: [Swift-devel] Welcome Yadu Nand - and initial work plan In-Reply-To: References: <1049627788.620455.1360603773784.JavaMail.root@mcs.anl.gov> Message-ID: @Mike, Thank you for the introductions and for the detailed plan. @Tim, Thanks :) I have worked with Justin and Mihael earlier for my GSoC project and over the past few months on Swift/T with Tim and Justin. I understand the difficulties in working across time-zones, but I am confident that we can cover for it via skype, email and IRC. I look forward to working with all of you closely, and I'm really excited that I will be working on swift full-time! For this week, I'm working towards getting access to all machines that I'd need, and getting 0.94RC running with tests. Could any of you help with tagging the place where the following (missing) resources need to be requested from (MCS resource / CI resource ) ? Midway - from http://rcc.uchicago.edu/user_documentation/general_user_account_request.html Fusion - MCS - from https://wiki.uchicago.edu/display/uc3/UC3+Quickstart. (needs cnet id) UC3 - Beagle - CI resource ? (I made this request in the CI account request process) Raven - Hera - Stampede - Kraken - Hopper - XSEDE - Create account -> https://portal.xsede.org (resource TG-ASC090068) Interpid - Create account -> http://accounts.alcf.anl.gov ( Pending ) Thanks, Yadu On Tue, Feb 12, 2013 at 9:24 AM, Tim Armstrong wrote: > > Welcome Yadu! > > > On Mon, Feb 11, 2013 at 11:29 AM, Michael Wilde wrote: >> >> Hi All, >> >> Yadu Nand, who you all know, starts today as temporary professional from India. Please welcome him to this new role. >> >> I'd like to use this opportunity to re-focus more of our group communication back on the Swift-Devel list. >> >> I've assigned Yadu the following initial plan/roadmap: >> >> - learn how to run Swift 0.94 >> - learn coasters >> - get access to the 0.94 test platforms (Midway, Fusion, MCS, UC3, Beagle, Raven, Hera, Stampede, Kraken, Hopper) >> - help David add and integrate app tests into 0.94 (doubling as the initial "stress" tests) >> - work with David on Swift 0.94 testing and release >> - learn Bugzilla >> - set up irc for us >> - add stress and performance tests (and capture and document performance plots) >> - learn and improve Swift logging >> - help improve docs on the way >> >> Then for longer term: >> >> - unify testing between Swift K (1.0) and T (2.0) >> - adapt and support Coasters for Swift 2.0 >> - unify and improve logging/tracing between 1.0 and 2.0 >> >> David, can you help Yadu this week to get on the necessary test platforms, to fid the apps to integrate into the test suite, and to run (and improve the documentation of) the Swift/K test suite? >> >> Thanks, >> >> - Mike >> >> >> >> >> >> >> >> >> -- >> Michael Wilde >> Computation Institute, University of Chicago >> Mathematics and Computer Science Division >> Argonne National Laboratory >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From calendarsites at insticc.org Tue Feb 12 09:00:54 2013 From: calendarsites at insticc.org (CalendarSites) Date: Tue, 12 Feb 2013 15:00:54 -0000 Subject: [Swift-devel] CFP Special Session on Cloud Interoperability - CI 2013 Message-ID: <045901ce0931$c42d3e70$4c87bb50$@insticc.org> CALL FOR PAPERS Special Session on Cloud Interoperability - CI 2013 In conjunction with the 3rd International Conference on Cloud Computing and Services Science - CLOSER 2013 Website: http://closer.scitevents.org/CI.aspx May 8 - 10, 2013 Aachen, Germany Co-organized by: RWTH Sponsored by: INSTICC INSTICC is Member of: WfMC and OMG Technically sponsored by: SINTEF and IFIP IMPORTANT DATES: Paper Submission: March 1, 2013 Authors Notification: March 19, 2013 Camera Ready and Registration: March 28, 2013 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - We would like to draw your attention to the Cloud Interoperability - CI 2013 (held in conjunction with the 3rd International Conference on Cloud Computing and Services Science- CLOSER 2013), whose submission deadline is scheduled for March 1, 2013. Cloud interoperability is one of the most challenging aspects of cloud computing. For the final user, interoperability means the possibility of migrating from one IaaS provider to another, or to manage in a seamless way the resources provided by distinct providers: thus the risk of being locked to a given provider is avoided, and cloud operation is made more reliable and secure. For the intermediate provider that offers value added services using a cloud infrastructure, interoperability means the possibility to offer the same service using different providers, without redesigning the application each time. Being the cloud a fast evolving creature, standardization proposals must be extremely flexible and not bound to current technologies or short term visions, while being simple to adhere to. As an alternative, some propose the design of intermediate layers that work as adapters for distinct providers. We invite practitioners, researchers and industry representatives to submit original ideas to the Closer Special Session on Cloud Interoperability: we will consider with special interest articles that: - demonstrate the impact of Cloud Interoperability with well documented use cases and experiments, - formalize aspects of cloud operation that are relevant for interoperability - describe standardization initiatives that aim at Cloud Interoperation. The conference will be sponsored by the Institute for Systems and Technologies of Information, Control and Communication (INSTICC), co-organized with the Aachen University (RWTH) and technically sponsored by the SINTEF and IFIP. INSTICC is Member of the Workflow Management Coalition (WfMC) and Object Management Group (OMG). The purpose of the 3rd International Conference on Cloud Computing and Services Science, CLOSER 2013, focuses on the emerging area of Cloud Computing, inspired by some latest advances that concern the infrastructure, operations, and available services through the global network. Further, the conference considers as essential the link to Services Science, acknowledging the service-orientation in most current IT-driven collaborations. The conference is nevertheless not about the union of these two (already broad) fields, but about Cloud Computing where we are also interested in how Services Science can provide theory, methods and techniques to design, analyze, manage, market and study various aspects of Cloud Computing. Submitted papers will be subject to a double-blind review process. All papers accepted to the Special Session will be published in a special section of the conference proceedings book, under an ISBN reference, on paper and on CD-ROM. The proceedings will be submitted for indexation by Thomson Reuters Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI (Elsevier Index). All papers presented at the conference venue will be available at the SCITEPRESS Digital Library ( http://www.scitepress.org/DigitalLibrary/). SCITEPRESS is member of CrossRef ( http://www.crossref.org/). CLOSER 2013 will be held in conjunction with WEBIST 2013 (9th International Conference on Web Information Systems and Technologies - http://www.webist.org/), SMARTGREENS 2013 (International Conference on Smart Grids and Green IT Systems - http://www.smartgreens.org/) and CSEDU 2013 (International Conference on Computer Supported Education - http://www.csedu.org/) in Aachen, Germany. Registration to CLOSER will enable free attendance to any sessions of WEBIST, SMARTGREENS and CSEDU conferences as a non-speaker. Please check further details at the Special Session website ( http://closer.scitevents.org/CI.aspx) and the main conference website (http://closer.scitevents.org). CHAIR Augusto Ciuffoletti Universit? di Pisa Italy In case of questions, please feel free to contact us: closer.secretariat at scitevents.org PROGRAM COMMITTEE: Please check the special session?s program committee members at http://closer.scitevents.org/CI.aspx -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Tue Feb 12 09:47:54 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Tue, 12 Feb 2013 09:47:54 -0600 (CST) Subject: [Swift-devel] Welcome Yadu Nand - and initial work plan In-Reply-To: Message-ID: <1549357758.62400.1360684074306.JavaMail.root@mcs.anl.gov> Yadu, For your CI login, also request "OSG Education Gridlab" - for access to our lab hosts, bridled and communicado at ci.uchicago.ed. For Midway, look for the UChicago RCC (Research COmputing Center) and request login access (they will enable your CNet ID for access); ask to be put in the project under /project/wilde Same for UC3. For Beagle, you first nedd a CI login, then need to request access to the Beagle project; see the CI Computing Resources Page, FAQ, HPC topic. David, can you locate the latest swiftdevel info page on local resources and update it to reflect the latest info/request pages? You might need to move it from the old wiki. This will be a good reference for all new collaborators as well as for currnet team members that need additional login access. Thanks, - Mike ----- Original Message ----- > From: "Yadu Nand" > To: "Tim Armstrong" > Cc: "Michael Wilde" , "swift-devel" > Sent: Tuesday, February 12, 2013 6:21:45 AM > Subject: Re: [Swift-devel] Welcome Yadu Nand - and initial work plan > > @Mike, Thank you for the introductions and for the detailed plan. > > @Tim, Thanks :) > > I have worked with Justin and Mihael earlier for my GSoC project and > over the past > few months on Swift/T with Tim and Justin. I understand the > difficulties in working > across time-zones, but I am confident that we can cover for it via > skype, email and IRC. > > I look forward to working with all of you closely, and I'm really > excited that I will be > working on swift full-time! > > For this week, I'm working towards getting access to all machines > that I'd need, > and getting 0.94RC running with tests. > > Could any of you help with tagging the place where the following > (missing) resources need to > be requested from (MCS resource / CI resource ) ? > > Midway - from > http://rcc.uchicago.edu/user_documentation/general_user_account_request.html > Fusion - > MCS - from https://wiki.uchicago.edu/display/uc3/UC3+Quickstart . > (needs cnet id) > UC3 - > Beagle - CI resource ? (I made this request in the CI account request > process) > Raven - > Hera - > Stampede - > Kraken - > Hopper - > XSEDE - Create account -> https://portal.xsede.org (resource > TG-ASC090068) > Interpid - Create account -> http://accounts.alcf.anl.gov ( Pending ) > > > Thanks, > Yadu > > On Tue, Feb 12, 2013 at 9:24 AM, Tim Armstrong < > tim.g.armstrong at gmail.com > wrote: > > > > Welcome Yadu! > > > > > > On Mon, Feb 11, 2013 at 11:29 AM, Michael Wilde < wilde at mcs.anl.gov > > > wrote: > >> > >> Hi All, > >> > >> Yadu Nand, who you all know, starts today as temporary > >> professional from India. Please welcome him to this new role. > >> > >> I'd like to use this opportunity to re-focus more of our group > >> communication back on the Swift-Devel list. > >> > >> I've assigned Yadu the following initial plan/roadmap: > >> > >> - learn how to run Swift 0.94 > >> - learn coasters > >> - get access to the 0.94 test platforms (Midway, Fusion, MCS, UC3, > >> Beagle, Raven, Hera, Stampede, Kraken, Hopper) > >> - help David add and integrate app tests into 0.94 (doubling as > >> the initial "stress" tests) > >> - work with David on Swift 0.94 testing and release > >> - learn Bugzilla > >> - set up irc for us > >> - add stress and performance tests (and capture and document > >> performance plots) > >> - learn and improve Swift logging > >> - help improve docs on the way > >> > >> Then for longer term: > >> > >> - unify testing between Swift K (1.0) and T (2.0) > >> - adapt and support Coasters for Swift 2.0 > >> - unify and improve logging/tracing between 1.0 and 2.0 > >> > >> David, can you help Yadu this week to get on the necessary test > >> platforms, to fid the apps to integrate into the test suite, and > >> to run (and improve the documentation of) the Swift/K test suite? > >> > >> Thanks, > >> > >> - Mike > >> > >> > >> > >> > >> > >> > >> > >> > >> -- > >> Michael Wilde > >> Computation Institute, University of Chicago > >> Mathematics and Computer Science Division > >> Argonne National Laboratory > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > From davidk at ci.uchicago.edu Tue Feb 12 11:07:28 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Tue, 12 Feb 2013 11:07:28 -0600 (CST) Subject: [Swift-devel] Welcome Yadu Nand - and initial work plan In-Reply-To: Message-ID: <1602966480.1445973.1360688848387.JavaMail.root@ci.uchicago.edu> Yadu, I put together a list with links at https://sites.google.com/site/swiftdevel/home/request-computing-resources. I think I have all the machines you listed - let me know if any are missing. David ----- Original Message ----- > From: "Yadu Nand" > To: "Tim Armstrong" > Cc: "swift-devel" > Sent: Tuesday, February 12, 2013 6:21:45 AM > Subject: Re: [Swift-devel] Welcome Yadu Nand - and initial work plan > @Mike, Thank you for the introductions and for the detailed plan. > @Tim, Thanks :) > I have worked with Justin and Mihael earlier for my GSoC project and > over the past > few months on Swift/T with Tim and Justin. I understand the > difficulties in working > across time-zones, but I am confident that we can cover for it via > skype, email and IRC. > I look forward to working with all of you closely, and I'm really > excited that I will be > working on swift full-time! > For this week, I'm working towards getting access to all machines > that I'd need, > and getting 0.94RC running with tests. > Could any of you help with tagging the place where the following > (missing) resources need to > be requested from (MCS resource / CI resource ) ? > Midway - from > http://rcc.uchicago.edu/user_documentation/general_user_account_request.html > Fusion - > MCS - from https://wiki.uchicago.edu/display/uc3/UC3+Quickstart . > (needs cnet id) > UC3 - > Beagle - CI resource ? (I made this request in the CI account request > process) > Raven - > Hera - > Stampede - > Kraken - > Hopper - > XSEDE - Create account -> https://portal.xsede.org (resource > TG-ASC090068) > Interpid - Create account -> http://accounts.alcf.anl.gov ( Pending ) > Thanks, > Yadu > On Tue, Feb 12, 2013 at 9:24 AM, Tim Armstrong < > tim.g.armstrong at gmail.com > wrote: > > > > Welcome Yadu! > > > > > > On Mon, Feb 11, 2013 at 11:29 AM, Michael Wilde < wilde at mcs.anl.gov > > > wrote: > >> > >> Hi All, > >> > >> Yadu Nand, who you all know, starts today as temporary > >> professional from India. Please welcome him to this new role. > >> > >> I'd like to use this opportunity to re-focus more of our group > >> communication back on the Swift-Devel list. > >> > >> I've assigned Yadu the following initial plan/roadmap: > >> > >> - learn how to run Swift 0.94 > >> - learn coasters > >> - get access to the 0.94 test platforms (Midway, Fusion, MCS, UC3, > >> Beagle, Raven, Hera, Stampede, Kraken, Hopper) > >> - help David add and integrate app tests into 0.94 (doubling as > >> the initial "stress" tests) > >> - work with David on Swift 0.94 testing and release > >> - learn Bugzilla > >> - set up irc for us > >> - add stress and performance tests (and capture and document > >> performance plots) > >> - learn and improve Swift logging > >> - help improve docs on the way > >> > >> Then for longer term: > >> > >> - unify testing between Swift K (1.0) and T (2.0) > >> - adapt and support Coasters for Swift 2.0 > >> - unify and improve logging/tracing between 1.0 and 2.0 > >> > >> David, can you help Yadu this week to get on the necessary test > >> platforms, to fid the apps to integrate into the test suite, and > >> to run (and improve the documentation of) the Swift/K test suite? > >> > >> Thanks, > >> > >> - Mike > >> > >> > >> > >> > >> > >> > >> > >> > >> -- > >> Michael Wilde > >> Computation Institute, University of Chicago > >> Mathematics and Computer Science Division > >> Argonne National Laboratory > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From iraicu at cs.iit.edu Tue Feb 12 12:33:27 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Tue, 12 Feb 2013 12:33:27 -0600 Subject: [Swift-devel] CFP Special Session on Cloud Interoperability - CI 2013 In-Reply-To: <045901ce0931$c42d3e70$4c87bb50$@insticc.org> References: <045901ce0931$c42d3e70$4c87bb50$@insticc.org> Message-ID: <511A8AF7.8060800@cs.iit.edu> Can someone add "calendarsites at insticc.org" to the automatic reject list? And/Or remove their membership? Ioan On 2/12/2013 9:00 AM, CalendarSites wrote: > > *CALL FOR PAPERS* > > Special Session on Cloud Interoperability - CI 2013 > > In conjunction with the 3rd International Conference on Cloud > Computing and Services Science - CLOSER 2013 > > *Website*: http://closer.scitevents.org/CI.aspx > > May 8 - 10, 2013 > > Aachen, Germany > > *Co-organized by: *RWTH** > > *Sponsored by: *INSTICC** > > *INSTICC is Member of:*WfMC and OMG > > *Technically sponsored by:*SINTEF and IFIP > > *IMPORTANT DATES:* > > ** > > Paper Submission: *March 1, 2013* > Authors Notification: *March 19, 2013* > Camera Ready and Registration: *March 28, 2013* > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > We would like to draw your attention to the Cloud Interoperability - > CI 2013 (held in conjunction with the 3rd International Conference on > Cloud Computing and Services Science- CLOSER 2013), whose submission > deadline is scheduled for *March 1, 2013*. > > Cloud interoperability is one of the most challenging aspects of cloud > computing. For the final user, interoperability means the possibility > of migrating from one IaaS provider to another, or to manage in a > seamless way the resources provided by distinct providers: thus the > risk of being locked to a given provider is avoided, and cloud > operation is made more reliable and secure. For the intermediate > provider that offers value added services using a cloud > infrastructure, interoperability means the possibility to offer the > same service using different providers, without redesigning the > application each time. Being the cloud a fast evolving creature, > standardization proposals must be extremely flexible and not bound to > current technologies or short term visions, while being simple to > adhere to. As an alternative, some propose the design of intermediate > layers that work as adapters for distinct providers. > > We invite practitioners, researchers and industry representatives to > submit original ideas to the Closer Special Session on Cloud > Interoperability: we will consider with special interest articles that: > > - demonstrate the impact of Cloud Interoperability with well > documented use cases and experiments, > - formalize aspects of cloud operation that are relevant for > interoperability > > - describe standardization initiatives that aim at Cloud Interoperation. > > > The conference will be sponsored by the Institute for Systems and > Technologies of Information, Control and Communication (INSTICC), > co-organized with the Aachen University (RWTH) and technically > sponsored by the SINTEF and IFIP. INSTICC is Member of the Workflow > Management Coalition (WfMC) and Object Management Group (OMG). > > The purpose of the3rd International Conference on Cloud Computing and > Services Science, CLOSER 2013, focuses on the emerging area of Cloud > Computing, inspired by some latest advances that concern the > infrastructure, operations, and available services through the global > network. Further, the conference considers as essential the link to > Services Science, acknowledging the service-orientation in most > current IT-driven collaborations. The conference is nevertheless not > about the union of these two (already broad) fields, but about Cloud > Computing where we are also interested in how Services Science can > provide theory, methods and techniques to design, analyze, manage, > market and study various aspects of Cloud Computing. > > Submitted papers will be subject to a double-blind review process. > > All papers accepted to the Special Session will be published in a > special section of the conference proceedings book, under an ISBN > reference, on paper and on CD-ROM. > > The proceedings will be submitted for indexation by Thomson Reuters > Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI > (Elsevier Index). All papers presented at the conference venue will be > available at the SCITEPRESS Digital Library > (http://www.scitepress.org/DigitalLibrary/). SCITEPRESS is member of > CrossRef (http://www.crossref.org/). > > CLOSER 2013 will be held in conjunction with WEBIST 2013 (9th > International Conference on Web Information Systems and Technologies - > http://www.webist.org/), SMARTGREENS 2013 (International Conference on > Smart Grids and Green IT Systems - http://www.smartgreens.org/) and > CSEDU 2013 (International Conference on Computer Supported Education - > http://www.csedu.org/) in Aachen, Germany. > > Registration to CLOSER will enable free attendance to any sessions of > WEBIST, SMARTGREENS and CSEDU conferences as a non-speaker. > > Please check further details at the Special Session website > > (http://closer.scitevents.org/CI.aspx) and the main conference website > (http://closer.scitevents.org). > > *CHAIR* > > *Augusto Ciuffoletti* > Universit? di Pisa > Italy > > In case of questions, please feel free to contact us: > closer.secretariat at scitevents.org > > > *PROGRAM COMMITTEE:* > > Please check the special session?s program committee members at > http://closer.scitevents.org/CI.aspx > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud, IEEE/ACM DataCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Feb 12 14:29:44 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 12 Feb 2013 12:29:44 -0800 Subject: [Swift-devel] CFP Special Session on Cloud Interoperability - CI 2013 In-Reply-To: <511A8AF7.8060800@cs.iit.edu> References: <045901ce0931$c42d3e70$4c87bb50$@insticc.org> <511A8AF7.8060800@cs.iit.edu> Message-ID: <1360700984.10525.0.camel@echo> Done. On Tue, 2013-02-12 at 12:33 -0600, Ioan Raicu wrote: > Can someone add "calendarsites at insticc.org" to the automatic reject > list? And/Or remove their membership? > Ioan > On 2/12/2013 9:00 AM, CalendarSites wrote: > > > > *CALL FOR PAPERS* > > > > Special Session on Cloud Interoperability - CI 2013 > > > > In conjunction with the 3rd International Conference on Cloud > > Computing and Services Science - CLOSER 2013 > > > > *Website*: http://closer.scitevents.org/CI.aspx > > > > May 8 - 10, 2013 > > > > Aachen, Germany > > > > *Co-organized by: *RWTH** > > > > *Sponsored by: *INSTICC** > > > > *INSTICC is Member of:*WfMC and OMG > > > > *Technically sponsored by:*SINTEF and IFIP > > > > *IMPORTANT DATES:* > > > > ** > > > > Paper Submission: *March 1, 2013* > > Authors Notification: *March 19, 2013* > > Camera Ready and Registration: *March 28, 2013* > > > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > > > We would like to draw your attention to the Cloud Interoperability - > > CI 2013 (held in conjunction with the 3rd International Conference on > > Cloud Computing and Services Science- CLOSER 2013), whose submission > > deadline is scheduled for *March 1, 2013*. > > > > Cloud interoperability is one of the most challenging aspects of cloud > > computing. For the final user, interoperability means the possibility > > of migrating from one IaaS provider to another, or to manage in a > > seamless way the resources provided by distinct providers: thus the > > risk of being locked to a given provider is avoided, and cloud > > operation is made more reliable and secure. For the intermediate > > provider that offers value added services using a cloud > > infrastructure, interoperability means the possibility to offer the > > same service using different providers, without redesigning the > > application each time. Being the cloud a fast evolving creature, > > standardization proposals must be extremely flexible and not bound to > > current technologies or short term visions, while being simple to > > adhere to. As an alternative, some propose the design of intermediate > > layers that work as adapters for distinct providers. > > > > We invite practitioners, researchers and industry representatives to > > submit original ideas to the Closer Special Session on Cloud > > Interoperability: we will consider with special interest articles that: > > > > - demonstrate the impact of Cloud Interoperability with well > > documented use cases and experiments, > > - formalize aspects of cloud operation that are relevant for > > interoperability > > > > - describe standardization initiatives that aim at Cloud Interoperation. > > > > > > The conference will be sponsored by the Institute for Systems and > > Technologies of Information, Control and Communication (INSTICC), > > co-organized with the Aachen University (RWTH) and technically > > sponsored by the SINTEF and IFIP. INSTICC is Member of the Workflow > > Management Coalition (WfMC) and Object Management Group (OMG). > > > > The purpose of the3rd International Conference on Cloud Computing and > > Services Science, CLOSER 2013, focuses on the emerging area of Cloud > > Computing, inspired by some latest advances that concern the > > infrastructure, operations, and available services through the global > > network. Further, the conference considers as essential the link to > > Services Science, acknowledging the service-orientation in most > > current IT-driven collaborations. The conference is nevertheless not > > about the union of these two (already broad) fields, but about Cloud > > Computing where we are also interested in how Services Science can > > provide theory, methods and techniques to design, analyze, manage, > > market and study various aspects of Cloud Computing. > > > > Submitted papers will be subject to a double-blind review process. > > > > All papers accepted to the Special Session will be published in a > > special section of the conference proceedings book, under an ISBN > > reference, on paper and on CD-ROM. > > > > The proceedings will be submitted for indexation by Thomson Reuters > > Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI > > (Elsevier Index). All papers presented at the conference venue will be > > available at the SCITEPRESS Digital Library > > (http://www.scitepress.org/DigitalLibrary/). SCITEPRESS is member of > > CrossRef (http://www.crossref.org/). > > > > CLOSER 2013 will be held in conjunction with WEBIST 2013 (9th > > International Conference on Web Information Systems and Technologies - > > http://www.webist.org/), SMARTGREENS 2013 (International Conference on > > Smart Grids and Green IT Systems - http://www.smartgreens.org/) and > > CSEDU 2013 (International Conference on Computer Supported Education - > > http://www.csedu.org/) in Aachen, Germany. > > > > Registration to CLOSER will enable free attendance to any sessions of > > WEBIST, SMARTGREENS and CSEDU conferences as a non-speaker. > > > > Please check further details at the Special Session website > > > > (http://closer.scitevents.org/CI.aspx) and the main conference website > > (http://closer.scitevents.org). > > > > *CHAIR* > > > > *Augusto Ciuffoletti* > > Universit? di Pisa > > Italy > > > > In case of questions, please feel free to contact us: > > closer.secretariat at scitevents.org > > > > > > *PROGRAM COMMITTEE:* > > > > Please check the special session?s program committee members at > > http://closer.scitevents.org/CI.aspx > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > Can someone add "calendarsites at insticc.org" to the automatic reject > list? And/Or remove their membership? > Ioan > On 2/12/2013 9:00 AM, CalendarSites wrote: > > > CALL FOR PAPERS > > > > > > > > Special Session on Cloud Interoperability - CI 2013 > > > > In conjunction with the 3rd International Conference on Cloud > > Computing and Services Science - CLOSER 2013 > > > > > > > > Website: http://closer.scitevents.org/CI.aspx > > > > > > > > May 8 - 10, 2013 > > > > Aachen, Germany > > > > Co-organized by: RWTH > > > > Sponsored by: INSTICC > > > > INSTICC is Member of: WfMC and OMG > > > > Technically sponsored by: SINTEF and IFIP > > > > > > > > IMPORTANT DATES: > > > > > > > > Paper Submission: March 1, 2013 > > Authors Notification: March 19, 2013 > > Camera Ready and Registration: March 28, 2013 > > > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > > > > > > > > We would like to draw your attention to the Cloud Interoperability - > > CI 2013 (held in conjunction with the 3rd International Conference > > on Cloud Computing and Services Science- CLOSER 2013), whose > > submission deadline is scheduled for March 1, 2013. > > > > Cloud interoperability is one of the most challenging aspects of > > cloud computing. For the final user, interoperability means the > > possibility of migrating from one IaaS provider to another, or to > > manage in a seamless way the resources provided by distinct > > providers: thus the risk of being locked to a given provider is > > avoided, and cloud operation is made more reliable and secure. For > > the intermediate provider that offers value added services using a > > cloud infrastructure, interoperability means the possibility to > > offer the same service using different providers, without > > redesigning the application each time. Being the cloud a fast > > evolving creature, standardization proposals must be extremely > > flexible and not bound to current technologies or short term > > visions, while being simple to adhere to. As an alternative, some > > propose the design of intermediate layers that work as adapters for > > distinct providers. > > > > We invite practitioners, researchers and industry representatives to > > submit original ideas to the Closer Special Session on Cloud > > Interoperability: we will consider with special interest articles > > that: > > > > - demonstrate the impact of Cloud Interoperability with well > > documented use cases and experiments, > > - formalize aspects of cloud operation that are relevant for > > interoperability > > > > - describe standardization initiatives that aim at Cloud > > Interoperation. > > > > > > The conference will be sponsored by the Institute for Systems and > > Technologies of Information, Control and Communication (INSTICC), > > co-organized with the Aachen University (RWTH) and technically > > sponsored by the SINTEF and IFIP. INSTICC is Member of the Workflow > > Management Coalition (WfMC) and Object Management Group (OMG). > > > > > > > > The purpose of the 3rd International Conference on Cloud Computing > > and Services Science, CLOSER 2013, focuses on the emerging area of > > Cloud Computing, inspired by some latest advances that concern the > > infrastructure, operations, and available services through the > > global network. Further, the conference considers as essential the > > link to Services Science, acknowledging the service-orientation in > > most current IT-driven collaborations. The conference is > > nevertheless not about the union of these two (already broad) > > fields, but about Cloud Computing where we are also interested in > > how Services Science can provide theory, methods and techniques to > > design, analyze, manage, market and study various aspects of Cloud > > Computing. > > > > > > > > Submitted papers will be subject to a double-blind review process. > > > > All papers accepted to the Special Session will be published in a > > special section of the conference proceedings book, under an ISBN > > reference, on paper and on CD-ROM. > > > > > > > > The proceedings will be submitted for indexation by Thomson Reuters > > Conference Proceedings Citation Index (ISI), INSPEC, DBLP and EI > > (Elsevier Index). All papers presented at the conference venue will > > be available at the SCITEPRESS Digital Library > > (http://www.scitepress.org/DigitalLibrary/). SCITEPRESS is member of > > CrossRef (http://www.crossref.org/). > > > > > > > > CLOSER 2013 will be held in conjunction with WEBIST 2013 (9th > > International Conference on Web Information Systems and Technologies > > - http://www.webist.org/), SMARTGREENS 2013 (International > > Conference on Smart Grids and Green IT Systems - > > http://www.smartgreens.org/) and CSEDU 2013 (International > > Conference on Computer Supported Education - http://www.csedu.org/) > > in Aachen, Germany. > > > > Registration to CLOSER will enable free attendance to any sessions > > of WEBIST, SMARTGREENS and CSEDU conferences as a non-speaker. > > > > > > > > Please check further details at the Special Session website > > > > (http://closer.scitevents.org/CI.aspx) and the main conference > > website (http://closer.scitevents.org). > > > > > > > > CHAIR > > > > > > > > Augusto Ciuffoletti > > Universit? di Pisa > > Italy > > > > > > > > In case of questions, please feel free to contact us: > > closer.secretariat at scitevents.org > > > > > > > > PROGRAM COMMITTEE: > > > > Please check the special session?s program committee members at > > http://closer.scitevents.org/CI.aspx > > > > > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -- > ================================================================= > Ioan Raicu, Ph.D. > Assistant Professor, Illinois Institute of Technology (IIT) > Guest Research Faculty, Argonne National Laboratory (ANL) > ================================================================= > Data-Intensive Distributed Systems Laboratory, CS/IIT > Distributed Systems Laboratory, MCS/ANL > ================================================================= > Editor: IEEE TCC, Springer JoCCASA > Chair: IEEE/ACM MTAGS, ACM ScienceCloud, IEEE/ACM DataCloud > ================================================================= > Cel: 1-847-722-0876 > Office: 1-312-567-5704 > Email: iraicu at cs.iit.edu > Web: http://www.cs.iit.edu/~iraicu/ > Web: http://datasys.cs.iit.edu/ > LinkedIn: http://www.linkedin.com/in/ioanraicu > Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ > ================================================================= > ================================================================= > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From hategan at mcs.anl.gov Tue Feb 12 20:22:38 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 12 Feb 2013 18:22:38 -0800 Subject: [Swift-devel] faster branch Message-ID: <1360722158.14396.22.camel@echo> Hi, Over the past couple of weeks I committed the faster branch. It's essentially a re-implementation of the karajan language. The gist of it is that it is now a semi-compiled language. What I mean by that is that there is no code generation, so the initial parse tree is converted into a compiled tree. Variable and function references are resolved at compile-time. This provides considerable improvements in performance from two fronts: 1. The interpreter was changed from an event-based system (a system that made sense many years ago, but has little relevance for swift) to a lightweight threading implementation based on exceptions. 2. The compiler can make optimizations that weren't possible with a dynamic language. For example, once the swift and logging settings are known, code that conditionally executes based on a swift property or logging statements can be optimized away and have no run-time overhead. There is also a benefit in memory consumed. Variables are now stored in arrays rather than hashtables. This should reduce the run-time overhead of the engine. There are a few more relevant changes from swift's perspective: - there is no more karajan xml. Keeping two inter-operable syntaxes was a pain in the back. - sites.xml is now parsed with a DOM parser - all the coaster code has been moved into provider-coaster. Karajan is now independent of that (and the other way around). Most of the tests pass, but that means little. So if you can give it a shot, please do. If it works sufficiently well, we can merge this into trunk. Code is at: https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ https://svn.ci.uchicago.edu/svn/vdl2/branches/faster Mihael (PS: Contrary to my initial beliefs, the TUI seems to mostly work, but I haven't tested in detail). From yadudoc1729 at gmail.com Wed Feb 13 00:09:36 2013 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Wed, 13 Feb 2013 11:39:36 +0530 Subject: [Swift-devel] #swift-exm channel up for testing out Message-ID: Hi, I have set up an IRC channel for Swift discussions, please feel free to join in for discussions. Server : Freenode ( irc.freenode.net ) Channel : #swift-exm Here's a brief tutorial to get started : http://en.wikipedia.org/wiki/Wikipedia:IRC/Tutorial Please, make sure you register your nickname, so that someone else doesn't hijack it. Getting an IRC chat client is better, but if you just want to something quick, you could use http://webchat.freenode.net/ -- Regards, Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From lpesce at uchicago.edu Wed Feb 13 11:13:31 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Wed, 13 Feb 2013 11:13:31 -0600 Subject: [Swift-devel] faster branch In-Reply-To: <1360722158.14396.22.camel@echo> References: <1360722158.14396.22.camel@echo> Message-ID: <827D75DA-F9F2-4B1F-ACD5-1FC71E55CBC4@uchicago.edu> Mihael, Sounds great. Do you think that I can try and use it for medium scale simulations on Beagle as a test? (say 100 nodes, few thousands of concurrent jobs, just running for a couple of hours) The current versions of swift just can't handle the workload on Beagle, or at least don't seem to, and your changes seem to fit perfectly with our needs. Speed in handling complicated (or at least large) workflows and memory reductions are the weakest spot right now, so I am eager to try it. If you give me the green light, I can try and install it and then run it. It would be awesome because next week I need to get some fairly large (millions of tasks) workflows running. On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote: > Hi, > > Over the past couple of weeks I committed the faster branch. It's > essentially a re-implementation of the karajan language. > > The gist of it is that it is now a semi-compiled language. What I mean > by that is that there is no code generation, so the initial parse tree > is converted into a compiled tree. Variable and function references are > resolved at compile-time. This provides considerable improvements in > performance from two fronts: > > 1. The interpreter was changed from an event-based system (a system that > made sense many years ago, but has little relevance for swift) to a > lightweight threading implementation based on exceptions. > 2. The compiler can make optimizations that weren't possible with a > dynamic language. For example, once the swift and logging settings are > known, code that conditionally executes based on a swift property or > logging statements can be optimized away and have no run-time overhead. > > There is also a benefit in memory consumed. Variables are now stored in > arrays rather than hashtables. This should reduce the run-time overhead > of the engine. > > There are a few more relevant changes from swift's perspective: > - there is no more karajan xml. Keeping two inter-operable syntaxes was > a pain in the back. > - sites.xml is now parsed with a DOM parser > - all the coaster code has been moved into provider-coaster. Karajan is > now independent of that (and the other way around). > > Most of the tests pass, but that means little. So if you can give it a > shot, please do. If it works sufficiently well, we can merge this into > trunk. > > Code is at: > https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > > Mihael > > (PS: Contrary to my initial beliefs, the TUI seems to mostly work, but I > haven't tested in detail). > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From hategan at mcs.anl.gov Wed Feb 13 12:22:56 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 13 Feb 2013 10:22:56 -0800 Subject: [Swift-devel] faster branch In-Reply-To: <827D75DA-F9F2-4B1F-ACD5-1FC71E55CBC4@uchicago.edu> References: <1360722158.14396.22.camel@echo> <827D75DA-F9F2-4B1F-ACD5-1FC71E55CBC4@uchicago.edu> Message-ID: <1360779776.18488.4.camel@echo> On Wed, 2013-02-13 at 11:13 -0600, Lorenzo Pesce wrote: > Mihael, > > Sounds great. > > Do you think that I can try and use it for medium scale simulations on Beagle as a test? (say 100 nodes, few thousands of concurrent jobs, just running for a couple of hours) > > The current versions of swift just can't handle the workload on Beagle, or at least don't seem to, and your changes seem to fit perfectly with our needs. The change was motivated by your needs in part. > Speed in handling complicated (or at least large) workflows and memory reductions are the weakest spot right now, so I am eager to try it. > > If you give me the green light, I can try and install it and then run it. Please, go ahead as long as we have the understanding that there is a higher than normal likelihood of problems. > > It would be awesome because next week I need to get some fairly large (millions of tasks) workflows running. I can't promise this will work, but it might be worth a shot. Mihael From lpesce at uchicago.edu Wed Feb 13 13:08:07 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Wed, 13 Feb 2013 13:08:07 -0600 Subject: [Swift-devel] faster branch In-Reply-To: <1360779776.18488.4.camel@echo> References: <1360722158.14396.22.camel@echo> <827D75DA-F9F2-4B1F-ACD5-1FC71E55CBC4@uchicago.edu> <1360779776.18488.4.camel@echo> Message-ID: <6B63DC2A-0CE6-473A-B98E-8B763B42C576@uchicago.edu> Thanks a lot. Sounds interesting and sensible. I will try to put it to work ASAP, perhaps even today (I have to hack a genomics pipeline together and that would be a good test). We'll get swift use and running on Beagle squared up, I am looking forward to runs zillions of jobs on tens of steps of intertwined workflows. :-) BTW, today the second group claimed the name swift-seq for genomics analysis. We'll have a copyright internecine war... On Feb 13, 2013, at 12:22 PM, Mihael Hategan wrote: > On Wed, 2013-02-13 at 11:13 -0600, Lorenzo Pesce wrote: >> Mihael, >> >> Sounds great. >> >> Do you think that I can try and use it for medium scale simulations on Beagle as a test? (say 100 nodes, few thousands of concurrent jobs, just running for a couple of hours) >> >> The current versions of swift just can't handle the workload on Beagle, or at least don't seem to, and your changes seem to fit perfectly with our needs. > > The change was motivated by your needs in part. > >> Speed in handling complicated (or at least large) workflows and memory reductions are the weakest spot right now, so I am eager to try it. >> >> If you give me the green light, I can try and install it and then run it. > > Please, go ahead as long as we have the understanding that there is a > higher than normal likelihood of problems. > >> >> It would be awesome because next week I need to get some fairly large (millions of tasks) workflows running. > > I can't promise this will work, but it might be worth a shot. > > Mihael > > From iraicu at cs.iit.edu Wed Feb 13 14:46:37 2013 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Wed, 13 Feb 2013 14:46:37 -0600 Subject: [Swift-devel] CFP: ACM ScienceCloud 2013 -- deadline extension to 02/25/13 Message-ID: <511BFBAD.2080006@cs.iit.edu> ------------------------------------------------------------------------------- *** Call for Papers *** 4th Workshop on Scientific Cloud Computing (ScienceCloud) 2013 Co-located with ACM HPDC 2013, New York City, NY, USA -- June 17th, 2013 http://datasys.cs.iit.edu/events/ScienceCloud2013/ ------------------------------------------------------------------------------- *** Deadline Extension to February 25th, 2013 *** ------------------------------------------------------------------------------- The 4th workshop on Scientific Cloud Computing (ScienceCloud) will provide the scientific community a dedicated forum for discussing new research, development, and deployment efforts in running these kinds of scientific computing workloads on Cloud Computing infrastructures. The ScienceCloud workshop will focus on the use of cloud-based technologies to meet new compute-intensive and data- intensive scientific challenges that are not well served by the current supercomputers, grids and HPC clusters. This workshop encourages interaction and cross-pollination between those developing applications, algorithms, software, hardware and networking, emphasizing scientific computing for such cloud platforms. We believe the workshop will be an excellent place to help the community define the current state, determine future goals, and define architectures and services for future science clouds. TOPICS ------------------------------------------------------------------------------- We invite the submission of original work that is related to the topics below. The papers can be either short (4 pages) position papers, or long (8 pages) research papers. Topics of interest include (in the context of Cloud Computing): - Scientific application cases studies on Cloud infrastructure - Performance evaluation of Cloud environments and technologies - Fault tolerance and reliability in cloud systems - Data-intensive workloads and tools on Clouds - Use of programming models such as Map-Reduce and its implementations - Storage cloud architectures - I/O and Data management in the Cloud - Workflow and resource management in the Cloud - Use of cloud technologies (e.g., NoSQL databases) for scientific applications - Data streaming and dynamic applications on Clouds - Dynamic resource provisioning - Many-Task Computing in the Cloud - Application of cloud concepts in HPC environments or vice versa - High performance parallel file systems in virtual environments - Virtualized high performance I/O network interconnects - Virtualization - Distributed Operating Systems - Many-core computing and accelerators (e.g. GPUs, MIC) in the Cloud - Cloud security IMPORTANT DATES ------------------------------------------------------------------------------- - Paper submission: February 25th, 2013 (11:59PM PST) - Acceptance notification: March 18th, 2013 - Final papers due: April 15th, 2013 PAPER SUBMISSION ------------------------------------------------------------------------------- Authors are invited to submit papers with unpublished, original work of not more than 8 pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages (including all text, figures, and references), as per ACM 8.5 x 11 manuscript guidelines (document templates can be found at http://www.acm.org/sigs/publications/proceedings-templates). A 250 word abstract and the final paper in PDF format must be submitted online at https://cmt.research.microsoft.com/ScienceCloud2013/ before the deadline of February 25th, 2013 at 11:59PM PST. Papers will be peer-reviewed, and accepted papers will be published in the workshop proceedings as part of the ACM digital library. Notifications of the paper decisions will be sent out by March 18th, 2013. Selected excellent work will be invited to submit extended versions of the workshop paper to a special issue journal. Submission implies the willingness of at least one of the authors to register and present the paper. GENERAL CHAIRS ------------------------------------------------------------------------------- - Ioan Raicu, Illinois Institute of Technology & Argonne National Lab., USA - Yogesh Simmhan, University of Southern California, USA PROGRAM COMMITTEE CHAIRS ------------------------------------------------------------------------------- - Kyle Chard, University of Chicago, USA - Gabriel Antoniu, INRIA, France - Lavanya Ramakrishnan, Lawrence Berkeley National Lab, USA STEERING COMMITTEE ------------------------------------------------------------------------------- - Ian Foster, University of Chicago & Argonne National Laboratory, USA - Pete Beckman, University of Chicago & Argonne National Laboratory, USA - Carole Goble, University of Manchester, UK - Dennis Gannon, Microsoft Research, USA - Robert Grossman, University of Chicago, USA - Kate Keahey, University of Chicago & Argonne National Laboratory, USA - Ed Lazowska, University of Washington & Computing Community Consortium, USA - David O'Hallaron, Carnegie Mellon University & Intel Labs, USA - Jack Dongarra, University of Tennessee, USA - Geoffrey Fox, Indiana University, USA PROGRAM COMMITTEE ------------------------------------------------------------------------------- - Samer Al-Kiswany (University of British Columbia) - Roger Barga (Microsoft Research) - Roy Campbell (University of Illinois at Urbana Champaign) - Charlie Catlett (Argonne National Laboratory) - Simon Caton (KIT) - David Chiu (Washington State University) - Jack Dongara (University of Tennessee) - Ake Edlund (Royal Institute of Technology) - Chathura Herath (Indiana University) - Neil Chue Hong (University of Edinburgh) - Adriana Iamnitchi (University of South Florida) - Shantenu Jha (Louisiana State University) - Hui Jin (Illinois Institute of Technology) - Carl Kesselman (University of Southern California) - Thilo Kielmann (Vrije University) - Gregor von Laszewski (Indiana University) - Shiyong Lu (Wayne State University) - Wei Lu (Microsoft Research) - Andr Luckow (Louisiana State University) - David Martin (Argonne National Laboratory) - Gabriel Mateescu (Virginia Tech) - Paolo Missier (University of Manchester) - Ruben Montero (Universidad Complutense de Madrid) - Reagan Moore (University of North Carolina) - Jose Moreira (IBM Research) - Christine Morin (INRIA Rennes) - Pasquale Pagano (ISTI) - Beth Plale (Indiana University) - Omer Rana (Cardiff University) - Matei Ripeanu (University of British Columbia) - Josh Simons (VMWare) - Douglas Thain (University of Notre Dame) - Johan Tordsson (Ume University) - Vasudeva Varma (IIIT-Hyderabad) - Zhifeng Yun (Louisiana State University) - Yong Zhao (University of Electronic and Science Technology of China) -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud, IEEE/ACM DataCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= From lpesce at uchicago.edu Thu Feb 14 10:20:56 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Thu, 14 Feb 2013 10:20:56 -0600 Subject: [Swift-devel] faster branch In-Reply-To: <1360722158.14396.22.camel@echo> References: <1360722158.14396.22.camel@echo> Message-ID: <1ABD74FF-6657-4ECE-BF1C-38142FB72323@uchicago.edu> Sorry for being dumb here. It seems to be a bit different from the process defined on the web site: $ mkdir swift-0.93 $ cd swift-0.93 $ svn co https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog $ cd cog/modules $ svn co https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift $ cd swift $ ant redist what should I do? Just download the src and assume the rest is in? Sorry for the silly question, I would like to install it today, but I don't have the time to search for more details. On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote: > Hi, > > Over the past couple of weeks I committed the faster branch. It's > essentially a re-implementation of the karajan language. > > The gist of it is that it is now a semi-compiled language. What I mean > by that is that there is no code generation, so the initial parse tree > is converted into a compiled tree. Variable and function references are > resolved at compile-time. This provides considerable improvements in > performance from two fronts: > > 1. The interpreter was changed from an event-based system (a system that > made sense many years ago, but has little relevance for swift) to a > lightweight threading implementation based on exceptions. > 2. The compiler can make optimizations that weren't possible with a > dynamic language. For example, once the swift and logging settings are > known, code that conditionally executes based on a swift property or > logging statements can be optimized away and have no run-time overhead. > > There is also a benefit in memory consumed. Variables are now stored in > arrays rather than hashtables. This should reduce the run-time overhead > of the engine. > > There are a few more relevant changes from swift's perspective: > - there is no more karajan xml. Keeping two inter-operable syntaxes was > a pain in the back. > - sites.xml is now parsed with a DOM parser > - all the coaster code has been moved into provider-coaster. Karajan is > now independent of that (and the other way around). > > Most of the tests pass, but that means little. So if you can give it a > shot, please do. If it works sufficiently well, we can merge this into > trunk. > > Code is at: > https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > > Mihael > > (PS: Contrary to my initial beliefs, the TUI seems to mostly work, but I > haven't tested in detail). > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Thu Feb 14 10:32:25 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 14 Feb 2013 10:32:25 -0600 (CST) Subject: [Swift-devel] faster branch In-Reply-To: <1ABD74FF-6657-4ECE-BF1C-38142FB72323@uchicago.edu> Message-ID: <1072148968.525093.1360859545645.JavaMail.root@mcs.anl.gov> David, can you build this and place it in a Beagle module? A good exercise for Yadu as well. Thanks, - Mike ----- Original Message ----- > From: "Lorenzo Pesce" > To: "Mihael Hategan" > Cc: "Swift Devel" > Sent: Thursday, February 14, 2013 10:20:56 AM > Subject: Re: [Swift-devel] faster branch > > > Sorry for being dumb here. It seems to be a bit different from the > process defined on the web site: > $ mkdir swift-0.93 $ cd swift-0.93 $ svn co > https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog > $ cd cog/modules $ svn co > https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift $ > cd swift $ ant redist > > > what should I do? Just download the src and assume the rest is in? > > > Sorry for the silly question, I would like to install it today, but I > don't have the time to search for more details. > > > > > > > On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote: > > > > Hi, > > Over the past couple of weeks I committed the faster branch. It's > essentially a re-implementation of the karajan language. > > The gist of it is that it is now a semi-compiled language. What I > mean > by that is that there is no code generation, so the initial parse > tree > is converted into a compiled tree. Variable and function references > are > resolved at compile-time. This provides considerable improvements in > performance from two fronts: > > 1. The interpreter was changed from an event-based system (a system > that > made sense many years ago, but has little relevance for swift) to a > lightweight threading implementation based on exceptions. > 2. The compiler can make optimizations that weren't possible with a > dynamic language. For example, once the swift and logging settings > are > known, code that conditionally executes based on a swift property or > logging statements can be optimized away and have no run-time > overhead. > > There is also a benefit in memory consumed. Variables are now stored > in > arrays rather than hashtables. This should reduce the run-time > overhead > of the engine. > > There are a few more relevant changes from swift's perspective: > - there is no more karajan xml. Keeping two inter-operable syntaxes > was > a pain in the back. > - sites.xml is now parsed with a DOM parser > - all the coaster code has been moved into provider-coaster. Karajan > is > now independent of that (and the other way around). > > Most of the tests pass, but that means little. So if you can give it > a > shot, please do. If it works sufficiently well, we can merge this > into > trunk. > > Code is at: > https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > > Mihael > > (PS: Contrary to my initial beliefs, the TUI seems to mostly work, > but I > haven't tested in detail). > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From lpesce at uchicago.edu Thu Feb 14 11:32:45 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Thu, 14 Feb 2013 11:32:45 -0600 Subject: [Swift-devel] faster branch In-Reply-To: <1072148968.525093.1360859545645.JavaMail.root@mcs.anl.gov> References: <1072148968.525093.1360859545645.JavaMail.root@mcs.anl.gov> Message-ID: Thanks a lot Mike. Some day I would like to learn how to build them so I stop asking. I keep hoping I will have time, but it never seems to happen... On Feb 14, 2013, at 10:32 AM, Michael Wilde wrote: > David, can you build this and place it in a Beagle module? > > A good exercise for Yadu as well. > > Thanks, > > - Mike > > ----- Original Message ----- >> From: "Lorenzo Pesce" >> To: "Mihael Hategan" >> Cc: "Swift Devel" >> Sent: Thursday, February 14, 2013 10:20:56 AM >> Subject: Re: [Swift-devel] faster branch >> >> >> Sorry for being dumb here. It seems to be a bit different from the >> process defined on the web site: >> $ mkdir swift-0.93 $ cd swift-0.93 $ svn co >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog >> $ cd cog/modules $ svn co >> https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift $ >> cd swift $ ant redist >> >> >> what should I do? Just download the src and assume the rest is in? >> >> >> Sorry for the silly question, I would like to install it today, but I >> don't have the time to search for more details. >> >> >> >> >> >> >> On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote: >> >> >> >> Hi, >> >> Over the past couple of weeks I committed the faster branch. It's >> essentially a re-implementation of the karajan language. >> >> The gist of it is that it is now a semi-compiled language. What I >> mean >> by that is that there is no code generation, so the initial parse >> tree >> is converted into a compiled tree. Variable and function references >> are >> resolved at compile-time. This provides considerable improvements in >> performance from two fronts: >> >> 1. The interpreter was changed from an event-based system (a system >> that >> made sense many years ago, but has little relevance for swift) to a >> lightweight threading implementation based on exceptions. >> 2. The compiler can make optimizations that weren't possible with a >> dynamic language. For example, once the swift and logging settings >> are >> known, code that conditionally executes based on a swift property or >> logging statements can be optimized away and have no run-time >> overhead. >> >> There is also a benefit in memory consumed. Variables are now stored >> in >> arrays rather than hashtables. This should reduce the run-time >> overhead >> of the engine. >> >> There are a few more relevant changes from swift's perspective: >> - there is no more karajan xml. Keeping two inter-operable syntaxes >> was >> a pain in the back. >> - sites.xml is now parsed with a DOM parser >> - all the coaster code has been moved into provider-coaster. Karajan >> is >> now independent of that (and the other way around). >> >> Most of the tests pass, but that means little. So if you can give it >> a >> shot, please do. If it works sufficiently well, we can merge this >> into >> trunk. >> >> Code is at: >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ >> https://svn.ci.uchicago.edu/svn/vdl2/branches/faster >> >> Mihael >> >> (PS: Contrary to my initial beliefs, the TUI seems to mostly work, >> but I >> haven't tested in detail). >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> From davidk at ci.uchicago.edu Thu Feb 14 12:47:22 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Thu, 14 Feb 2013 12:47:22 -0600 (CST) Subject: [Swift-devel] faster branch In-Reply-To: Message-ID: <1760992605.2872848.1360867642909.JavaMail.root@ci.uchicago.edu> Lorenzo, I have a module available now on beagle called swift/fast. ----- Original Message ----- > From: "Lorenzo Pesce" > To: "Michael Wilde" > Cc: "Swift Devel" > Sent: Thursday, February 14, 2013 11:32:45 AM > Subject: Re: [Swift-devel] faster branch > Thanks a lot Mike. > Some day I would like to learn how to build them so I stop asking. I > keep hoping I will have time, but it never seems to happen... > On Feb 14, 2013, at 10:32 AM, Michael Wilde wrote: > > David, can you build this and place it in a Beagle module? > > > > A good exercise for Yadu as well. > > > > Thanks, > > > > - Mike > > > > ----- Original Message ----- > >> From: "Lorenzo Pesce" > >> To: "Mihael Hategan" > >> Cc: "Swift Devel" > >> Sent: Thursday, February 14, 2013 10:20:56 AM > >> Subject: Re: [Swift-devel] faster branch > >> > >> > >> Sorry for being dumb here. It seems to be a bit different from the > >> process defined on the web site: > >> $ mkdir swift-0.93 $ cd swift-0.93 $ svn co > >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog > >> $ cd cog/modules $ svn co > >> https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift $ > >> cd swift $ ant redist > >> > >> > >> what should I do? Just download the src and assume the rest is in? > >> > >> > >> Sorry for the silly question, I would like to install it today, > >> but I > >> don't have the time to search for more details. > >> > >> > >> > >> > >> > >> > >> On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote: > >> > >> > >> > >> Hi, > >> > >> Over the past couple of weeks I committed the faster branch. It's > >> essentially a re-implementation of the karajan language. > >> > >> The gist of it is that it is now a semi-compiled language. What I > >> mean > >> by that is that there is no code generation, so the initial parse > >> tree > >> is converted into a compiled tree. Variable and function > >> references > >> are > >> resolved at compile-time. This provides considerable improvements > >> in > >> performance from two fronts: > >> > >> 1. The interpreter was changed from an event-based system (a > >> system > >> that > >> made sense many years ago, but has little relevance for swift) to > >> a > >> lightweight threading implementation based on exceptions. > >> 2. The compiler can make optimizations that weren't possible with > >> a > >> dynamic language. For example, once the swift and logging settings > >> are > >> known, code that conditionally executes based on a swift property > >> or > >> logging statements can be optimized away and have no run-time > >> overhead. > >> > >> There is also a benefit in memory consumed. Variables are now > >> stored > >> in > >> arrays rather than hashtables. This should reduce the run-time > >> overhead > >> of the engine. > >> > >> There are a few more relevant changes from swift's perspective: > >> - there is no more karajan xml. Keeping two inter-operable > >> syntaxes > >> was > >> a pain in the back. > >> - sites.xml is now parsed with a DOM parser > >> - all the coaster code has been moved into provider-coaster. > >> Karajan > >> is > >> now independent of that (and the other way around). > >> > >> Most of the tests pass, but that means little. So if you can give > >> it > >> a > >> shot, please do. If it works sufficiently well, we can merge this > >> into > >> trunk. > >> > >> Code is at: > >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > >> https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > >> > >> Mihael > >> > >> (PS: Contrary to my initial beliefs, the TUI seems to mostly work, > >> but I > >> haven't tested in detail). > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >> > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >> > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Thu Feb 14 12:47:24 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 14 Feb 2013 12:47:24 -0600 (CST) Subject: [Swift-devel] 0.94 testing In-Reply-To: <457914193.540988.1360867110890.JavaMail.root@mcs.anl.gov> Message-ID: <72306501.541941.1360867644335.JavaMail.root@mcs.anl.gov> Hi David, Mihael, Ive asked Yadu to do the following as an initial test-enhancement activity: - write .swift script to run a large # of parallel non-app functions (1M, 10M) - write .swift script to do the same with an app() task; possibly with an external helper process to clean up files as they pile up. - add these to a controllable stress-test group, where the tester can adjust intensity of each subtest - gradually extend to remote sites and coasters, provider staging, etc. Mihael, Justin, can you in particular help Yadu with ideas, guidance, and if possible by contributing personal unit tests that you likely already have on hand from your own development work? David, I think its likely going to be a while before Yadu can move on to automating the site test battery. So for 0.94, lets push ahead towards a release using the same manual site testing as before. Then the longer-term roadmap for the test suite should include: - integration of MODIS tutorial/demo into the test suite - integration of DSSAT, AMWG, SciCol and OOPS-Loops into test suite - automation of site config tests in test suite - harmonization of Swift 1.0 (K) and Swift 2.0 (T) tests and frameworks We'll need more detailed plans, but I wanted to at least put down a roadmap for discussion. Thanks, - Mike -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory From wilde at mcs.anl.gov Thu Feb 14 13:08:56 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 14 Feb 2013 13:08:56 -0600 (CST) Subject: [Swift-devel] Fwd: 0.94 release note draft In-Reply-To: <406459202.1623059.1359090378666.JavaMail.root@ci.uchicago.edu> Message-ID: <1048561590.545450.1360868936831.JavaMail.root@mcs.anl.gov> Some notes toward an 0.94 release notes document. There's a longer list, I think in an IM chat transcript, that we need to incorporate. Please send additional items to this thread for David to integrate. Thanks, - Mike ----- Forwarded Message ----- From: "David Kelly" To: "Michael Wilde" Sent: Thursday, January 24, 2013 11:06:18 PM Subject: Re: 0.94 release note draft Mike, I just have the quick notes I took from our meeting. These combined with your emails are all the changes that I'm aware of at this point. Iterate differences Walltime hard limit with coasters Associate arrays changes Tracebacker Coaster changes / parameters Slurm and LSF providers Condor provider changes ssh-cl TUI hang checker @functions (strjoin, and possibly others) Dynamic profiles Wrapper staging Pass-thru (PBS attributes) MPI support ----- Original Message ----- > From: "Michael Wilde" > To: "David Kelly" > Sent: Thursday, January 24, 2013 7:56:54 PM > Subject: 0.94 release note draft > > > Hi David, > > I recall I sent you a few batches of line items to list in 0.94 > release notes. Did you gather those somewhere where I can review > them? (Need them for a status report) > > Thanks, > > - Mike > > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > > From lpesce at uchicago.edu Thu Feb 14 13:16:34 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Thu, 14 Feb 2013 13:16:34 -0600 Subject: [Swift-devel] faster branch In-Reply-To: <1760992605.2872848.1360867642909.JavaMail.root@ci.uchicago.edu> References: <1760992605.2872848.1360867642909.JavaMail.root@ci.uchicago.edu> Message-ID: <79578C42-DE8F-44B0-8883-3292499A24CB@uchicago.edu> Thanks a lot! On Feb 14, 2013, at 12:47 PM, David Kelly wrote: > Lorenzo, > > I have a module available now on beagle called swift/fast. > > From: "Lorenzo Pesce" > To: "Michael Wilde" > Cc: "Swift Devel" > Sent: Thursday, February 14, 2013 11:32:45 AM > Subject: Re: [Swift-devel] faster branch > > Thanks a lot Mike. > > Some day I would like to learn how to build them so I stop asking. I keep hoping I will have time, but it never seems to happen... > > On Feb 14, 2013, at 10:32 AM, Michael Wilde wrote: > > > David, can you build this and place it in a Beagle module? > > > > A good exercise for Yadu as well. > > > > Thanks, > > > > - Mike > > > > ----- Original Message ----- > >> From: "Lorenzo Pesce" > >> To: "Mihael Hategan" > >> Cc: "Swift Devel" > >> Sent: Thursday, February 14, 2013 10:20:56 AM > >> Subject: Re: [Swift-devel] faster branch > >> > >> > >> Sorry for being dumb here. It seems to be a bit different from the > >> process defined on the web site: > >> $ mkdir swift-0.93 $ cd swift-0.93 $ svn co > >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog > >> $ cd cog/modules $ svn co > >> https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift $ > >> cd swift $ ant redist > >> > >> > >> what should I do? Just download the src and assume the rest is in? > >> > >> > >> Sorry for the silly question, I would like to install it today, but I > >> don't have the time to search for more details. > >> > >> > >> > >> > >> > >> > >> On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote: > >> > >> > >> > >> Hi, > >> > >> Over the past couple of weeks I committed the faster branch. It's > >> essentially a re-implementation of the karajan language. > >> > >> The gist of it is that it is now a semi-compiled language. What I > >> mean > >> by that is that there is no code generation, so the initial parse > >> tree > >> is converted into a compiled tree. Variable and function references > >> are > >> resolved at compile-time. This provides considerable improvements in > >> performance from two fronts: > >> > >> 1. The interpreter was changed from an event-based system (a system > >> that > >> made sense many years ago, but has little relevance for swift) to a > >> lightweight threading implementation based on exceptions. > >> 2. The compiler can make optimizations that weren't possible with a > >> dynamic language. For example, once the swift and logging settings > >> are > >> known, code that conditionally executes based on a swift property or > >> logging statements can be optimized away and have no run-time > >> overhead. > >> > >> There is also a benefit in memory consumed. Variables are now stored > >> in > >> arrays rather than hashtables. This should reduce the run-time > >> overhead > >> of the engine. > >> > >> There are a few more relevant changes from swift's perspective: > >> - there is no more karajan xml. Keeping two inter-operable syntaxes > >> was > >> a pain in the back. > >> - sites.xml is now parsed with a DOM parser > >> - all the coaster code has been moved into provider-coaster. Karajan > >> is > >> now independent of that (and the other way around). > >> > >> Most of the tests pass, but that means little. So if you can give it > >> a > >> shot, please do. If it works sufficiently well, we can merge this > >> into > >> trunk. > >> > >> Code is at: > >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > >> https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > >> > >> Mihael > >> > >> (PS: Contrary to my initial beliefs, the TUI seems to mostly work, > >> but I > >> haven't tested in detail). > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >> > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >> > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Thu Feb 14 13:28:04 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 14 Feb 2013 13:28:04 -0600 (CST) Subject: [Swift-devel] faster branch In-Reply-To: <1360722158.14396.22.camel@echo> Message-ID: <1405994865.548723.1360870084737.JavaMail.root@mcs.anl.gov> Mihael, this is great. I'm eager to see us assemble a good set of stress tests into a test-group, and then apply it to both 0.94 and faster/. Then get faster/ into trunk, and build on it for 0.95. - Mike ----- Original Message ----- > From: "Mihael Hategan" > To: "Swift Devel" > Sent: Tuesday, February 12, 2013 8:22:38 PM > Subject: [Swift-devel] faster branch > > Hi, > > Over the past couple of weeks I committed the faster branch. It's > essentially a re-implementation of the karajan language. > > The gist of it is that it is now a semi-compiled language. What I > mean > by that is that there is no code generation, so the initial parse > tree > is converted into a compiled tree. Variable and function references > are > resolved at compile-time. This provides considerable improvements in > performance from two fronts: > > 1. The interpreter was changed from an event-based system (a system > that > made sense many years ago, but has little relevance for swift) to a > lightweight threading implementation based on exceptions. > 2. The compiler can make optimizations that weren't possible with a > dynamic language. For example, once the swift and logging settings > are > known, code that conditionally executes based on a swift property or > logging statements can be optimized away and have no run-time > overhead. > > There is also a benefit in memory consumed. Variables are now stored > in > arrays rather than hashtables. This should reduce the run-time > overhead > of the engine. > > There are a few more relevant changes from swift's perspective: > - there is no more karajan xml. Keeping two inter-operable syntaxes > was > a pain in the back. > - sites.xml is now parsed with a DOM parser > - all the coaster code has been moved into provider-coaster. Karajan > is > now independent of that (and the other way around). > > Most of the tests pass, but that means little. So if you can give it > a > shot, please do. If it works sufficiently well, we can merge this > into > trunk. > > Code is at: > https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > > Mihael > > (PS: Contrary to my initial beliefs, the TUI seems to mostly work, > but I > haven't tested in detail). > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From wilde at mcs.anl.gov Thu Feb 14 13:35:04 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 14 Feb 2013 13:35:04 -0600 (CST) Subject: [Swift-devel] wrapper staging In-Reply-To: <1338074612.29839.2.camel@blabla> Message-ID: <960437189.550084.1360870504295.JavaMail.root@mcs.anl.gov> Hi All, I'd like to resume the discussion of this feature, in particular how it might offer an effective way to a very user-friendly direct IO mechanism. Which experience is indicating that we urgently need. We need to do a case analysis, including: - workdir/jobdir on shared fs - jobdir on local fs - job dir below swift cwd (something we dont do but makes sense to consider) How input is handled How output is handled Who creates necessary dirs Also need to consider how various kinds of pathnames and URIs are treated: - /fully/qualified - relative - relative/structured - various URIs: gsiftp:// etc Mike ----- Original Message ----- > From: "Mihael Hategan" > To: "Swift Devel" > Sent: Saturday, May 26, 2012 6:23:32 PM > Subject: [Swift-devel] wrapper staging > > There has been some discussion in the past where we agreed that it > might > be a good idea to try a staging mechanism in which the wrapper (or > some > other entity on the worker node) does the staging. > > I added a skeleton for that in trunk. The relevant files are > vdl-int-wrapper-staging.k > and _swiftwrap.wrapperstaging. > > There is a basic implementation that recognizes "file://" URLs and > does > a cp for stage-ins and outs. > > Mihael > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From hategan at mcs.anl.gov Thu Feb 14 13:41:12 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 14 Feb 2013 11:41:12 -0800 Subject: [Swift-devel] faster branch In-Reply-To: <1405994865.548723.1360870084737.JavaMail.root@mcs.anl.gov> References: <1405994865.548723.1360870084737.JavaMail.root@mcs.anl.gov> Message-ID: <1360870872.32346.4.camel@echo> I agree. My tests are not very glamorous. I usually do the following for stress tests: - language/interpreter: foreach(largenumber), fake provider - coaster staging: many-cat, various file sizes - basic coasters: foreach(largenumber), local:local with some reasonable number of worker arrangements (i.e. 2 blocks * 2 workers per block * 2 concurrent jobs per worker). I think these are pretty easy to construct (which is why I use them), but I'll be happy to help with the details. Mihael On Thu, 2013-02-14 at 13:28 -0600, Michael Wilde wrote: > Mihael, this is great. > > I'm eager to see us assemble a good set of stress tests into a test-group, and then apply it to both 0.94 and faster/. > > Then get faster/ into trunk, and build on it for 0.95. > > - Mike > > ----- Original Message ----- > > From: "Mihael Hategan" > > To: "Swift Devel" > > Sent: Tuesday, February 12, 2013 8:22:38 PM > > Subject: [Swift-devel] faster branch > > > > Hi, > > > > Over the past couple of weeks I committed the faster branch. It's > > essentially a re-implementation of the karajan language. > > > > The gist of it is that it is now a semi-compiled language. What I > > mean > > by that is that there is no code generation, so the initial parse > > tree > > is converted into a compiled tree. Variable and function references > > are > > resolved at compile-time. This provides considerable improvements in > > performance from two fronts: > > > > 1. The interpreter was changed from an event-based system (a system > > that > > made sense many years ago, but has little relevance for swift) to a > > lightweight threading implementation based on exceptions. > > 2. The compiler can make optimizations that weren't possible with a > > dynamic language. For example, once the swift and logging settings > > are > > known, code that conditionally executes based on a swift property or > > logging statements can be optimized away and have no run-time > > overhead. > > > > There is also a benefit in memory consumed. Variables are now stored > > in > > arrays rather than hashtables. This should reduce the run-time > > overhead > > of the engine. > > > > There are a few more relevant changes from swift's perspective: > > - there is no more karajan xml. Keeping two inter-operable syntaxes > > was > > a pain in the back. > > - sites.xml is now parsed with a DOM parser > > - all the coaster code has been moved into provider-coaster. Karajan > > is > > now independent of that (and the other way around). > > > > Most of the tests pass, but that means little. So if you can give it > > a > > shot, please do. If it works sufficiently well, we can merge this > > into > > trunk. > > > > Code is at: > > https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > > https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > > > > Mihael > > > > (PS: Contrary to my initial beliefs, the TUI seems to mostly work, > > but I > > haven't tested in detail). > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > From lpesce at uchicago.edu Mon Feb 18 10:37:21 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Mon, 18 Feb 2013 10:37:21 -0600 Subject: [Swift-devel] faster branch In-Reply-To: <1760992605.2872848.1360867642909.JavaMail.root@ci.uchicago.edu> References: <1760992605.2872848.1360867642909.JavaMail.root@ci.uchicago.edu> Message-ID: Thanks again. I hope to start testing it today on a medium sized genomics pipeline. I will keep you posted on the developments. On Feb 14, 2013, at 12:47 PM, David Kelly wrote: > Lorenzo, > > I have a module available now on beagle called swift/fast. > > From: "Lorenzo Pesce" > To: "Michael Wilde" > Cc: "Swift Devel" > Sent: Thursday, February 14, 2013 11:32:45 AM > Subject: Re: [Swift-devel] faster branch > > Thanks a lot Mike. > > Some day I would like to learn how to build them so I stop asking. I keep hoping I will have time, but it never seems to happen... > > On Feb 14, 2013, at 10:32 AM, Michael Wilde wrote: > > > David, can you build this and place it in a Beagle module? > > > > A good exercise for Yadu as well. > > > > Thanks, > > > > - Mike > > > > ----- Original Message ----- > >> From: "Lorenzo Pesce" > >> To: "Mihael Hategan" > >> Cc: "Swift Devel" > >> Sent: Thursday, February 14, 2013 10:20:56 AM > >> Subject: Re: [Swift-devel] faster branch > >> > >> > >> Sorry for being dumb here. It seems to be a bit different from the > >> process defined on the web site: > >> $ mkdir swift-0.93 $ cd swift-0.93 $ svn co > >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/4.1.9/src/cog > >> $ cd cog/modules $ svn co > >> https://svn.ci.uchicago.edu/svn/vdl2/branches/release-0.93 swift $ > >> cd swift $ ant redist > >> > >> > >> what should I do? Just download the src and assume the rest is in? > >> > >> > >> Sorry for the silly question, I would like to install it today, but I > >> don't have the time to search for more details. > >> > >> > >> > >> > >> > >> > >> On Feb 12, 2013, at 8:22 PM, Mihael Hategan wrote: > >> > >> > >> > >> Hi, > >> > >> Over the past couple of weeks I committed the faster branch. It's > >> essentially a re-implementation of the karajan language. > >> > >> The gist of it is that it is now a semi-compiled language. What I > >> mean > >> by that is that there is no code generation, so the initial parse > >> tree > >> is converted into a compiled tree. Variable and function references > >> are > >> resolved at compile-time. This provides considerable improvements in > >> performance from two fronts: > >> > >> 1. The interpreter was changed from an event-based system (a system > >> that > >> made sense many years ago, but has little relevance for swift) to a > >> lightweight threading implementation based on exceptions. > >> 2. The compiler can make optimizations that weren't possible with a > >> dynamic language. For example, once the swift and logging settings > >> are > >> known, code that conditionally executes based on a swift property or > >> logging statements can be optimized away and have no run-time > >> overhead. > >> > >> There is also a benefit in memory consumed. Variables are now stored > >> in > >> arrays rather than hashtables. This should reduce the run-time > >> overhead > >> of the engine. > >> > >> There are a few more relevant changes from swift's perspective: > >> - there is no more karajan xml. Keeping two inter-operable syntaxes > >> was > >> a pain in the back. > >> - sites.xml is now parsed with a DOM parser > >> - all the coaster code has been moved into provider-coaster. Karajan > >> is > >> now independent of that (and the other way around). > >> > >> Most of the tests pass, but that means little. So if you can give it > >> a > >> shot, please do. If it works sufficiently well, we can merge this > >> into > >> trunk. > >> > >> Code is at: > >> https://cogkit.svn.sourceforge.net/svnroot/cogkit/branches/faster/ > >> https://svn.ci.uchicago.edu/svn/vdl2/branches/faster > >> > >> Mihael > >> > >> (PS: Contrary to my initial beliefs, the TUI seems to mostly work, > >> but I > >> haven't tested in detail). > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >> > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > >> > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lpesce at uchicago.edu Tue Feb 19 13:26:20 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Tue, 19 Feb 2013 13:26:20 -0600 Subject: [Swift-devel] First tests with swift faster References: Message-ID: This is the content of the file where we have the first complaint from swift (see attached): CI-DEB000002 pbs.aprun;pbs.mpp;depth=24 24 172800 0:10:00 100 100 200 1 1 47.99 10000 /lustre/beagle/samseaver/GS/swift.workdir Any ideas? Begin forwarded message: > From: Sam Seaver > Date: February 19, 2013 1:16:28 PM CST > To: Lorenzo Pesce > Subject: Re: How are things going? > > I got this error. I suspect using the new SWIFT_HOME directory means that there's possibly a missing parameter someplace: > > should we resume a previous calculation? [y/N] y > rlog files displayed in reverse time order > should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] > y > Using GS-20130203-0717-jgeppt98.0.rlog > [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of element 'config'. > > Execution failed: > Failed to parse site catalog > swift:siteCatalog @ scheduler.k, line: 31 > Caused by: Invalid pool entry 'pbs': > swift:siteCatalog @ scheduler.k, line: 31 > Caused by: java.lang.IllegalArgumentException: Missing URL > at org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) > at org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) > at org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) > at org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) > at org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > at org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > at k.thr.LWThread.run(LWThread.java:243) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > > > On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver wrote: > OK, it got to the point where it really did hang. I'm retrying, but with your suggestions. The other three finished fine! > > Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > > > On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce wrote: > Hmm... > > foreach.max.threads=100 > > maybe you should increase this number a bit and see what happens. > > Also, I would try to replace > > SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 > > with > > SWIFT_HOME=/soft/swift/fast > > Keep me posted. Let's get this rolling. > > if it doesn't work, I can redo the packing. > > > > > On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: > >> Actually, the ten agents job does seem to be stuck in a strange loop. It is incrementing the number of jobs that has finished successfully, and at a fast pace, but the number of jobs its starting is decrementing much more slowly, its almost as its repeatedly attempting the same set of parameters multiple times... >> >> I'll see what it's doing in the morning >> S >> >> >> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver wrote: >> Seems to have worked overall this time! >> >> I resume four jobs, each were for a different number of agents (10,100,1000,10000) it made it easier for me to decide on the app time. Two of them have already finished i.e.: >> >> Progress: time: Mon, 18 Feb 2013 23:50:12 +0000 Active:4 Checking status:1 Finished in previous run:148098 Finished successfully:37897 >> Progress: time: Mon, 18 Feb 2013 23:50:15 +0000 Active:2 Checking status:1 Finished in previous run:148098 Finished successfully:37899 >> Final status: Mon, 18 Feb 2013 23:50:15 +0000 Finished in previous run:148098 Finished successfully:37902 >> >> and the only one that is showing any failure (50/110000), is the ten agents version which is so short I can understand why, but its still actively trying to run jobs and is actively finishing jobs, so that's good. >> >> Yay! >> >> >> >> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo Pesce wrote: >> Good. Keep me posted, I would really like to solve your problems in running on Beagle this week, I wish that Swift would have been friendlier. >> >> On Feb 18, 2013, at 1:01 PM, Sam Seaver wrote: >> >>> I just resumed the jobs that I'd killed before the system went down, lets see how it does. I always did a mini-review of the data I've got and it seems to be working as expected. >>> >>> >>> On Mon, Feb 18, 2013 at 12:28 PM, Lorenzo Pesce wrote: >>> I have lost track a bit of what's up. I am happy to try and go over it with you when you are ready. >>> >>> Some of the problems of swift might have improved with a new version and the new system. >>> >>> >>> On Feb 18, 2013, at 12:22 PM, Sam Seaver wrote: >>> >>>> They're not, I've not looked since Beagle came back up. Will do so later today. >>>> S >>>> >>>> >>>> On Mon, Feb 18, 2013 at 12:20 PM, Lorenzo Pesce wrote: >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>> >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >> >> >> >> >> -- >> Postdoctoral Fellow >> Mathematics and Computer Science Division >> Argonne National Laboratory >> 9700 S. Cass Avenue >> Argonne, IL 60439 >> >> http://www.linkedin.com/pub/sam-seaver/0/412/168 >> samseaver at gmail.com >> (773) 796-7144 >> >> "We shall not cease from exploration >> And the end of all our exploring >> Will be to arrive where we started >> And know the place for the first time." >> --T. S. Eliot >> >> >> >> -- >> Postdoctoral Fellow >> Mathematics and Computer Science Division >> Argonne National Laboratory >> 9700 S. Cass Avenue >> Argonne, IL 60439 >> >> http://www.linkedin.com/pub/sam-seaver/0/412/168 >> samseaver at gmail.com >> (773) 796-7144 >> >> "We shall not cease from exploration >> And the end of all our exploring >> Will be to arrive where we started >> And know the place for the first time." >> --T. S. Eliot > > > > > -- > Postdoctoral Fellow > Mathematics and Computer Science Division > Argonne National Laboratory > 9700 S. Cass Avenue > Argonne, IL 60439 > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > samseaver at gmail.com > (773) 796-7144 > > "We shall not cease from exploration > And the end of all our exploring > Will be to arrive where we started > And know the place for the first time." > --T. S. Eliot > > > > -- > Postdoctoral Fellow > Mathematics and Computer Science Division > Argonne National Laboratory > 9700 S. Cass Avenue > Argonne, IL 60439 > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > samseaver at gmail.com > (773) 796-7144 > > "We shall not cease from exploration > And the end of all our exploring > Will be to arrive where we started > And know the place for the first time." > --T. S. Eliot -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Feb 19 13:52:56 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 19 Feb 2013 11:52:56 -0800 Subject: [Swift-devel] First tests with swift faster In-Reply-To: References: Message-ID: <1361303576.31289.5.camel@echo> The way the site catalog is parsed has changed. I committed a fix in svn. A workaround is to make sure you specify a url: ... ... ... The URL is actually required for the coaster provider. Mihael On Tue, 2013-02-19 at 13:26 -0600, Lorenzo Pesce wrote: > This is the content of the file where we have the first complaint from swift (see attached): > > > > > CI-DEB000002 > > pbs.aprun;pbs.mpp;depth=24 > > > 24 > 172800 > 0:10:00 > 100 > 100 > > 200 > 1 > 1 > > 47.99 > 10000 > > > > /lustre/beagle/samseaver/GS/swift.workdir > > > > Any ideas? > > Begin forwarded message: > > > From: Sam Seaver > > Date: February 19, 2013 1:16:28 PM CST > > To: Lorenzo Pesce > > Subject: Re: How are things going? > > > > I got this error. I suspect using the new SWIFT_HOME directory means that there's possibly a missing parameter someplace: > > > > should we resume a previous calculation? [y/N] y > > rlog files displayed in reverse time order > > should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] > > y > > Using GS-20130203-0717-jgeppt98.0.rlog > > [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of element 'config'. > > > > Execution failed: > > Failed to parse site catalog > > swift:siteCatalog @ scheduler.k, line: 31 > > Caused by: Invalid pool entry 'pbs': > > swift:siteCatalog @ scheduler.k, line: 31 > > Caused by: java.lang.IllegalArgumentException: Missing URL > > at org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) > > at org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) > > at org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) > > at org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) > > at org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > at org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) > > at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > > at k.thr.LWThread.run(LWThread.java:243) > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:722) > > > > > > On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver wrote: > > OK, it got to the point where it really did hang. I'm retrying, but with your suggestions. The other three finished fine! > > > > Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > > > > > > On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce wrote: > > Hmm... > > > > foreach.max.threads=100 > > > > maybe you should increase this number a bit and see what happens. > > > > Also, I would try to replace > > > > SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 > > > > with > > > > SWIFT_HOME=/soft/swift/fast > > > > Keep me posted. Let's get this rolling. > > > > if it doesn't work, I can redo the packing. > > > > > > > > > > On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: > > > >> Actually, the ten agents job does seem to be stuck in a strange loop. It is incrementing the number of jobs that has finished successfully, and at a fast pace, but the number of jobs its starting is decrementing much more slowly, its almost as its repeatedly attempting the same set of parameters multiple times... > >> > >> I'll see what it's doing in the morning > >> S > >> > >> > >> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver wrote: > >> Seems to have worked overall this time! > >> > >> I resume four jobs, each were for a different number of agents (10,100,1000,10000) it made it easier for me to decide on the app time. Two of them have already finished i.e.: > >> > >> Progress: time: Mon, 18 Feb 2013 23:50:12 +0000 Active:4 Checking status:1 Finished in previous run:148098 Finished successfully:37897 > >> Progress: time: Mon, 18 Feb 2013 23:50:15 +0000 Active:2 Checking status:1 Finished in previous run:148098 Finished successfully:37899 > >> Final status: Mon, 18 Feb 2013 23:50:15 +0000 Finished in previous run:148098 Finished successfully:37902 > >> > >> and the only one that is showing any failure (50/110000), is the ten agents version which is so short I can understand why, but its still actively trying to run jobs and is actively finishing jobs, so that's good. > >> > >> Yay! > >> > >> > >> > >> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo Pesce wrote: > >> Good. Keep me posted, I would really like to solve your problems in running on Beagle this week, I wish that Swift would have been friendlier. > >> > >> On Feb 18, 2013, at 1:01 PM, Sam Seaver wrote: > >> > >>> I just resumed the jobs that I'd killed before the system went down, lets see how it does. I always did a mini-review of the data I've got and it seems to be working as expected. > >>> > >>> > >>> On Mon, Feb 18, 2013 at 12:28 PM, Lorenzo Pesce wrote: > >>> I have lost track a bit of what's up. I am happy to try and go over it with you when you are ready. > >>> > >>> Some of the problems of swift might have improved with a new version and the new system. > >>> > >>> > >>> On Feb 18, 2013, at 12:22 PM, Sam Seaver wrote: > >>> > >>>> They're not, I've not looked since Beagle came back up. Will do so later today. > >>>> S > >>>> > >>>> > >>>> On Mon, Feb 18, 2013 at 12:20 PM, Lorenzo Pesce wrote: > >>>> > >>>> > >>>> > >>>> -- > >>>> Postdoctoral Fellow > >>>> Mathematics and Computer Science Division > >>>> Argonne National Laboratory > >>>> 9700 S. Cass Avenue > >>>> Argonne, IL 60439 > >>>> > >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>> samseaver at gmail.com > >>>> (773) 796-7144 > >>>> > >>>> "We shall not cease from exploration > >>>> And the end of all our exploring > >>>> Will be to arrive where we started > >>>> And know the place for the first time." > >>>> --T. S. Eliot > >>> > >>> > >>> > >>> > >>> -- > >>> Postdoctoral Fellow > >>> Mathematics and Computer Science Division > >>> Argonne National Laboratory > >>> 9700 S. Cass Avenue > >>> Argonne, IL 60439 > >>> > >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>> samseaver at gmail.com > >>> (773) 796-7144 > >>> > >>> "We shall not cease from exploration > >>> And the end of all our exploring > >>> Will be to arrive where we started > >>> And know the place for the first time." > >>> --T. S. Eliot > >> > >> > >> > >> > >> -- > >> Postdoctoral Fellow > >> Mathematics and Computer Science Division > >> Argonne National Laboratory > >> 9700 S. Cass Avenue > >> Argonne, IL 60439 > >> > >> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >> samseaver at gmail.com > >> (773) 796-7144 > >> > >> "We shall not cease from exploration > >> And the end of all our exploring > >> Will be to arrive where we started > >> And know the place for the first time." > >> --T. S. Eliot > >> > >> > >> > >> -- > >> Postdoctoral Fellow > >> Mathematics and Computer Science Division > >> Argonne National Laboratory > >> 9700 S. Cass Avenue > >> Argonne, IL 60439 > >> > >> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >> samseaver at gmail.com > >> (773) 796-7144 > >> > >> "We shall not cease from exploration > >> And the end of all our exploring > >> Will be to arrive where we started > >> And know the place for the first time." > >> --T. S. Eliot > > > > > > > > > > -- > > Postdoctoral Fellow > > Mathematics and Computer Science Division > > Argonne National Laboratory > > 9700 S. Cass Avenue > > Argonne, IL 60439 > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > samseaver at gmail.com > > (773) 796-7144 > > > > "We shall not cease from exploration > > And the end of all our exploring > > Will be to arrive where we started > > And know the place for the first time." > > --T. S. Eliot > > > > > > > > -- > > Postdoctoral Fellow > > Mathematics and Computer Science Division > > Argonne National Laboratory > > 9700 S. Cass Avenue > > Argonne, IL 60439 > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > samseaver at gmail.com > > (773) 796-7144 > > > > "We shall not cease from exploration > > And the end of all our exploring > > Will be to arrive where we started > > And know the place for the first time." > > --T. S. Eliot > > This is the content of the file where we have the first complaint from > swift (see attached): > > > > > CI-DEB000002 > > > key="providerAttributes">pbs.aprun;pbs.mpp;depth=24 > > > > > 24 > 172800 > 0:10:00 > 100 > 100 > > > 200 > 1 > 1 > > > 47.99 > 10000 > > > > > > /lustre/beagle/samseaver/GS/swift.workdir > > > > > Any ideas? > > Begin forwarded message: > > > From: Sam Seaver > > > > Date: February 19, 2013 1:16:28 PM CST > > > > To: Lorenzo Pesce > > > > Subject: Re: How are things going? > > > > > > I got this error. I suspect using the new SWIFT_HOME directory > > means that there's possibly a missing parameter someplace: > > > > > > should we resume a previous calculation? [y/N] y > > rlog files displayed in reverse time order > > should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] > > y > > Using GS-20130203-0717-jgeppt98.0.rlog > > [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of > > element 'config'. > > > > > > Execution failed: > > Failed to parse site catalog > > swift:siteCatalog @ scheduler.k, line: 31 > > Caused by: Invalid pool entry 'pbs': > > swift:siteCatalog @ scheduler.k, line: 31 > > Caused by: java.lang.IllegalArgumentException: Missing URL > > at > > org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) > > at > > org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) > > at > > org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) > > at > > org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) > > at > > org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > at > > org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) > > at > > org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > > at > > org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > > at > > org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > > at > > org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > > at k.thr.LWThread.run(LWThread.java:243) > > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at java.util.concurrent.ThreadPoolExecutor > > $Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:722) > > > > > > On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver > > wrote: > > OK, it got to the point where it really did hang. I'm > > retrying, but with your suggestions. The other three > > finished fine! > > > > > > Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting > > site:18147 Submitted:174 Active:96 Failed:2 Finished > > successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting > > site:18147 Submitted:174 Active:96 Failed:2 Finished > > successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting > > site:18147 Submitted:174 Active:96 Failed:2 Finished > > successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting > > site:18147 Submitted:174 Active:96 Failed:2 Finished > > successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting > > site:18147 Submitted:174 Active:96 Failed:2 Finished > > successfully:132323 Failed but can retry:183 > > Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting > > site:18147 Submitted:174 Active:96 Failed:2 Finished > > successfully:132323 Failed but can retry:183 > > > > > > On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce > > wrote: > > Hmm... > > > > > > foreach.max.threads=100 > > > > > > maybe you should increase this number a bit and see > > what happens. > > > > > > Also, I would try to replace > > > > > > SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 > > > > > > with > > > > > > SWIFT_HOME=/soft/swift/fast > > > > > > Keep me posted. Let's get this rolling. > > > > > > if it doesn't work, I can redo the packing. > > > > > > > > > > > > > > > > On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: > > > > > Actually, the ten agents job does seem to be stuck > > > in a strange loop. It is incrementing the number > > > of jobs that has finished successfully, and at a > > > fast pace, but the number of jobs its starting is > > > decrementing much more slowly, its almost as its > > > repeatedly attempting the same set of parameters > > > multiple times... > > > > > > > > > I'll see what it's doing in the morning > > > S > > > > > > > > > On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver > > > wrote: > > > Seems to have worked overall this time! > > > > > > > > > I resume four jobs, each were for a > > > different number of agents > > > (10,100,1000,10000) it made it easier for > > > me to decide on the app time. Two of them > > > have already finished i.e.: > > > > > > > > > Progress: time: Mon, 18 Feb 2013 23:50:12 > > > +0000 Active:4 Checking status:1 > > > Finished in previous run:148098 Finished > > > successfully:37897 > > > Progress: time: Mon, 18 Feb 2013 23:50:15 > > > +0000 Active:2 Checking status:1 > > > Finished in previous run:148098 Finished > > > successfully:37899 > > > Final status: Mon, 18 Feb 2013 23:50:15 > > > +0000 Finished in previous run:148098 > > > Finished successfully:37902 > > > > > > > > > and the only one that is showing any > > > failure (50/110000), is the ten agents > > > version which is so short I can understand > > > why, but its still actively trying to run > > > jobs and is actively finishing jobs, so > > > that's good. > > > > > > > > > Yay! > > > > > > > > > > > > > > > On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo > > > Pesce wrote: > > > Good. Keep me posted, I would > > > really like to solve your problems > > > in running on Beagle this week, I > > > wish that Swift would have been > > > friendlier. > > > > > > On Feb 18, 2013, at 1:01 PM, Sam > > > Seaver wrote: > > > > > > > I just resumed the jobs that I'd > > > > killed before the system went > > > > down, lets see how it does. I > > > > always did a mini-review of the > > > > data I've got and it seems to be > > > > working as expected. > > > > > > > > > > > > On Mon, Feb 18, 2013 at 12:28 > > > > PM, Lorenzo Pesce > > > > wrote: > > > > I have lost track a bit > > > > of what's up. I am happy > > > > to try and go over it > > > > with you when you are > > > > ready. > > > > > > > > > > > > Some of the problems of > > > > swift might have > > > > improved with a new > > > > version and the new > > > > system. > > > > > > > > > > > > > > > > On Feb 18, 2013, at > > > > 12:22 PM, Sam Seaver > > > > wrote: > > > > > > > > > They're not, I've not > > > > > looked since Beagle > > > > > came back up. Will do > > > > > so later today. > > > > > S > > > > > > > > > > > > > > > On Mon, Feb 18, 2013 > > > > > at 12:20 PM, Lorenzo > > > > > Pesce > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Postdoctoral Fellow > > > > > Mathematics and > > > > > Computer Science > > > > > Division > > > > > Argonne National > > > > > Laboratory > > > > > 9700 S. Cass Avenue > > > > > Argonne, IL 60439 > > > > > > > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > > > > samseaver at gmail.com > > > > > (773) 796-7144 > > > > > > > > > > "We shall not cease > > > > > from exploration > > > > > And the end of all our > > > > > exploring > > > > > Will be to arrive > > > > > where we started > > > > > And know the place for > > > > > the first time." > > > > > --T. S. Eliot > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Postdoctoral Fellow > > > > Mathematics and Computer Science > > > > Division > > > > Argonne National Laboratory > > > > 9700 S. Cass Avenue > > > > Argonne, IL 60439 > > > > > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > > > samseaver at gmail.com > > > > (773) 796-7144 > > > > > > > > "We shall not cease from > > > > exploration > > > > And the end of all our exploring > > > > Will be to arrive where we > > > > started > > > > And know the place for the first > > > > time." > > > > --T. S. Eliot > > > > > > > > > > > > > > > > > > > > > -- > > > Postdoctoral Fellow > > > Mathematics and Computer Science Division > > > Argonne National Laboratory > > > 9700 S. Cass Avenue > > > Argonne, IL 60439 > > > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > > samseaver at gmail.com > > > (773) 796-7144 > > > > > > "We shall not cease from exploration > > > And the end of all our exploring > > > Will be to arrive where we started > > > And know the place for the first time." > > > --T. S. Eliot > > > > > > > > > > > > > > > -- > > > Postdoctoral Fellow > > > Mathematics and Computer Science Division > > > Argonne National Laboratory > > > 9700 S. Cass Avenue > > > Argonne, IL 60439 > > > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > > samseaver at gmail.com > > > (773) 796-7144 > > > > > > "We shall not cease from exploration > > > And the end of all our exploring > > > Will be to arrive where we started > > > And know the place for the first time." > > > --T. S. Eliot > > > > > > > > > > > > > > -- > > Postdoctoral Fellow > > Mathematics and Computer Science Division > > Argonne National Laboratory > > 9700 S. Cass Avenue > > Argonne, IL 60439 > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > samseaver at gmail.com > > (773) 796-7144 > > > > "We shall not cease from exploration > > And the end of all our exploring > > Will be to arrive where we started > > And know the place for the first time." > > --T. S. Eliot > > > > > > > > > > -- > > Postdoctoral Fellow > > Mathematics and Computer Science Division > > Argonne National Laboratory > > 9700 S. Cass Avenue > > Argonne, IL 60439 > > > > http://www.linkedin.com/pub/sam-seaver/0/412/168 > > samseaver at gmail.com > > (773) 796-7144 > > > > "We shall not cease from exploration > > And the end of all our exploring > > Will be to arrive where we started > > And know the place for the first time." > > --T. S. Eliot > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From lpesce at uchicago.edu Tue Feb 19 14:22:08 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Tue, 19 Feb 2013 14:22:08 -0600 Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1361303576.31289.5.camel@echo> References: <1361303576.31289.5.camel@echo> Message-ID: <99418EAD-AE31-4729-BFAC-A0DF78EA01C3@uchicago.edu> Thanks. Implemented, being tested by user. On Feb 19, 2013, at 1:52 PM, Mihael Hategan wrote: > The way the site catalog is parsed has changed. I committed a fix in > svn. A workaround is to make sure you specify a url: > > ... > > ... > > ... > > The URL is actually required for the coaster provider. > > Mihael > > On Tue, 2013-02-19 at 13:26 -0600, Lorenzo Pesce wrote: >> This is the content of the file where we have the first complaint from swift (see attached): >> >> >> >> >> CI-DEB000002 >> >> pbs.aprun;pbs.mpp;depth=24 >> >> >> 24 >> 172800 >> 0:10:00 >> 100 >> 100 >> >> 200 >> 1 >> 1 >> >> 47.99 >> 10000 >> >> >> >> /lustre/beagle/samseaver/GS/swift.workdir >> >> >> >> Any ideas? >> >> Begin forwarded message: >> >>> From: Sam Seaver >>> Date: February 19, 2013 1:16:28 PM CST >>> To: Lorenzo Pesce >>> Subject: Re: How are things going? >>> >>> I got this error. I suspect using the new SWIFT_HOME directory means that there's possibly a missing parameter someplace: >>> >>> should we resume a previous calculation? [y/N] y >>> rlog files displayed in reverse time order >>> should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] >>> y >>> Using GS-20130203-0717-jgeppt98.0.rlog >>> [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of element 'config'. >>> >>> Execution failed: >>> Failed to parse site catalog >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: Invalid pool entry 'pbs': >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: java.lang.IllegalArgumentException: Missing URL >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) >>> at org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>> at k.thr.LWThread.run(LWThread.java:243) >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >>> at java.lang.Thread.run(Thread.java:722) >>> >>> >>> On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver wrote: >>> OK, it got to the point where it really did hang. I'm retrying, but with your suggestions. The other three finished fine! >>> >>> Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> >>> >>> On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce wrote: >>> Hmm... >>> >>> foreach.max.threads=100 >>> >>> maybe you should increase this number a bit and see what happens. >>> >>> Also, I would try to replace >>> >>> SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 >>> >>> with >>> >>> SWIFT_HOME=/soft/swift/fast >>> >>> Keep me posted. Let's get this rolling. >>> >>> if it doesn't work, I can redo the packing. >>> >>> >>> >>> >>> On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: >>> >>>> Actually, the ten agents job does seem to be stuck in a strange loop. It is incrementing the number of jobs that has finished successfully, and at a fast pace, but the number of jobs its starting is decrementing much more slowly, its almost as its repeatedly attempting the same set of parameters multiple times... >>>> >>>> I'll see what it's doing in the morning >>>> S >>>> >>>> >>>> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver wrote: >>>> Seems to have worked overall this time! >>>> >>>> I resume four jobs, each were for a different number of agents (10,100,1000,10000) it made it easier for me to decide on the app time. Two of them have already finished i.e.: >>>> >>>> Progress: time: Mon, 18 Feb 2013 23:50:12 +0000 Active:4 Checking status:1 Finished in previous run:148098 Finished successfully:37897 >>>> Progress: time: Mon, 18 Feb 2013 23:50:15 +0000 Active:2 Checking status:1 Finished in previous run:148098 Finished successfully:37899 >>>> Final status: Mon, 18 Feb 2013 23:50:15 +0000 Finished in previous run:148098 Finished successfully:37902 >>>> >>>> and the only one that is showing any failure (50/110000), is the ten agents version which is so short I can understand why, but its still actively trying to run jobs and is actively finishing jobs, so that's good. >>>> >>>> Yay! >>>> >>>> >>>> >>>> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo Pesce wrote: >>>> Good. Keep me posted, I would really like to solve your problems in running on Beagle this week, I wish that Swift would have been friendlier. >>>> >>>> On Feb 18, 2013, at 1:01 PM, Sam Seaver wrote: >>>> >>>>> I just resumed the jobs that I'd killed before the system went down, lets see how it does. I always did a mini-review of the data I've got and it seems to be working as expected. >>>>> >>>>> >>>>> On Mon, Feb 18, 2013 at 12:28 PM, Lorenzo Pesce wrote: >>>>> I have lost track a bit of what's up. I am happy to try and go over it with you when you are ready. >>>>> >>>>> Some of the problems of swift might have improved with a new version and the new system. >>>>> >>>>> >>>>> On Feb 18, 2013, at 12:22 PM, Sam Seaver wrote: >>>>> >>>>>> They're not, I've not looked since Beagle came back up. Will do so later today. >>>>>> S >>>>>> >>>>>> >>>>>> On Mon, Feb 18, 2013 at 12:20 PM, Lorenzo Pesce wrote: >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Postdoctoral Fellow >>>>>> Mathematics and Computer Science Division >>>>>> Argonne National Laboratory >>>>>> 9700 S. Cass Avenue >>>>>> Argonne, IL 60439 >>>>>> >>>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>>> samseaver at gmail.com >>>>>> (773) 796-7144 >>>>>> >>>>>> "We shall not cease from exploration >>>>>> And the end of all our exploring >>>>>> Will be to arrive where we started >>>>>> And know the place for the first time." >>>>>> --T. S. Eliot >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Postdoctoral Fellow >>>>> Mathematics and Computer Science Division >>>>> Argonne National Laboratory >>>>> 9700 S. Cass Avenue >>>>> Argonne, IL 60439 >>>>> >>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>> samseaver at gmail.com >>>>> (773) 796-7144 >>>>> >>>>> "We shall not cease from exploration >>>>> And the end of all our exploring >>>>> Will be to arrive where we started >>>>> And know the place for the first time." >>>>> --T. S. Eliot >>>> >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>> >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >> >> This is the content of the file where we have the first complaint from >> swift (see attached): >> >> >> >> >> CI-DEB000002 >> >> >> > key="providerAttributes">pbs.aprun;pbs.mpp;depth=24 >> >> >> >> >> 24 >> 172800 >> 0:10:00 >> 100 >> 100 >> >> >> 200 >> 1 >> 1 >> >> >> 47.99 >> 10000 >> >> >> >> >> >> /lustre/beagle/samseaver/GS/swift.workdir >> >> >> >> >> Any ideas? >> >> Begin forwarded message: >> >>> From: Sam Seaver >>> >>> Date: February 19, 2013 1:16:28 PM CST >>> >>> To: Lorenzo Pesce >>> >>> Subject: Re: How are things going? >>> >>> >>> I got this error. I suspect using the new SWIFT_HOME directory >>> means that there's possibly a missing parameter someplace: >>> >>> >>> should we resume a previous calculation? [y/N] y >>> rlog files displayed in reverse time order >>> should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] >>> y >>> Using GS-20130203-0717-jgeppt98.0.rlog >>> [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of >>> element 'config'. >>> >>> >>> Execution failed: >>> Failed to parse site catalog >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: Invalid pool entry 'pbs': >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: java.lang.IllegalArgumentException: Missing URL >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) >>> at >>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at >>> org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at >>> org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>> at k.thr.LWThread.run(LWThread.java:243) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >>> at java.util.concurrent.ThreadPoolExecutor >>> $Worker.run(ThreadPoolExecutor.java:603) >>> at java.lang.Thread.run(Thread.java:722) >>> >>> >>> On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver >>> wrote: >>> OK, it got to the point where it really did hang. I'm >>> retrying, but with your suggestions. The other three >>> finished fine! >>> >>> >>> Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> >>> >>> On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce >>> wrote: >>> Hmm... >>> >>> >>> foreach.max.threads=100 >>> >>> >>> maybe you should increase this number a bit and see >>> what happens. >>> >>> >>> Also, I would try to replace >>> >>> >>> SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 >>> >>> >>> with >>> >>> >>> SWIFT_HOME=/soft/swift/fast >>> >>> >>> Keep me posted. Let's get this rolling. >>> >>> >>> if it doesn't work, I can redo the packing. >>> >>> >>> >>> >>> >>> >>> >>> On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: >>> >>>> Actually, the ten agents job does seem to be stuck >>>> in a strange loop. It is incrementing the number >>>> of jobs that has finished successfully, and at a >>>> fast pace, but the number of jobs its starting is >>>> decrementing much more slowly, its almost as its >>>> repeatedly attempting the same set of parameters >>>> multiple times... >>>> >>>> >>>> I'll see what it's doing in the morning >>>> S >>>> >>>> >>>> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver >>>> wrote: >>>> Seems to have worked overall this time! >>>> >>>> >>>> I resume four jobs, each were for a >>>> different number of agents >>>> (10,100,1000,10000) it made it easier for >>>> me to decide on the app time. Two of them >>>> have already finished i.e.: >>>> >>>> >>>> Progress: time: Mon, 18 Feb 2013 23:50:12 >>>> +0000 Active:4 Checking status:1 >>>> Finished in previous run:148098 Finished >>>> successfully:37897 >>>> Progress: time: Mon, 18 Feb 2013 23:50:15 >>>> +0000 Active:2 Checking status:1 >>>> Finished in previous run:148098 Finished >>>> successfully:37899 >>>> Final status: Mon, 18 Feb 2013 23:50:15 >>>> +0000 Finished in previous run:148098 >>>> Finished successfully:37902 >>>> >>>> >>>> and the only one that is showing any >>>> failure (50/110000), is the ten agents >>>> version which is so short I can understand >>>> why, but its still actively trying to run >>>> jobs and is actively finishing jobs, so >>>> that's good. >>>> >>>> >>>> Yay! >>>> >>>> >>>> >>>> >>>> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo >>>> Pesce wrote: >>>> Good. Keep me posted, I would >>>> really like to solve your problems >>>> in running on Beagle this week, I >>>> wish that Swift would have been >>>> friendlier. >>>> >>>> On Feb 18, 2013, at 1:01 PM, Sam >>>> Seaver wrote: >>>> >>>>> I just resumed the jobs that I'd >>>>> killed before the system went >>>>> down, lets see how it does. I >>>>> always did a mini-review of the >>>>> data I've got and it seems to be >>>>> working as expected. >>>>> >>>>> >>>>> On Mon, Feb 18, 2013 at 12:28 >>>>> PM, Lorenzo Pesce >>>>> wrote: >>>>> I have lost track a bit >>>>> of what's up. I am happy >>>>> to try and go over it >>>>> with you when you are >>>>> ready. >>>>> >>>>> >>>>> Some of the problems of >>>>> swift might have >>>>> improved with a new >>>>> version and the new >>>>> system. >>>>> >>>>> >>>>> >>>>> On Feb 18, 2013, at >>>>> 12:22 PM, Sam Seaver >>>>> wrote: >>>>> >>>>>> They're not, I've not >>>>>> looked since Beagle >>>>>> came back up. Will do >>>>>> so later today. >>>>>> S >>>>>> >>>>>> >>>>>> On Mon, Feb 18, 2013 >>>>>> at 12:20 PM, Lorenzo >>>>>> Pesce >>>>>> >>>>>> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Postdoctoral Fellow >>>>>> Mathematics and >>>>>> Computer Science >>>>>> Division >>>>>> Argonne National >>>>>> Laboratory >>>>>> 9700 S. Cass Avenue >>>>>> Argonne, IL 60439 >>>>>> >>>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>>> samseaver at gmail.com >>>>>> (773) 796-7144 >>>>>> >>>>>> "We shall not cease >>>>>> from exploration >>>>>> And the end of all our >>>>>> exploring >>>>>> Will be to arrive >>>>>> where we started >>>>>> And know the place for >>>>>> the first time." >>>>>> --T. S. Eliot >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Postdoctoral Fellow >>>>> Mathematics and Computer Science >>>>> Division >>>>> Argonne National Laboratory >>>>> 9700 S. Cass Avenue >>>>> Argonne, IL 60439 >>>>> >>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>> samseaver at gmail.com >>>>> (773) 796-7144 >>>>> >>>>> "We shall not cease from >>>>> exploration >>>>> And the end of all our exploring >>>>> Will be to arrive where we >>>>> started >>>>> And know the place for the first >>>>> time." >>>>> --T. S. Eliot >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>>> >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>> >>> >>> >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >>> >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > From lpesce at uchicago.edu Wed Feb 20 14:17:15 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Wed, 20 Feb 2013 14:17:15 -0600 Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1361303576.31289.5.camel@echo> References: <1361303576.31289.5.camel@echo> Message-ID: <63B3E3BC-A90A-40C2-AFA4-AD9D6B4AFEA9@uchicago.edu> Trying with a different project. I have no idea here either. Could not initialize shared directory on pbs exception @ swift-int.k, line: 380 Caused by: null Caused by: org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: java.lang.NullPointerException Caused by: java.lang.NullPointerException at org.globus.cog.abstraction.impl.common.task.ServiceImpl.hashCode(ServiceImpl.java:171) at java.util.HashMap.getEntry(HashMap.java:344) at java.util.HashMap.containsKey(HashMap.java:335) at org.globus.cog.abstraction.impl.file.FileResourceCache.getResource(FileResourceCache.java:77) at org.globus.cog.abstraction.impl.file.CachingDelegatedFileOperationHandler.getResource(CachingDelegatedFileOperationHandler.java:75) at org.globus.cog.abstraction.impl.file.CachingDelegatedFileOperationHandler.submit(CachingDelegatedFileOperationHandler.java:40) at org.globus.cog.abstraction.impl.common.task.CachingFileOperationTaskHandler.submit(CachingFileOperationTaskHandler.java:28) at org.globus.cog.karajan.scheduler.submitQueue.NonBlockingSubmit.run(NonBlockingSubmit.java:113) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) parallelFor @ swift.k, line: 206 Thanks! Lorenzo On Feb 19, 2013, at 1:52 PM, Mihael Hategan wrote: > The way the site catalog is parsed has changed. I committed a fix in > svn. A workaround is to make sure you specify a url: > > ... > > ... > > ... > > The URL is actually required for the coaster provider. > > Mihael > > On Tue, 2013-02-19 at 13:26 -0600, Lorenzo Pesce wrote: >> This is the content of the file where we have the first complaint from swift (see attached): >> >> >> >> >> CI-DEB000002 >> >> pbs.aprun;pbs.mpp;depth=24 >> >> >> 24 >> 172800 >> 0:10:00 >> 100 >> 100 >> >> 200 >> 1 >> 1 >> >> 47.99 >> 10000 >> >> >> >> /lustre/beagle/samseaver/GS/swift.workdir >> >> >> >> Any ideas? >> >> Begin forwarded message: >> >>> From: Sam Seaver >>> Date: February 19, 2013 1:16:28 PM CST >>> To: Lorenzo Pesce >>> Subject: Re: How are things going? >>> >>> I got this error. I suspect using the new SWIFT_HOME directory means that there's possibly a missing parameter someplace: >>> >>> should we resume a previous calculation? [y/N] y >>> rlog files displayed in reverse time order >>> should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] >>> y >>> Using GS-20130203-0717-jgeppt98.0.rlog >>> [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of element 'config'. >>> >>> Execution failed: >>> Failed to parse site catalog >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: Invalid pool entry 'pbs': >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: java.lang.IllegalArgumentException: Missing URL >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) >>> at org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>> at k.thr.LWThread.run(LWThread.java:243) >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >>> at java.lang.Thread.run(Thread.java:722) >>> >>> >>> On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver wrote: >>> OK, it got to the point where it really did hang. I'm retrying, but with your suggestions. The other three finished fine! >>> >>> Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 >>> >>> >>> On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce wrote: >>> Hmm... >>> >>> foreach.max.threads=100 >>> >>> maybe you should increase this number a bit and see what happens. >>> >>> Also, I would try to replace >>> >>> SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 >>> >>> with >>> >>> SWIFT_HOME=/soft/swift/fast >>> >>> Keep me posted. Let's get this rolling. >>> >>> if it doesn't work, I can redo the packing. >>> >>> >>> >>> >>> On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: >>> >>>> Actually, the ten agents job does seem to be stuck in a strange loop. It is incrementing the number of jobs that has finished successfully, and at a fast pace, but the number of jobs its starting is decrementing much more slowly, its almost as its repeatedly attempting the same set of parameters multiple times... >>>> >>>> I'll see what it's doing in the morning >>>> S >>>> >>>> >>>> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver wrote: >>>> Seems to have worked overall this time! >>>> >>>> I resume four jobs, each were for a different number of agents (10,100,1000,10000) it made it easier for me to decide on the app time. Two of them have already finished i.e.: >>>> >>>> Progress: time: Mon, 18 Feb 2013 23:50:12 +0000 Active:4 Checking status:1 Finished in previous run:148098 Finished successfully:37897 >>>> Progress: time: Mon, 18 Feb 2013 23:50:15 +0000 Active:2 Checking status:1 Finished in previous run:148098 Finished successfully:37899 >>>> Final status: Mon, 18 Feb 2013 23:50:15 +0000 Finished in previous run:148098 Finished successfully:37902 >>>> >>>> and the only one that is showing any failure (50/110000), is the ten agents version which is so short I can understand why, but its still actively trying to run jobs and is actively finishing jobs, so that's good. >>>> >>>> Yay! >>>> >>>> >>>> >>>> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo Pesce wrote: >>>> Good. Keep me posted, I would really like to solve your problems in running on Beagle this week, I wish that Swift would have been friendlier. >>>> >>>> On Feb 18, 2013, at 1:01 PM, Sam Seaver wrote: >>>> >>>>> I just resumed the jobs that I'd killed before the system went down, lets see how it does. I always did a mini-review of the data I've got and it seems to be working as expected. >>>>> >>>>> >>>>> On Mon, Feb 18, 2013 at 12:28 PM, Lorenzo Pesce wrote: >>>>> I have lost track a bit of what's up. I am happy to try and go over it with you when you are ready. >>>>> >>>>> Some of the problems of swift might have improved with a new version and the new system. >>>>> >>>>> >>>>> On Feb 18, 2013, at 12:22 PM, Sam Seaver wrote: >>>>> >>>>>> They're not, I've not looked since Beagle came back up. Will do so later today. >>>>>> S >>>>>> >>>>>> >>>>>> On Mon, Feb 18, 2013 at 12:20 PM, Lorenzo Pesce wrote: >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Postdoctoral Fellow >>>>>> Mathematics and Computer Science Division >>>>>> Argonne National Laboratory >>>>>> 9700 S. Cass Avenue >>>>>> Argonne, IL 60439 >>>>>> >>>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>>> samseaver at gmail.com >>>>>> (773) 796-7144 >>>>>> >>>>>> "We shall not cease from exploration >>>>>> And the end of all our exploring >>>>>> Will be to arrive where we started >>>>>> And know the place for the first time." >>>>>> --T. S. Eliot >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Postdoctoral Fellow >>>>> Mathematics and Computer Science Division >>>>> Argonne National Laboratory >>>>> 9700 S. Cass Avenue >>>>> Argonne, IL 60439 >>>>> >>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>> samseaver at gmail.com >>>>> (773) 796-7144 >>>>> >>>>> "We shall not cease from exploration >>>>> And the end of all our exploring >>>>> Will be to arrive where we started >>>>> And know the place for the first time." >>>>> --T. S. Eliot >>>> >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>> >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >> >> This is the content of the file where we have the first complaint from >> swift (see attached): >> >> >> >> >> CI-DEB000002 >> >> >> > key="providerAttributes">pbs.aprun;pbs.mpp;depth=24 >> >> >> >> >> 24 >> 172800 >> 0:10:00 >> 100 >> 100 >> >> >> 200 >> 1 >> 1 >> >> >> 47.99 >> 10000 >> >> >> >> >> >> /lustre/beagle/samseaver/GS/swift.workdir >> >> >> >> >> Any ideas? >> >> Begin forwarded message: >> >>> From: Sam Seaver >>> >>> Date: February 19, 2013 1:16:28 PM CST >>> >>> To: Lorenzo Pesce >>> >>> Subject: Re: How are things going? >>> >>> >>> I got this error. I suspect using the new SWIFT_HOME directory >>> means that there's possibly a missing parameter someplace: >>> >>> >>> should we resume a previous calculation? [y/N] y >>> rlog files displayed in reverse time order >>> should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] >>> y >>> Using GS-20130203-0717-jgeppt98.0.rlog >>> [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of >>> element 'config'. >>> >>> >>> Execution failed: >>> Failed to parse site catalog >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: Invalid pool entry 'pbs': >>> swift:siteCatalog @ scheduler.k, line: 31 >>> Caused by: java.lang.IllegalArgumentException: Missing URL >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) >>> at >>> org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) >>> at >>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at >>> org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) >>> at >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) >>> at >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) >>> at >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) >>> at >>> org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) >>> at k.thr.LWThread.run(LWThread.java:243) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >>> at java.util.concurrent.ThreadPoolExecutor >>> $Worker.run(ThreadPoolExecutor.java:603) >>> at java.lang.Thread.run(Thread.java:722) >>> >>> >>> On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver >>> wrote: >>> OK, it got to the point where it really did hang. I'm >>> retrying, but with your suggestions. The other three >>> finished fine! >>> >>> >>> Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting >>> site:18147 Submitted:174 Active:96 Failed:2 Finished >>> successfully:132323 Failed but can retry:183 >>> >>> >>> On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce >>> wrote: >>> Hmm... >>> >>> >>> foreach.max.threads=100 >>> >>> >>> maybe you should increase this number a bit and see >>> what happens. >>> >>> >>> Also, I would try to replace >>> >>> >>> SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 >>> >>> >>> with >>> >>> >>> SWIFT_HOME=/soft/swift/fast >>> >>> >>> Keep me posted. Let's get this rolling. >>> >>> >>> if it doesn't work, I can redo the packing. >>> >>> >>> >>> >>> >>> >>> >>> On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: >>> >>>> Actually, the ten agents job does seem to be stuck >>>> in a strange loop. It is incrementing the number >>>> of jobs that has finished successfully, and at a >>>> fast pace, but the number of jobs its starting is >>>> decrementing much more slowly, its almost as its >>>> repeatedly attempting the same set of parameters >>>> multiple times... >>>> >>>> >>>> I'll see what it's doing in the morning >>>> S >>>> >>>> >>>> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver >>>> wrote: >>>> Seems to have worked overall this time! >>>> >>>> >>>> I resume four jobs, each were for a >>>> different number of agents >>>> (10,100,1000,10000) it made it easier for >>>> me to decide on the app time. Two of them >>>> have already finished i.e.: >>>> >>>> >>>> Progress: time: Mon, 18 Feb 2013 23:50:12 >>>> +0000 Active:4 Checking status:1 >>>> Finished in previous run:148098 Finished >>>> successfully:37897 >>>> Progress: time: Mon, 18 Feb 2013 23:50:15 >>>> +0000 Active:2 Checking status:1 >>>> Finished in previous run:148098 Finished >>>> successfully:37899 >>>> Final status: Mon, 18 Feb 2013 23:50:15 >>>> +0000 Finished in previous run:148098 >>>> Finished successfully:37902 >>>> >>>> >>>> and the only one that is showing any >>>> failure (50/110000), is the ten agents >>>> version which is so short I can understand >>>> why, but its still actively trying to run >>>> jobs and is actively finishing jobs, so >>>> that's good. >>>> >>>> >>>> Yay! >>>> >>>> >>>> >>>> >>>> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo >>>> Pesce wrote: >>>> Good. Keep me posted, I would >>>> really like to solve your problems >>>> in running on Beagle this week, I >>>> wish that Swift would have been >>>> friendlier. >>>> >>>> On Feb 18, 2013, at 1:01 PM, Sam >>>> Seaver wrote: >>>> >>>>> I just resumed the jobs that I'd >>>>> killed before the system went >>>>> down, lets see how it does. I >>>>> always did a mini-review of the >>>>> data I've got and it seems to be >>>>> working as expected. >>>>> >>>>> >>>>> On Mon, Feb 18, 2013 at 12:28 >>>>> PM, Lorenzo Pesce >>>>> wrote: >>>>> I have lost track a bit >>>>> of what's up. I am happy >>>>> to try and go over it >>>>> with you when you are >>>>> ready. >>>>> >>>>> >>>>> Some of the problems of >>>>> swift might have >>>>> improved with a new >>>>> version and the new >>>>> system. >>>>> >>>>> >>>>> >>>>> On Feb 18, 2013, at >>>>> 12:22 PM, Sam Seaver >>>>> wrote: >>>>> >>>>>> They're not, I've not >>>>>> looked since Beagle >>>>>> came back up. Will do >>>>>> so later today. >>>>>> S >>>>>> >>>>>> >>>>>> On Mon, Feb 18, 2013 >>>>>> at 12:20 PM, Lorenzo >>>>>> Pesce >>>>>> >>>>>> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Postdoctoral Fellow >>>>>> Mathematics and >>>>>> Computer Science >>>>>> Division >>>>>> Argonne National >>>>>> Laboratory >>>>>> 9700 S. Cass Avenue >>>>>> Argonne, IL 60439 >>>>>> >>>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>>> samseaver at gmail.com >>>>>> (773) 796-7144 >>>>>> >>>>>> "We shall not cease >>>>>> from exploration >>>>>> And the end of all our >>>>>> exploring >>>>>> Will be to arrive >>>>>> where we started >>>>>> And know the place for >>>>>> the first time." >>>>>> --T. S. Eliot >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Postdoctoral Fellow >>>>> Mathematics and Computer Science >>>>> Division >>>>> Argonne National Laboratory >>>>> 9700 S. Cass Avenue >>>>> Argonne, IL 60439 >>>>> >>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>>> samseaver at gmail.com >>>>> (773) 796-7144 >>>>> >>>>> "We shall not cease from >>>>> exploration >>>>> And the end of all our exploring >>>>> Will be to arrive where we >>>>> started >>>>> And know the place for the first >>>>> time." >>>>> --T. S. Eliot >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>>> >>>> >>>> >>>> >>>> -- >>>> Postdoctoral Fellow >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 S. Cass Avenue >>>> Argonne, IL 60439 >>>> >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>>> samseaver at gmail.com >>>> (773) 796-7144 >>>> >>>> "We shall not cease from exploration >>>> And the end of all our exploring >>>> Will be to arrive where we started >>>> And know the place for the first time." >>>> --T. S. Eliot >>> >>> >>> >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >>> >>> >>> >>> >>> -- >>> Postdoctoral Fellow >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 S. Cass Avenue >>> Argonne, IL 60439 >>> >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 >>> samseaver at gmail.com >>> (773) 796-7144 >>> >>> "We shall not cease from exploration >>> And the end of all our exploring >>> Will be to arrive where we started >>> And know the place for the first time." >>> --T. S. Eliot >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > From hategan at mcs.anl.gov Wed Feb 20 14:55:12 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 20 Feb 2013 12:55:12 -0800 Subject: [Swift-devel] First tests with swift faster In-Reply-To: <63B3E3BC-A90A-40C2-AFA4-AD9D6B4AFEA9@uchicago.edu> References: <1361303576.31289.5.camel@echo> <63B3E3BC-A90A-40C2-AFA4-AD9D6B4AFEA9@uchicago.edu> Message-ID: <1361393712.8356.1.camel@echo> Same idea, except with the filesystem tag. It needs a URL. Also, the job manager attribute should be "jobManager", not jobmanager. I'm working on proper validation of the sites file so that these issues are clearly stated and checked. Mihael On Wed, 2013-02-20 at 14:17 -0600, Lorenzo Pesce wrote: > Trying with a different project. I have no idea here either. > > Could not initialize shared directory on pbs > exception @ swift-int.k, line: 380 > Caused by: null > Caused by: org.globus.cog.abstraction.impl.common.task.TaskSubmissionException: java.lang.NullPointerException > Caused by: java.lang.NullPointerException > at org.globus.cog.abstraction.impl.common.task.ServiceImpl.hashCode(ServiceImpl.java:171) > at java.util.HashMap.getEntry(HashMap.java:344) > at java.util.HashMap.containsKey(HashMap.java:335) > at org.globus.cog.abstraction.impl.file.FileResourceCache.getResource(FileResourceCache.java:77) > at org.globus.cog.abstraction.impl.file.CachingDelegatedFileOperationHandler.getResource(CachingDelegatedFileOperationHandler.java:75) > at org.globus.cog.abstraction.impl.file.CachingDelegatedFileOperationHandler.submit(CachingDelegatedFileOperationHandler.java:40) > at org.globus.cog.abstraction.impl.common.task.CachingFileOperationTaskHandler.submit(CachingFileOperationTaskHandler.java:28) > at org.globus.cog.karajan.scheduler.submitQueue.NonBlockingSubmit.run(NonBlockingSubmit.java:113) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > > parallelFor @ swift.k, line: 206 > > Thanks! > > Lorenzo > > On Feb 19, 2013, at 1:52 PM, Mihael Hategan wrote: > > > The way the site catalog is parsed has changed. I committed a fix in > > svn. A workaround is to make sure you specify a url: > > > > ... > > > > ... > > > > ... > > > > The URL is actually required for the coaster provider. > > > > Mihael > > > > On Tue, 2013-02-19 at 13:26 -0600, Lorenzo Pesce wrote: > >> This is the content of the file where we have the first complaint from swift (see attached): > >> > >> > >> > >> > >> CI-DEB000002 > >> > >> pbs.aprun;pbs.mpp;depth=24 > >> > >> > >> 24 > >> 172800 > >> 0:10:00 > >> 100 > >> 100 > >> > >> 200 > >> 1 > >> 1 > >> > >> 47.99 > >> 10000 > >> > >> > >> > >> /lustre/beagle/samseaver/GS/swift.workdir > >> > >> > >> > >> Any ideas? > >> > >> Begin forwarded message: > >> > >>> From: Sam Seaver > >>> Date: February 19, 2013 1:16:28 PM CST > >>> To: Lorenzo Pesce > >>> Subject: Re: How are things going? > >>> > >>> I got this error. I suspect using the new SWIFT_HOME directory means that there's possibly a missing parameter someplace: > >>> > >>> should we resume a previous calculation? [y/N] y > >>> rlog files displayed in reverse time order > >>> should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] > >>> y > >>> Using GS-20130203-0717-jgeppt98.0.rlog > >>> [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of element 'config'. > >>> > >>> Execution failed: > >>> Failed to parse site catalog > >>> swift:siteCatalog @ scheduler.k, line: 31 > >>> Caused by: Invalid pool entry 'pbs': > >>> swift:siteCatalog @ scheduler.k, line: 31 > >>> Caused by: java.lang.IllegalArgumentException: Missing URL > >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) > >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) > >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) > >>> at org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) > >>> at org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > >>> at org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) > >>> at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > >>> at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > >>> at org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > >>> at k.thr.LWThread.run(LWThread.java:243) > >>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > >>> at java.lang.Thread.run(Thread.java:722) > >>> > >>> > >>> On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver wrote: > >>> OK, it got to the point where it really did hang. I'm retrying, but with your suggestions. The other three finished fine! > >>> > >>> Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting site:18147 Submitted:174 Active:96 Failed:2 Finished successfully:132323 Failed but can retry:183 > >>> > >>> > >>> On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce wrote: > >>> Hmm... > >>> > >>> foreach.max.threads=100 > >>> > >>> maybe you should increase this number a bit and see what happens. > >>> > >>> Also, I would try to replace > >>> > >>> SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 > >>> > >>> with > >>> > >>> SWIFT_HOME=/soft/swift/fast > >>> > >>> Keep me posted. Let's get this rolling. > >>> > >>> if it doesn't work, I can redo the packing. > >>> > >>> > >>> > >>> > >>> On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: > >>> > >>>> Actually, the ten agents job does seem to be stuck in a strange loop. It is incrementing the number of jobs that has finished successfully, and at a fast pace, but the number of jobs its starting is decrementing much more slowly, its almost as its repeatedly attempting the same set of parameters multiple times... > >>>> > >>>> I'll see what it's doing in the morning > >>>> S > >>>> > >>>> > >>>> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver wrote: > >>>> Seems to have worked overall this time! > >>>> > >>>> I resume four jobs, each were for a different number of agents (10,100,1000,10000) it made it easier for me to decide on the app time. Two of them have already finished i.e.: > >>>> > >>>> Progress: time: Mon, 18 Feb 2013 23:50:12 +0000 Active:4 Checking status:1 Finished in previous run:148098 Finished successfully:37897 > >>>> Progress: time: Mon, 18 Feb 2013 23:50:15 +0000 Active:2 Checking status:1 Finished in previous run:148098 Finished successfully:37899 > >>>> Final status: Mon, 18 Feb 2013 23:50:15 +0000 Finished in previous run:148098 Finished successfully:37902 > >>>> > >>>> and the only one that is showing any failure (50/110000), is the ten agents version which is so short I can understand why, but its still actively trying to run jobs and is actively finishing jobs, so that's good. > >>>> > >>>> Yay! > >>>> > >>>> > >>>> > >>>> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo Pesce wrote: > >>>> Good. Keep me posted, I would really like to solve your problems in running on Beagle this week, I wish that Swift would have been friendlier. > >>>> > >>>> On Feb 18, 2013, at 1:01 PM, Sam Seaver wrote: > >>>> > >>>>> I just resumed the jobs that I'd killed before the system went down, lets see how it does. I always did a mini-review of the data I've got and it seems to be working as expected. > >>>>> > >>>>> > >>>>> On Mon, Feb 18, 2013 at 12:28 PM, Lorenzo Pesce wrote: > >>>>> I have lost track a bit of what's up. I am happy to try and go over it with you when you are ready. > >>>>> > >>>>> Some of the problems of swift might have improved with a new version and the new system. > >>>>> > >>>>> > >>>>> On Feb 18, 2013, at 12:22 PM, Sam Seaver wrote: > >>>>> > >>>>>> They're not, I've not looked since Beagle came back up. Will do so later today. > >>>>>> S > >>>>>> > >>>>>> > >>>>>> On Mon, Feb 18, 2013 at 12:20 PM, Lorenzo Pesce wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Postdoctoral Fellow > >>>>>> Mathematics and Computer Science Division > >>>>>> Argonne National Laboratory > >>>>>> 9700 S. Cass Avenue > >>>>>> Argonne, IL 60439 > >>>>>> > >>>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>>>> samseaver at gmail.com > >>>>>> (773) 796-7144 > >>>>>> > >>>>>> "We shall not cease from exploration > >>>>>> And the end of all our exploring > >>>>>> Will be to arrive where we started > >>>>>> And know the place for the first time." > >>>>>> --T. S. Eliot > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Postdoctoral Fellow > >>>>> Mathematics and Computer Science Division > >>>>> Argonne National Laboratory > >>>>> 9700 S. Cass Avenue > >>>>> Argonne, IL 60439 > >>>>> > >>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>>> samseaver at gmail.com > >>>>> (773) 796-7144 > >>>>> > >>>>> "We shall not cease from exploration > >>>>> And the end of all our exploring > >>>>> Will be to arrive where we started > >>>>> And know the place for the first time." > >>>>> --T. S. Eliot > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Postdoctoral Fellow > >>>> Mathematics and Computer Science Division > >>>> Argonne National Laboratory > >>>> 9700 S. Cass Avenue > >>>> Argonne, IL 60439 > >>>> > >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>> samseaver at gmail.com > >>>> (773) 796-7144 > >>>> > >>>> "We shall not cease from exploration > >>>> And the end of all our exploring > >>>> Will be to arrive where we started > >>>> And know the place for the first time." > >>>> --T. S. Eliot > >>>> > >>>> > >>>> > >>>> -- > >>>> Postdoctoral Fellow > >>>> Mathematics and Computer Science Division > >>>> Argonne National Laboratory > >>>> 9700 S. Cass Avenue > >>>> Argonne, IL 60439 > >>>> > >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>> samseaver at gmail.com > >>>> (773) 796-7144 > >>>> > >>>> "We shall not cease from exploration > >>>> And the end of all our exploring > >>>> Will be to arrive where we started > >>>> And know the place for the first time." > >>>> --T. S. Eliot > >>> > >>> > >>> > >>> > >>> -- > >>> Postdoctoral Fellow > >>> Mathematics and Computer Science Division > >>> Argonne National Laboratory > >>> 9700 S. Cass Avenue > >>> Argonne, IL 60439 > >>> > >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>> samseaver at gmail.com > >>> (773) 796-7144 > >>> > >>> "We shall not cease from exploration > >>> And the end of all our exploring > >>> Will be to arrive where we started > >>> And know the place for the first time." > >>> --T. S. Eliot > >>> > >>> > >>> > >>> -- > >>> Postdoctoral Fellow > >>> Mathematics and Computer Science Division > >>> Argonne National Laboratory > >>> 9700 S. Cass Avenue > >>> Argonne, IL 60439 > >>> > >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>> samseaver at gmail.com > >>> (773) 796-7144 > >>> > >>> "We shall not cease from exploration > >>> And the end of all our exploring > >>> Will be to arrive where we started > >>> And know the place for the first time." > >>> --T. S. Eliot > >> > >> This is the content of the file where we have the first complaint from > >> swift (see attached): > >> > >> > >> > >> > >> CI-DEB000002 > >> > >> > >> >> key="providerAttributes">pbs.aprun;pbs.mpp;depth=24 > >> > >> > >> > >> > >> 24 > >> 172800 > >> 0:10:00 > >> 100 > >> 100 > >> > >> > >> 200 > >> 1 > >> 1 > >> > >> > >> 47.99 > >> 10000 > >> > >> > >> > >> > >> > >> /lustre/beagle/samseaver/GS/swift.workdir > >> > >> > >> > >> > >> Any ideas? > >> > >> Begin forwarded message: > >> > >>> From: Sam Seaver > >>> > >>> Date: February 19, 2013 1:16:28 PM CST > >>> > >>> To: Lorenzo Pesce > >>> > >>> Subject: Re: How are things going? > >>> > >>> > >>> I got this error. I suspect using the new SWIFT_HOME directory > >>> means that there's possibly a missing parameter someplace: > >>> > >>> > >>> should we resume a previous calculation? [y/N] y > >>> rlog files displayed in reverse time order > >>> should I use GS-20130203-0717-jgeppt98.0.rlog ?[y/n] > >>> y > >>> Using GS-20130203-0717-jgeppt98.0.rlog > >>> [Error] GS_sites.xml:1:9: cvc-elt.1: Cannot find the declaration of > >>> element 'config'. > >>> > >>> > >>> Execution failed: > >>> Failed to parse site catalog > >>> swift:siteCatalog @ scheduler.k, line: 31 > >>> Caused by: Invalid pool entry 'pbs': > >>> swift:siteCatalog @ scheduler.k, line: 31 > >>> Caused by: java.lang.IllegalArgumentException: Missing URL > >>> at > >>> org.griphyn.vdl.karajan.lib.SiteCatalog.execution(SiteCatalog.java:173) > >>> at > >>> org.griphyn.vdl.karajan.lib.SiteCatalog.pool(SiteCatalog.java:100) > >>> at > >>> org.griphyn.vdl.karajan.lib.SiteCatalog.buildResources(SiteCatalog.java:60) > >>> at > >>> org.griphyn.vdl.karajan.lib.SiteCatalog.function(SiteCatalog.java:48) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.functions.AbstractFunction.runBody(AbstractFunction.java:38) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:147) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.Import.runBody(Import.java:269) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:154) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:87) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:63) > >>> at > >>> org.globus.cog.karajan.compiled.nodes.Main.run(Main.java:79) > >>> at k.thr.LWThread.run(LWThread.java:243) > >>> at > >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > >>> at java.util.concurrent.ThreadPoolExecutor > >>> $Worker.run(ThreadPoolExecutor.java:603) > >>> at java.lang.Thread.run(Thread.java:722) > >>> > >>> > >>> On Tue, Feb 19, 2013 at 1:13 PM, Sam Seaver > >>> wrote: > >>> OK, it got to the point where it really did hang. I'm > >>> retrying, but with your suggestions. The other three > >>> finished fine! > >>> > >>> > >>> Progress: time: Tue, 19 Feb 2013 19:08:53 +0000 Selecting > >>> site:18147 Submitted:174 Active:96 Failed:2 Finished > >>> successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:09:23 +0000 Selecting > >>> site:18147 Submitted:174 Active:96 Failed:2 Finished > >>> successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:09:53 +0000 Selecting > >>> site:18147 Submitted:174 Active:96 Failed:2 Finished > >>> successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:10:23 +0000 Selecting > >>> site:18147 Submitted:174 Active:96 Failed:2 Finished > >>> successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:10:53 +0000 Selecting > >>> site:18147 Submitted:174 Active:96 Failed:2 Finished > >>> successfully:132323 Failed but can retry:183 > >>> Progress: time: Tue, 19 Feb 2013 19:11:23 +0000 Selecting > >>> site:18147 Submitted:174 Active:96 Failed:2 Finished > >>> successfully:132323 Failed but can retry:183 > >>> > >>> > >>> On Tue, Feb 19, 2013 at 8:51 AM, Lorenzo Pesce > >>> wrote: > >>> Hmm... > >>> > >>> > >>> foreach.max.threads=100 > >>> > >>> > >>> maybe you should increase this number a bit and see > >>> what happens. > >>> > >>> > >>> Also, I would try to replace > >>> > >>> > >>> SWIFT_HOME=/home/wilde/swift/rev/swift-r6151-cog-r3552 > >>> > >>> > >>> with > >>> > >>> > >>> SWIFT_HOME=/soft/swift/fast > >>> > >>> > >>> Keep me posted. Let's get this rolling. > >>> > >>> > >>> if it doesn't work, I can redo the packing. > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> On Feb 19, 2013, at 1:07 AM, Sam Seaver wrote: > >>> > >>>> Actually, the ten agents job does seem to be stuck > >>>> in a strange loop. It is incrementing the number > >>>> of jobs that has finished successfully, and at a > >>>> fast pace, but the number of jobs its starting is > >>>> decrementing much more slowly, its almost as its > >>>> repeatedly attempting the same set of parameters > >>>> multiple times... > >>>> > >>>> > >>>> I'll see what it's doing in the morning > >>>> S > >>>> > >>>> > >>>> On Tue, Feb 19, 2013 at 1:00 AM, Sam Seaver > >>>> wrote: > >>>> Seems to have worked overall this time! > >>>> > >>>> > >>>> I resume four jobs, each were for a > >>>> different number of agents > >>>> (10,100,1000,10000) it made it easier for > >>>> me to decide on the app time. Two of them > >>>> have already finished i.e.: > >>>> > >>>> > >>>> Progress: time: Mon, 18 Feb 2013 23:50:12 > >>>> +0000 Active:4 Checking status:1 > >>>> Finished in previous run:148098 Finished > >>>> successfully:37897 > >>>> Progress: time: Mon, 18 Feb 2013 23:50:15 > >>>> +0000 Active:2 Checking status:1 > >>>> Finished in previous run:148098 Finished > >>>> successfully:37899 > >>>> Final status: Mon, 18 Feb 2013 23:50:15 > >>>> +0000 Finished in previous run:148098 > >>>> Finished successfully:37902 > >>>> > >>>> > >>>> and the only one that is showing any > >>>> failure (50/110000), is the ten agents > >>>> version which is so short I can understand > >>>> why, but its still actively trying to run > >>>> jobs and is actively finishing jobs, so > >>>> that's good. > >>>> > >>>> > >>>> Yay! > >>>> > >>>> > >>>> > >>>> > >>>> On Mon, Feb 18, 2013 at 1:09 PM, Lorenzo > >>>> Pesce wrote: > >>>> Good. Keep me posted, I would > >>>> really like to solve your problems > >>>> in running on Beagle this week, I > >>>> wish that Swift would have been > >>>> friendlier. > >>>> > >>>> On Feb 18, 2013, at 1:01 PM, Sam > >>>> Seaver wrote: > >>>> > >>>>> I just resumed the jobs that I'd > >>>>> killed before the system went > >>>>> down, lets see how it does. I > >>>>> always did a mini-review of the > >>>>> data I've got and it seems to be > >>>>> working as expected. > >>>>> > >>>>> > >>>>> On Mon, Feb 18, 2013 at 12:28 > >>>>> PM, Lorenzo Pesce > >>>>> wrote: > >>>>> I have lost track a bit > >>>>> of what's up. I am happy > >>>>> to try and go over it > >>>>> with you when you are > >>>>> ready. > >>>>> > >>>>> > >>>>> Some of the problems of > >>>>> swift might have > >>>>> improved with a new > >>>>> version and the new > >>>>> system. > >>>>> > >>>>> > >>>>> > >>>>> On Feb 18, 2013, at > >>>>> 12:22 PM, Sam Seaver > >>>>> wrote: > >>>>> > >>>>>> They're not, I've not > >>>>>> looked since Beagle > >>>>>> came back up. Will do > >>>>>> so later today. > >>>>>> S > >>>>>> > >>>>>> > >>>>>> On Mon, Feb 18, 2013 > >>>>>> at 12:20 PM, Lorenzo > >>>>>> Pesce > >>>>>> > >>>>>> wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Postdoctoral Fellow > >>>>>> Mathematics and > >>>>>> Computer Science > >>>>>> Division > >>>>>> Argonne National > >>>>>> Laboratory > >>>>>> 9700 S. Cass Avenue > >>>>>> Argonne, IL 60439 > >>>>>> > >>>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>>>> samseaver at gmail.com > >>>>>> (773) 796-7144 > >>>>>> > >>>>>> "We shall not cease > >>>>>> from exploration > >>>>>> And the end of all our > >>>>>> exploring > >>>>>> Will be to arrive > >>>>>> where we started > >>>>>> And know the place for > >>>>>> the first time." > >>>>>> --T. S. Eliot > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Postdoctoral Fellow > >>>>> Mathematics and Computer Science > >>>>> Division > >>>>> Argonne National Laboratory > >>>>> 9700 S. Cass Avenue > >>>>> Argonne, IL 60439 > >>>>> > >>>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>>> samseaver at gmail.com > >>>>> (773) 796-7144 > >>>>> > >>>>> "We shall not cease from > >>>>> exploration > >>>>> And the end of all our exploring > >>>>> Will be to arrive where we > >>>>> started > >>>>> And know the place for the first > >>>>> time." > >>>>> --T. S. Eliot > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Postdoctoral Fellow > >>>> Mathematics and Computer Science Division > >>>> Argonne National Laboratory > >>>> 9700 S. Cass Avenue > >>>> Argonne, IL 60439 > >>>> > >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>> samseaver at gmail.com > >>>> (773) 796-7144 > >>>> > >>>> "We shall not cease from exploration > >>>> And the end of all our exploring > >>>> Will be to arrive where we started > >>>> And know the place for the first time." > >>>> --T. S. Eliot > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Postdoctoral Fellow > >>>> Mathematics and Computer Science Division > >>>> Argonne National Laboratory > >>>> 9700 S. Cass Avenue > >>>> Argonne, IL 60439 > >>>> > >>>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>>> samseaver at gmail.com > >>>> (773) 796-7144 > >>>> > >>>> "We shall not cease from exploration > >>>> And the end of all our exploring > >>>> Will be to arrive where we started > >>>> And know the place for the first time." > >>>> --T. S. Eliot > >>> > >>> > >>> > >>> > >>> > >>> > >>> -- > >>> Postdoctoral Fellow > >>> Mathematics and Computer Science Division > >>> Argonne National Laboratory > >>> 9700 S. Cass Avenue > >>> Argonne, IL 60439 > >>> > >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>> samseaver at gmail.com > >>> (773) 796-7144 > >>> > >>> "We shall not cease from exploration > >>> And the end of all our exploring > >>> Will be to arrive where we started > >>> And know the place for the first time." > >>> --T. S. Eliot > >>> > >>> > >>> > >>> > >>> -- > >>> Postdoctoral Fellow > >>> Mathematics and Computer Science Division > >>> Argonne National Laboratory > >>> 9700 S. Cass Avenue > >>> Argonne, IL 60439 > >>> > >>> http://www.linkedin.com/pub/sam-seaver/0/412/168 > >>> samseaver at gmail.com > >>> (773) 796-7144 > >>> > >>> "We shall not cease from exploration > >>> And the end of all our exploring > >>> Will be to arrive where we started > >>> And know the place for the first time." > >>> --T. S. Eliot > >> > >> > >> _______________________________________________ > >> Swift-devel mailing list > >> Swift-devel at ci.uchicago.edu > >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > From wilde at mcs.anl.gov Wed Feb 20 16:12:13 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Wed, 20 Feb 2013 16:12:13 -0600 (CST) Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1361393712.8356.1.camel@echo> Message-ID: <982631837.1678030.1361398333709.JavaMail.root@mcs.anl.gov> > Also, the job manager attribute should be "jobManager", not > jobmanager. I though in the past sites attributes were not case sensitive. Is this a change with the faster branch, or am I mistaken about prior behavior? From hategan at mcs.anl.gov Wed Feb 20 16:29:33 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 20 Feb 2013 14:29:33 -0800 Subject: [Swift-devel] First tests with swift faster In-Reply-To: <982631837.1678030.1361398333709.JavaMail.root@mcs.anl.gov> References: <982631837.1678030.1361398333709.JavaMail.root@mcs.anl.gov> Message-ID: <1361399373.1351.5.camel@echo> On Wed, 2013-02-20 at 16:12 -0600, Michael Wilde wrote: > > Also, the job manager attribute should be "jobManager", not > > jobmanager. > > I though in the past sites attributes were not case sensitive. > > Is this a change with the faster branch, or am I mistaken about prior behavior? That's correct. Preciously the sites file was a karajan file. That was case-insensitive. In the faster branch the sites file is parsed with a DOM parser. This will allow proper validation (which I'm working on right now). There will be two versions of the sites file. The default (2.0), will be case-sensitive and will allow app definitions in the sites file. The backwards compatible one (), will be case insensitive. But you will need to explicitly say version="1.0" in there to get that. This is tentative, so open to discussion. Mihael From wilde at mcs.anl.gov Wed Feb 20 22:19:35 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Wed, 20 Feb 2013 22:19:35 -0600 (CST) Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1361399373.1351.5.camel@echo> Message-ID: <759097444.1741494.1361420375550.JavaMail.root@mcs.anl.gov> > In the faster branch the sites file is parsed with a DOM parser. This > will allow proper validation (which I'm working on right now). > > There will be two versions of the sites file. The default (2.0), will > be > case-sensitive and will allow app definitions in the sites file. > > The backwards compatible one (), will be case > insensitive. But you will need to explicitly say version="1.0" in > there > to get that. Could we change the tag to eg or for 2.0 and leave the 1.0 parsing unchanged for true backwards compatibility? > This is tentative, so open to discussion. I think better parsing (specifically to get better error messages) is a good idea. Especially to catch mis-spelled tags (or values) that are now ignored, if thats possible. I want to get to the point where most users never type XML to use Swift, and the way to that is gensites, which should be coorinated with this change. If we're to use Coasters and the providers with Swift/T we should all be thinking about how we want this to work for users. - Mike From hategan at mcs.anl.gov Wed Feb 20 22:27:56 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Wed, 20 Feb 2013 20:27:56 -0800 Subject: [Swift-devel] First tests with swift faster In-Reply-To: <759097444.1741494.1361420375550.JavaMail.root@mcs.anl.gov> References: <759097444.1741494.1361420375550.JavaMail.root@mcs.anl.gov> Message-ID: <1361420876.12007.3.camel@echo> On Wed, 2013-02-20 at 22:19 -0600, Michael Wilde wrote: > > In the faster branch the sites file is parsed with a DOM parser. This > > will allow proper validation (which I'm working on right now). > > > > There will be two versions of the sites file. The default (2.0), will > > be > > case-sensitive and will allow app definitions in the sites file. > > > > The backwards compatible one (), will be case > > insensitive. But you will need to explicitly say version="1.0" in > > there > > to get that. > > Could we change the tag to eg or for 2.0 and leave the 1.0 parsing unchanged for true backwards compatibility? I see. So decide the version based on the root node. Makes good sense. > > > This is tentative, so open to discussion. > > I think better parsing (specifically to get better error messages) is a good idea. Especially to catch mis-spelled tags (or values) that are now ignored, if thats possible. Yes. Things will be validated against a schema (if xmlns is specified). Also the parser now complains about unrecognized tags. > > I want to get to the point where most users never type XML to use Swift, and the way to that is gensites, which should be coorinated with this change. This is an opportunity to deal with the issue. Perhaps a better configuration format might be the better choice. > > If we're to use Coasters and the providers with Swift/T we should all be thinking about how we want this to work for users. From wilde at mcs.anl.gov Wed Feb 20 22:44:05 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Wed, 20 Feb 2013 22:44:05 -0600 (CST) Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1361420876.12007.3.camel@echo> Message-ID: <1554981078.1742051.1361421845525.JavaMail.root@mcs.anl.gov> > > I want to get to the point where most users never type XML to use > > Swift, and the way to that is gensites, which should be coorinated > > with this change. > > This is an opportunity to deal with the issue. Perhaps a better > configuration format might be the better choice. Should we consider a simpler name/value format and eliminate XML here? A site could pretty much be defined by a single level set of name=value pairs. Basically make the input to gensites the actual format instead of requiring another layer? I should also mention that what goes on *inside* the Java code is similarly complex. Understanding how attributes go from sites.xml and swift.properties through the various config layers, especially with Coasters involved, has been a barrier to many developers, debugging efforts, and improvements. Im not sure the two are that closely related, but while we're working in the neighborhood, might want to consider some improvement (and documentation) as well. I think Justin took a first crack at documenting the config conventions, but Im not sure where that went. From wilde at mcs.anl.gov Wed Feb 20 22:48:34 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Wed, 20 Feb 2013 22:48:34 -0600 (CST) Subject: [Swift-devel] wrapper staging In-Reply-To: <960437189.550084.1360870504295.JavaMail.root@mcs.anl.gov> Message-ID: <2081308643.1746635.1361422114567.JavaMail.root@mcs.anl.gov> I'd like to move this item higher up on our priority list, hopefully after /faster gets to a stable resting point. This is another area where we should consider the needs of Swift/T and Swift/K together. - Mike ----- Original Message ----- > From: "Michael Wilde" > To: "Swift Devel" > Sent: Thursday, February 14, 2013 1:35:04 PM > Subject: Re: [Swift-devel] wrapper staging > > Hi All, > > I'd like to resume the discussion of this feature, in particular how > it might offer an effective way to a very user-friendly direct IO > mechanism. Which experience is indicating that we urgently need. > > We need to do a case analysis, including: > > - workdir/jobdir on shared fs > - jobdir on local fs > - job dir below swift cwd (something we dont do but makes sense to > consider) > > How input is handled > How output is handled > Who creates necessary dirs > > Also need to consider how various kinds of pathnames and URIs are > treated: > > - /fully/qualified > - relative > - relative/structured > - various URIs: gsiftp:// etc > > Mike > > ----- Original Message ----- > > From: "Mihael Hategan" > > To: "Swift Devel" > > Sent: Saturday, May 26, 2012 6:23:32 PM > > Subject: [Swift-devel] wrapper staging > > > > There has been some discussion in the past where we agreed that it > > might > > be a good idea to try a staging mechanism in which the wrapper (or > > some > > other entity on the worker node) does the staging. > > > > I added a skeleton for that in trunk. The relevant files are > > vdl-int-wrapper-staging.k > > and _swiftwrap.wrapperstaging. > > > > There is a basic implementation that recognizes "file://" URLs and > > does > > a cp for stage-ins and outs. > > > > Mihael > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > From hategan at mcs.anl.gov Thu Feb 21 03:45:32 2013 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 21 Feb 2013 01:45:32 -0800 Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1554981078.1742051.1361421845525.JavaMail.root@mcs.anl.gov> References: <1554981078.1742051.1361421845525.JavaMail.root@mcs.anl.gov> Message-ID: <1361439932.12656.6.camel@echo> On Wed, 2013-02-20 at 22:44 -0600, Michael Wilde wrote: > > > I want to get to the point where most users never type XML to use > > > Swift, and the way to that is gensites, which should be coorinated > > > with this change. > > > > This is an opportunity to deal with the issue. Perhaps a better > > configuration format might be the better choice. > > Should we consider a simpler name/value format and eliminate XML here? A site could pretty much be defined by a single level set of name=value pairs. Possibly. We do have to describe hierarchical information (site->(properties->sub-properties, apps->properties)), so the questions is whether name=value pairs will do the trick for that. > > Basically make the input to gensites the actual format instead of requiring another layer? Iff the input to gensites can fully describe the site "space", then yes. If not, then probably not. > > I should also mention that what goes on *inside* the Java code is > similarly complex. Understanding how attributes go from sites.xml and > swift.properties through the various config layers, especially with > Coasters involved, has been a barrier to many developers, debugging > efforts, and improvements. Im not sure the two are that closely > related, but while we're working in the neighborhood, might want to > consider some improvement (and documentation) as well. I think Justin > took a first crack at documenting the config conventions, but Im not > sure where that went. I think we're trying to abstract things away. In the same spirit that you don't really think about how CPU registers are assigned when you write some C program, you also shouldn't think about how exactly coasters are starring jobs. Mihael From wilde at mcs.anl.gov Thu Feb 21 08:27:22 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 21 Feb 2013 08:27:22 -0600 (CST) Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1361439932.12656.6.camel@echo> Message-ID: <659703998.1788860.1361456842272.JavaMail.root@mcs.anl.gov> > > Should we consider a simpler name/value format and eliminate XML > > here? A site could pretty much be defined by a single level set of > > name=value pairs. > > Possibly. We do have to describe hierarchical information > (site->(properties->sub-properties, apps->properties)), so the > questions > is whether name=value pairs will do the trick for that. gensites substitutes named values into template sites.xml files. We need to refine how the templates are located, and where site defaults are set (ie, whether there is a hierarchy of settings as well as a hierarchical structure within settings. The sites.xml approach we have today is OK, as long as the user typically needs to only specify a few values into the settings. Whats mostly broken today is attribute names that have gotten messy and unclear over time and add noise to the user's understanding (eg: globus profile vs karajan profile; maxtime vs maxwalltime). So we can stay with sites.xml and gensites, but cleaning up the parsing may be a good time to clean up the attribute names. > > Basically make the input to gensites the actual format instead of > > requiring another layer? > > Iff the input to gensites can fully describe the site "space", then > yes. > If not, then probably not. I suggest we revisit gensites as a group. Till now David has been using it but its not been socialized among the group nor properly exposed to users yet. Also, Lorenzo has a nice shell mechanism which ascii-prompts the user for run parameters; we might want to try to generalize it. This all ties back to the "swiftrun" command, and gensites and swiftrun and any changes to the config mechanism need to be designed together. > > > > I should also mention that what goes on *inside* the Java code is > > similarly complex. Understanding how attributes go from sites.xml > > and > > swift.properties through the various config layers, especially with > > Coasters involved, has been a barrier to many developers, debugging > > efforts, and improvements. Im not sure the two are that closely > > related, but while we're working in the neighborhood, might want to > > consider some improvement (and documentation) as well. I think > > Justin > > took a first crack at documenting the config conventions, but Im > > not > > sure where that went. > > I think we're trying to abstract things away. In the same spirit that > you don't really think about how CPU registers are assigned when you > write some C program, you also shouldn't think about how exactly > coasters are starring jobs. Im not sure I understand. Im not questioning abstraction. Im pointing out that the manner in which parameters make it from outside the code to inside the code has been a blocker to developers. Perhaps the current abstractions are OK, perhaps they could be improved. But they have proven difficulty to understand and use. Often just getting to the right sites parameter values in the Java code has been more difficult than dealing with the given algorithmic problem the developer was trying to solve (e.g., adding new parameters for new sites like Crays, SGE, and SLURM). Maybe that just needs documentation, or maybe it needs cleanup. But if, for example, we change to JSON for the external sites parameter spec, that might be reflected in a simpler and more clear abstraction for passing parameters inside the code. Similarly XMS and DOM, perhaps. - Mike From lpesce at uchicago.edu Thu Feb 21 08:49:00 2013 From: lpesce at uchicago.edu (Lorenzo Pesce) Date: Thu, 21 Feb 2013 08:49:00 -0600 Subject: [Swift-devel] First tests with swift faster In-Reply-To: <1554981078.1742051.1361421845525.JavaMail.root@mcs.anl.gov> References: <1554981078.1742051.1361421845525.JavaMail.root@mcs.anl.gov> Message-ID: <8889616A-CFC0-4317-8D97-188417C95163@uchicago.edu> On Feb 20, 2013, at 10:44 PM, Michael Wilde wrote: >>> I want to get to the point where most users never type XML to use >>> Swift, and the way to that is gensites, which should be coorinated >>> with this change. >> >> This is an opportunity to deal with the issue. Perhaps a better >> configuration format might be the better choice. > > Should we consider a simpler name/value format and eliminate XML here? A site could pretty much be defined by a single level set of name=value pairs. I give you my 2 cents as a "heavy" user. I like XML and I like it to be there. I never write one line of it because I generate the XML files using scripts. I plan to make these scripts available to the users on Beagle and all users if you find that it could help. But, I still think that XML is good. Sometimes I find names confusing (typically the *time* kinds), but again I rarely have to deal with them directly nor do my users. > Basically make the input to gensites the actual format instead of requiring another layer? I will have to look this one up to understand it better. > I should also mention that what goes on *inside* the Java code is similarly complex. Understanding how attributes go from sites.xml and swift.properties through the various config layers, especially with Coasters involved, has been a barrier to many developers, debugging efforts, and improvements. Im not sure the two are that closely related, but while we're working in the neighborhood, might want to consider some improvement (and documentation) as well. I think Justin took a first crack at documenting the config conventions, but Im not sure where that went. From davidk at ci.uchicago.edu Sat Feb 23 14:15:10 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Sat, 23 Feb 2013 14:15:10 -0600 (CST) Subject: [Swift-devel] Reading environment variables In-Reply-To: <835738438.6294875.1361649890579.JavaMail.root@ci.uchicago.edu> Message-ID: <950999395.6298238.1361650510763.JavaMail.root@ci.uchicago.edu> Is there a function available for reading environment variables within a swift script? I didn't see one mentioned in the userguide, but this seems to work for me: string home=@java("java.lang.System", "getenv", "HOME"); Should we have a function for this? @env("HOME")? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.g.armstrong at gmail.com Sat Feb 23 15:10:39 2013 From: tim.g.armstrong at gmail.com (Tim Armstrong) Date: Sat, 23 Feb 2013 15:10:39 -0600 Subject: [Swift-devel] Reading environment variables In-Reply-To: <950999395.6298238.1361650510763.JavaMail.root@ci.uchicago.edu> References: <835738438.6294875.1361649890579.JavaMail.root@ci.uchicago.edu> <950999395.6298238.1361650510763.JavaMail.root@ci.uchicago.edu> Message-ID: FWIW we have this function in Swift/T (string s) getenv(string key) On Sat, Feb 23, 2013 at 2:15 PM, David Kelly wrote: > Is there a function available for reading environment variables within a > swift script? I didn't see one mentioned in the userguide, but this seems > to work for me: > > string home=@java("java.lang.System", "getenv", "HOME"); > > Should we have a function for this? @env("HOME")? > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Thu Feb 28 14:39:19 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 28 Feb 2013 14:39:19 -0600 (CST) Subject: [Swift-devel] Workflow generator - possible ideas for testing In-Reply-To: <670263587.1157053.1358960088010.JavaMail.root@mcs.anl.gov> Message-ID: <335334259.1081028.1362083959355.JavaMail.root@mcs.anl.gov> Yadu, the workflow-pattern generator I mentioned was never published; a prelim report on it is attached. - Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: LMeyer.WorkflowGenReport.pdf Type: application/pdf Size: 263949 bytes Desc: not available URL: From wilde at mcs.anl.gov Thu Feb 28 14:52:11 2013 From: wilde at mcs.anl.gov (Michael Wilde) Date: Thu, 28 Feb 2013 14:52:11 -0600 (CST) Subject: [Swift-devel] Moving from Java 1.6 to 1.7 In-Reply-To: <23F50D9D-4C03-465F-8CFE-A613DDC2A8E8@slac.stanford.edu> Message-ID: <2146582586.1083203.1362084731406.JavaMail.root@mcs.anl.gov> I just saw this note on an OSG list, and thought we may want to consider this transition for Swift testing, and for package/module and other setup issues on the various sites where we more closely support Swift. - Mike ----- Forwarded Message ----- From: "Wei Yang" To: usatlas-t2-l at lists.bnl.gov Cc: osg-int at OPENSCIENCEGRID.ORG Sent: Thursday, February 28, 2013 2:19:11 PM Subject: java 1.6 and OSG software at SLAC As many of you aware, Oracle will no longer support Java 1.6. As a result, SLAC required all applications to move away from java 1.6 as quickly as possible. At SLAC, so far we are able to move to Java 1.7 (x86_64) with the following OSG software: RPM based bestman2; replace JAVA_HOME in/etc/sysconfig/bestman2. note: bestman2 expects JAVA_HOME/bin/java Pacman based osg 1.2.29/vdt-2.0.0p36 CE/GIP Pacman based osg 1.2.28/vdt-2.0.0p35 GUMS 1.3.17 make $VDT_LOCATION/jdk1.6 a symlink to Java 1.7. note tomcat expects $VDT_LOCATION/jdk1.6/bin/java regards, Wei Yang | yangw at slac.stanford.edu | 650-926-3338(O) From davidk at ci.uchicago.edu Thu Feb 28 15:08:51 2013 From: davidk at ci.uchicago.edu (David Kelly) Date: Thu, 28 Feb 2013 15:08:51 -0600 (CST) Subject: [Swift-devel] Weekly training sessions In-Reply-To: <1568462793.1945423.1362083469515.JavaMail.root@ci.uchicago.edu> Message-ID: <1455849172.1964578.1362085731588.JavaMail.root@ci.uchicago.edu> Hello, I was thinking it might be a good time to bring back the weekly developer/training sessions we used to have. I always found them very useful. I think Justin has a list of future topics somewhere (which I can't seem to find at the moment), and I'm sure there are a lot of new things to discuss as well. Some topics that come to mind: Swift-T tutorial An overview of coaster configurations (automatic/passive/persistent/etc) MPI Modis Understanding mappers Methods for approaching file I/O Running swift on EC2 Running swift on OSG Maybe we could alternate who gives the training each week so that no one developer has to spend too much time on it. Any interest? David Michael Kelly Systems Programmer University of Chicago Computation Institute 5735 S. Ellis Ave. Chicago, IL 60637 -------------- next part -------------- An HTML attachment was scrubbed... URL: From skrieder at iit.edu Thu Feb 28 16:44:34 2013 From: skrieder at iit.edu (Scott Krieder) Date: Thu, 28 Feb 2013 16:44:34 -0600 Subject: [Swift-devel] Weekly training sessions In-Reply-To: <1455849172.1964578.1362085731588.JavaMail.root@ci.uchicago.edu> References: <1568462793.1945423.1362083469515.JavaMail.root@ci.uchicago.edu> <1455849172.1964578.1362085731588.JavaMail.root@ci.uchicago.edu> Message-ID: As an outsider (if I get a vote) I would definitely be interested in attending. -Scott On Thu, Feb 28, 2013 at 3:08 PM, David Kelly wrote: > Hello, > > I was thinking it might be a good time to bring back the weekly > developer/training sessions we used to have. I always found them very > useful. I think Justin has a list of future topics somewhere (which I can't > seem to find at the moment), and I'm sure there are a lot of new things to > discuss as well. Some topics that come to mind: > > Swift-T tutorial > An overview of coaster configurations (automatic/passive/persistent/etc) > MPI > Modis > Understanding mappers > Methods for approaching file I/O > Running swift on EC2 > Running swift on OSG > > Maybe we could alternate who gives the training each week so that no one > developer has to spend too much time on it. > > Any interest? > > > > * David Michael Kelly* > Systems Programmer > University of Chicago Computation Institute > 5735 S. Ellis Ave. Chicago, IL 60637 > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -- Scott J. Krieder C: 419-685-0410 E: skrieder at iit.edu http://datasys.cs.iit.edu/~skrieder/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hockyg at gmail.com Thu Feb 28 16:46:40 2013 From: hockyg at gmail.com (Glen Hocky) Date: Thu, 28 Feb 2013 17:46:40 -0500 Subject: [Swift-devel] Weekly training sessions In-Reply-To: References: <1568462793.1945423.1362083469515.JavaMail.root@ci.uchicago.edu> <1455849172.1964578.1362085731588.JavaMail.root@ci.uchicago.edu> Message-ID: As a remote outsider, I'd be interested in watching a video (live or recorded, but i guess pref both?) -Glen On Thu, Feb 28, 2013 at 5:44 PM, Scott Krieder wrote: > As an outsider (if I get a vote) I would definitely be interested in > attending. > > -Scott > > On Thu, Feb 28, 2013 at 3:08 PM, David Kelly wrote: > >> Hello, >> >> I was thinking it might be a good time to bring back the weekly >> developer/training sessions we used to have. I always found them very >> useful. I think Justin has a list of future topics somewhere (which I can't >> seem to find at the moment), and I'm sure there are a lot of new things to >> discuss as well. Some topics that come to mind: >> >> Swift-T tutorial >> An overview of coaster configurations (automatic/passive/persistent/etc) >> MPI >> Modis >> Understanding mappers >> Methods for approaching file I/O >> Running swift on EC2 >> Running swift on OSG >> >> Maybe we could alternate who gives the training each week so that no one >> developer has to spend too much time on it. >> >> Any interest? >> >> >> >> * David Michael Kelly* >> Systems Programmer >> University of Chicago Computation Institute >> 5735 S. Ellis Ave. Chicago, IL 60637 >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> >> > > > -- > Scott J. Krieder > C: 419-685-0410 > E: skrieder at iit.edu > http://datasys.cs.iit.edu/~skrieder/ > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.g.armstrong at gmail.com Thu Feb 28 17:19:20 2013 From: tim.g.armstrong at gmail.com (Tim Armstrong) Date: Thu, 28 Feb 2013 17:19:20 -0600 Subject: [Swift-devel] Weekly training sessions In-Reply-To: References: <1568462793.1945423.1362083469515.JavaMail.root@ci.uchicago.edu> <1455849172.1964578.1362085731588.JavaMail.root@ci.uchicago.edu> Message-ID: I could probably put together a run-through of Swift/T at some point - a high level overview of the compiler/runtime stack, talk about some of the current limitations plus some of the language features we've been playing with. It would be good to communicate more with you guys so we can get each other more up to speed on the state of play, and maybe think through future directions for development. - Tim On Thu, Feb 28, 2013 at 4:46 PM, Glen Hocky wrote: > As a remote outsider, I'd be interested in watching a video (live or > recorded, but i guess pref both?) > -Glen > > > On Thu, Feb 28, 2013 at 5:44 PM, Scott Krieder wrote: > >> As an outsider (if I get a vote) I would definitely be interested in >> attending. >> >> -Scott >> >> On Thu, Feb 28, 2013 at 3:08 PM, David Kelly wrote: >> >>> Hello, >>> >>> I was thinking it might be a good time to bring back the weekly >>> developer/training sessions we used to have. I always found them very >>> useful. I think Justin has a list of future topics somewhere (which I can't >>> seem to find at the moment), and I'm sure there are a lot of new things to >>> discuss as well. Some topics that come to mind: >>> >>> Swift-T tutorial >>> An overview of coaster configurations (automatic/passive/persistent/etc) >>> MPI >>> Modis >>> Understanding mappers >>> Methods for approaching file I/O >>> Running swift on EC2 >>> Running swift on OSG >>> >>> Maybe we could alternate who gives the training each week so that no one >>> developer has to spend too much time on it. >>> >>> Any interest? >>> >>> >>> >>> * David Michael Kelly* >>> Systems Programmer >>> University of Chicago Computation Institute >>> 5735 S. Ellis Ave. Chicago, IL 60637 >>> >>> >>> _______________________________________________ >>> Swift-devel mailing list >>> Swift-devel at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>> >>> >> >> >> -- >> Scott J. Krieder >> C: 419-685-0410 >> E: skrieder at iit.edu >> http://datasys.cs.iit.edu/~skrieder/ >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> >> > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: