From marialemos72 at gmail.com Sat Jan 11 12:31:03 2014 From: marialemos72 at gmail.com (ML) Date: Sat, 11 Jan 2014 18:31:03 +0000 Subject: [Swift-user] CISTI'2014: List of Workshops Message-ID: <20140111183108.298827CC093@mailrelay.anl.gov> ********************************** WORKSHOPS ******************************************* CISTI'2014 - 9th Iberian Conference on Information Systems and Technologies Barcelona, Spain, June 18 - 21, 2014 http://www.aisti.eu/cisti2014/index.php/en/workshops **************************************************************************************** List of Workshops to be held in the CISTI'2014 context: - ARWC 2014 - 1st Workshop on Augmented Reality and Wearable Computing - ASDACS 2014 - 1st Workshop on Applied Statistics and Data Analysis using Computer Science - IoT 2014 - 1st Workshop on Internet of Things - SGaMePlay 2014 - 4th Iberian Workshop on Serious Games and Meaningful Play - TICAMES 2014 - 2nd Workshop on Information and Communication Technology in Higher Education: Learning Mathematics - WICTA 2014 - 1st Workshop on ICT for Audit - WISA 2014 - 6th Workshop on Intelligent Systems and Apllications - WLA 2014 - 1st Workshop on Learning Analytics - WNIS 2014 - 1st Workshop on Networks, Information and Society Detailed information about these workshops is available at http://www.aisti.eu/cisti2014/index.php/en/workshops Best regards, CISTI'2014 Team http://www.aisti.eu/cisti2014/index.php/en From ketan at mcs.anl.gov Mon Jan 13 11:01:39 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Mon, 13 Jan 2014 11:01:39 -0600 Subject: [Swift-user] readData is treating line as a single field Message-ID: Hi, I am trying to use readData() function in Swift to read fields into a Swift structure but seems it is reading the whole line as a single field. Here is my text file for the data: $ cat data.txt Survey_data Strategy_data Field_data quadui_outdir /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Survey_data_import_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Seasonal_strategy_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Field_Overlay_fixed.zip /home/kcm92/outdir_1 /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Survey_data_import_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Seasonal_strategy_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Field_Overlay_fixed.zip /home/kcm92/outdir_2 I have 4 fields and 2 rows. Here is the structure definition: type Data{ string Survey_data; string Field_data; string Strategy_data; string quadui_outdir; } Here is the Swift loop that I am using to process this data: Data d[] = readData(@arg("data")); /* run quadui */ foreach ditem in d{ tracef("Survey data is: %s", ditem.Survey_data); } In the output, I get the whole first row of the file instead of the expected first field. Any clues? Thanks, Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at mcs.anl.gov Mon Jan 13 11:05:32 2014 From: wilde at mcs.anl.gov (Wilde, Michael J.) Date: Mon, 13 Jan 2014 17:05:32 +0000 Subject: [Swift-user] readData is treating line as a single field In-Reply-To: References: Message-ID: <85C85E44DD880E498CEA5A501B27954BE9DAB5@DITKA.anl.gov> Did you try different field separators? I think in the past Ive used spaces. Perhaps the readData() code is not sufficiently white-space-aware yet? - Mike -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago ________________________________ From: swift-user-bounces at ci.uchicago.edu [swift-user-bounces at ci.uchicago.edu] on behalf of Ketan Maheshwari [ketan at mcs.anl.gov] Sent: Monday, January 13, 2014 11:01 AM To: Swift User Subject: [Swift-user] readData is treating line as a single field Hi, I am trying to use readData() function in Swift to read fields into a Swift structure but seems it is reading the whole line as a single field. Here is my text file for the data: $ cat data.txt Survey_data Strategy_data Field_data quadui_outdir /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Survey_data_import_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Seasonal_strategy_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Field_Overlay_fixed.zip /home/kcm92/outdir_1 /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Survey_data_import_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Seasonal_strategy_fixed.zip /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Field_Overlay_fixed.zip /home/kcm92/outdir_2 I have 4 fields and 2 rows. Here is the structure definition: type Data{ string Survey_data; string Field_data; string Strategy_data; string quadui_outdir; } Here is the Swift loop that I am using to process this data: Data d[] = readData(@arg("data")); /* run quadui */ foreach ditem in d{ tracef("Survey data is: %s", ditem.Survey_data); } In the output, I get the whole first row of the file instead of the expected first field. Any clues? Thanks, Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketan at mcs.anl.gov Mon Jan 13 11:46:40 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Mon, 13 Jan 2014 11:46:40 -0600 Subject: [Swift-user] readData is treating line as a single field In-Reply-To: <44953e222acc429b8895c7d5c132b4fe@NAGURSKI.anl.gov> References: <44953e222acc429b8895c7d5c132b4fe@NAGURSKI.anl.gov> Message-ID: Hi Mike, I am not sure how to introduce a field separator to readData(). I tried adding '|' between header and data fields but seems readData is complaining that file header does not match type and it expected space separated items: File header does not match type. Expected 4 whitespace separated items. Got 5 instead. swiftscript:readData, quadui-multi.swift, line 27 On Mon, Jan 13, 2014 at 11:05 AM, Wilde, Michael J. wrote: > Did you try different field separators? I think in the past Ive used > spaces. Perhaps the readData() code is not sufficiently white-space-aware > yet? > > - Mike > > > -- > Michael Wilde > Mathematics and Computer Science Computation Institute > Argonne National Laboratory The University of Chicago > > ------------------------------ > *From:* swift-user-bounces at ci.uchicago.edu [ > swift-user-bounces at ci.uchicago.edu] on behalf of Ketan Maheshwari [ > ketan at mcs.anl.gov] > *Sent:* Monday, January 13, 2014 11:01 AM > *To:* Swift User > *Subject:* [Swift-user] readData is treating line as a single field > > Hi, > > I am trying to use readData() function in Swift to read fields into a > Swift structure but seems it is reading the whole line as a single field. > > Here is my text file for the data: > > $ cat data.txt > Survey_data Strategy_data Field_data quadui_outdir > /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Survey_data_import_fixed.zip > /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Seasonal_strategy_fixed.zip > /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Field_Overlay_fixed.zip > /home/kcm92/outdir_1 > /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Survey_data_import_fixed.zip > /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Seasonal_strategy_fixed.zip > /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Field_Overlay_fixed.zip > /home/kcm92/outdir_2 > > I have 4 fields and 2 rows. > > Here is the structure definition: > > type Data{ > string Survey_data; > string Field_data; > string Strategy_data; > string quadui_outdir; > } > > Here is the Swift loop that I am using to process this data: > > Data d[] = readData(@arg("data")); > /* run quadui */ > foreach ditem in d{ > tracef("Survey data is: %s", ditem.Survey_data); > } > > In the output, I get the whole first row of the file instead of the > expected first field. > > Any clues? > > Thanks, > Ketan > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadudoc1729 at gmail.com Mon Jan 13 13:48:43 2014 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Mon, 13 Jan 2014 13:48:43 -0600 Subject: [Swift-user] readData is treating line as a single field In-Reply-To: References: <44953e222acc429b8895c7d5c132b4fe@NAGURSKI.anl.gov> Message-ID: Hi Ketan, Could you give me a sample file and code you're using to test this out ? -Yadu On Mon, Jan 13, 2014 at 11:46 AM, Ketan Maheshwari wrote: > Hi Mike, > > I am not sure how to introduce a field separator to readData(). I tried > adding '|' between header and data fields but seems readData is complaining > that file header does not match type and it expected space separated items: > > File header does not match type. Expected 4 whitespace separated items. > Got 5 instead. > swiftscript:readData, quadui-multi.swift, line 27 > > > > On Mon, Jan 13, 2014 at 11:05 AM, Wilde, Michael J. wrote: > >> Did you try different field separators? I think in the past Ive used >> spaces. Perhaps the readData() code is not sufficiently white-space-aware >> yet? >> >> - Mike >> >> >> -- >> Michael Wilde >> Mathematics and Computer Science Computation Institute >> Argonne National Laboratory The University of Chicago >> >> ------------------------------ >> *From:* swift-user-bounces at ci.uchicago.edu [ >> swift-user-bounces at ci.uchicago.edu] on behalf of Ketan Maheshwari [ >> ketan at mcs.anl.gov] >> *Sent:* Monday, January 13, 2014 11:01 AM >> *To:* Swift User >> *Subject:* [Swift-user] readData is treating line as a single field >> >> Hi, >> >> I am trying to use readData() function in Swift to read fields into a >> Swift structure but seems it is reading the whole line as a single field. >> >> Here is my text file for the data: >> >> $ cat data.txt >> Survey_data Strategy_data Field_data quadui_outdir >> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Survey_data_import_fixed.zip >> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Seasonal_strategy_fixed.zip >> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Field_Overlay_fixed.zip >> /home/kcm92/outdir_1 >> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Survey_data_import_fixed.zip >> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Seasonal_strategy_fixed.zip >> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Field_Overlay_fixed.zip >> /home/kcm92/outdir_2 >> >> I have 4 fields and 2 rows. >> >> Here is the structure definition: >> >> type Data{ >> string Survey_data; >> string Field_data; >> string Strategy_data; >> string quadui_outdir; >> } >> >> Here is the Swift loop that I am using to process this data: >> >> Data d[] = readData(@arg("data")); >> /* run quadui */ >> foreach ditem in d{ >> tracef("Survey data is: %s", ditem.Survey_data); >> } >> >> In the output, I get the whole first row of the file instead of the >> expected first field. >> >> Any clues? >> >> Thanks, >> Ketan >> > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketan at mcs.anl.gov Mon Jan 13 14:23:57 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Mon, 13 Jan 2014 14:23:57 -0600 Subject: [Swift-user] readData is treating line as a single field In-Reply-To: References: <44953e222acc429b8895c7d5c132b4fe@NAGURSKI.anl.gov> Message-ID: I found that the problem was a misplaced space in one of the paths in my original file used for the run. Removing the space fixed the issue. While trying to debug/reproduce, I found that readData() works with tabs, multiple spaces, combination of tabs and spaces and irregular number of spaces between fields. On Mon, Jan 13, 2014 at 1:48 PM, Yadu Nand wrote: > Hi Ketan, > > Could you give me a sample file and code you're using to test this out ? > > -Yadu > > > On Mon, Jan 13, 2014 at 11:46 AM, Ketan Maheshwari wrote: > >> Hi Mike, >> >> I am not sure how to introduce a field separator to readData(). I tried >> adding '|' between header and data fields but seems readData is complaining >> that file header does not match type and it expected space separated items: >> >> File header does not match type. Expected 4 whitespace separated items. >> Got 5 instead. >> swiftscript:readData, quadui-multi.swift, line 27 >> >> >> >> On Mon, Jan 13, 2014 at 11:05 AM, Wilde, Michael J. wrote: >> >>> Did you try different field separators? I think in the past Ive used >>> spaces. Perhaps the readData() code is not sufficiently white-space-aware >>> yet? >>> >>> - Mike >>> >>> >>> -- >>> Michael Wilde >>> Mathematics and Computer Science Computation Institute >>> Argonne National Laboratory The University of Chicago >>> >>> ------------------------------ >>> *From:* swift-user-bounces at ci.uchicago.edu [ >>> swift-user-bounces at ci.uchicago.edu] on behalf of Ketan Maheshwari [ >>> ketan at mcs.anl.gov] >>> *Sent:* Monday, January 13, 2014 11:01 AM >>> *To:* Swift User >>> *Subject:* [Swift-user] readData is treating line as a single field >>> >>> Hi, >>> >>> I am trying to use readData() function in Swift to read fields into a >>> Swift structure but seems it is reading the whole line as a single field. >>> >>> Here is my text file for the data: >>> >>> $ cat data.txt >>> Survey_data Strategy_data Field_data quadui_outdir >>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Survey_data_import_fixed.zip >>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Seasonal_strategy_fixed.zip >>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Field_Overlay_fixed.zip >>> /home/kcm92/outdir_1 >>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Survey_data_import_fixed.zip >>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Seasonal_strategy_fixed.zip >>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Field_Overlay_fixed.zip >>> /home/kcm92/outdir_2 >>> >>> I have 4 fields and 2 rows. >>> >>> Here is the structure definition: >>> >>> type Data{ >>> string Survey_data; >>> string Field_data; >>> string Strategy_data; >>> string quadui_outdir; >>> } >>> >>> Here is the Swift loop that I am using to process this data: >>> >>> Data d[] = readData(@arg("data")); >>> /* run quadui */ >>> foreach ditem in d{ >>> tracef("Survey data is: %s", ditem.Survey_data); >>> } >>> >>> In the output, I get the whole first row of the file instead of the >>> expected first field. >>> >>> Any clues? >>> >>> Thanks, >>> Ketan >>> >> >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >> > > > > -- > Yadu Nand B > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadudoc1729 at gmail.com Mon Jan 13 14:31:37 2014 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Mon, 13 Jan 2014 14:31:37 -0600 Subject: [Swift-user] readData is treating line as a single field In-Reply-To: References: <44953e222acc429b8895c7d5c132b4fe@NAGURSKI.anl.gov> Message-ID: Glad to hear that this isn't blocking you anymore. -Yadu- On Mon, Jan 13, 2014 at 2:23 PM, Ketan Maheshwari wrote: > I found that the problem was a misplaced space in one of the paths in my > original file used for the run. Removing the space fixed the issue. > > While trying to debug/reproduce, I found that readData() works with tabs, > multiple spaces, combination of tabs and spaces and irregular number of > spaces between fields. > > > On Mon, Jan 13, 2014 at 1:48 PM, Yadu Nand wrote: > >> Hi Ketan, >> >> Could you give me a sample file and code you're using to test this out ? >> >> -Yadu >> >> >> On Mon, Jan 13, 2014 at 11:46 AM, Ketan Maheshwari wrote: >> >>> Hi Mike, >>> >>> I am not sure how to introduce a field separator to readData(). I tried >>> adding '|' between header and data fields but seems readData is complaining >>> that file header does not match type and it expected space separated items: >>> >>> File header does not match type. Expected 4 whitespace separated items. >>> Got 5 instead. >>> swiftscript:readData, quadui-multi.swift, line 27 >>> >>> >>> >>> On Mon, Jan 13, 2014 at 11:05 AM, Wilde, Michael J. wrote: >>> >>>> Did you try different field separators? I think in the past Ive used >>>> spaces. Perhaps the readData() code is not sufficiently white-space-aware >>>> yet? >>>> >>>> - Mike >>>> >>>> >>>> -- >>>> Michael Wilde >>>> Mathematics and Computer Science Computation Institute >>>> Argonne National Laboratory The University of Chicago >>>> >>>> ------------------------------ >>>> *From:* swift-user-bounces at ci.uchicago.edu [ >>>> swift-user-bounces at ci.uchicago.edu] on behalf of Ketan Maheshwari [ >>>> ketan at mcs.anl.gov] >>>> *Sent:* Monday, January 13, 2014 11:01 AM >>>> *To:* Swift User >>>> *Subject:* [Swift-user] readData is treating line as a single field >>>> >>>> Hi, >>>> >>>> I am trying to use readData() function in Swift to read fields into a >>>> Swift structure but seems it is reading the whole line as a single field. >>>> >>>> Here is my text file for the data: >>>> >>>> $ cat data.txt >>>> Survey_data Strategy_data Field_data quadui_outdir >>>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Survey_data_import_fixed.zip >>>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Seasonal_strategy_fixed.zip >>>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/EMBU/Field_Overlay_fixed.zip >>>> /home/kcm92/outdir_1 >>>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Survey_data_import_fixed.zip >>>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Seasonal_strategy_fixed.zip >>>> /home/kcm92/faceit-pipeline/FIXED_ACE_DOME/ACCESS1/4.5/MID/ISHIARA/Field_Overlay_fixed.zip >>>> /home/kcm92/outdir_2 >>>> >>>> I have 4 fields and 2 rows. >>>> >>>> Here is the structure definition: >>>> >>>> type Data{ >>>> string Survey_data; >>>> string Field_data; >>>> string Strategy_data; >>>> string quadui_outdir; >>>> } >>>> >>>> Here is the Swift loop that I am using to process this data: >>>> >>>> Data d[] = readData(@arg("data")); >>>> /* run quadui */ >>>> foreach ditem in d{ >>>> tracef("Survey data is: %s", ditem.Survey_data); >>>> } >>>> >>>> In the output, I get the whole first row of the file instead of the >>>> expected first field. >>>> >>>> Any clues? >>>> >>>> Thanks, >>>> Ketan >>>> >>> >>> >>> _______________________________________________ >>> Swift-user mailing list >>> Swift-user at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >>> >> >> >> >> -- >> Yadu Nand B >> >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >> > > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From khawar.ahmad at cern.ch Wed Jan 15 09:34:40 2014 From: khawar.ahmad at cern.ch (Khawar Ahmad) Date: Wed, 15 Jan 2014 15:34:40 +0000 Subject: [Swift-user] sites.xml for cloud Message-ID: <5E046D241F2A1749918B990591AF54F7C81790D6@CERNXCHG32.cern.ch> Hi, I am trying to use SWIFT to run sample programs on my test Cloud setup. I am following a tutorial (http://swift-lang.org/tutorials/cloud/tutorial.html). However, the sites.xml file for Part4 is empty and I get the following error from swift "Execution failed: Could not load file sites.xml: com.thoughtworks.xstream.io.StreamException: : input contained no data" Can you tell me what are the settings to run jobs in the Cloud environment? The Cloud is setup using OpenStack. Regards, Khawar -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2720 bytes Desc: not available URL: From davidk at ci.uchicago.edu Wed Jan 15 10:17:46 2014 From: davidk at ci.uchicago.edu (David Kelly) Date: Wed, 15 Jan 2014 10:17:46 -0600 Subject: [Swift-user] sites.xml for cloud In-Reply-To: <5E046D241F2A1749918B990591AF54F7C81790D6@CERNXCHG32.cern.ch> References: <5E046D241F2A1749918B990591AF54F7C81790D6@CERNXCHG32.cern.ch> Message-ID: Hi Khawar, For running with the cloud tutorial, all the settings are defined in a file called scs/coaster-service.conf. The only things you should need to change there is the list of IP addresses, the remote username, and remote directories to use. When you run the command "source setup.sh", it should start the workers on remote sites and create a sites.xml file in each the part directories. Are you getting any errors when you run "source setup.sh"? Thanks, David On Wed, Jan 15, 2014 at 9:34 AM, Khawar Ahmad wrote: > Hi, > > I am trying to use SWIFT to run sample programs on my test Cloud setup. I > am following a tutorial ( > http://swift-lang.org/tutorials/cloud/tutorial.html). > However, the sites.xml file for Part4 is empty and I get the following > error from swift > "Execution failed: > Could not load file sites.xml: > com.thoughtworks.xstream.io.StreamException: : input contained no data" > > Can you tell me what are the settings to run jobs in the Cloud > environment? The Cloud is setup using OpenStack. > > Regards, > Khawar > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From khawar.ahmad at cern.ch Wed Jan 15 12:25:43 2014 From: khawar.ahmad at cern.ch (Khawar Ahmad) Date: Wed, 15 Jan 2014 18:25:43 +0000 Subject: [Swift-user] sites.xml for cloud In-Reply-To: References: <5E046D241F2A1749918B990591AF54F7C81790D6@CERNXCHG32.cern.ch> Message-ID: <5E046D241F2A1749918B990591AF54F7C8179254@CERNXCHG32.cern.ch> Hi David, I have done these changes. See the conf file below. xport WORKER_LOCATION=/home/ubuntu export WORKER_HOSTS="192.168.100.3 192.168.100.4" export WORKER_MODE=ssh export WORKER_USERNAME=ubuntu export IPADDR="192.168.100.2" export WORKER_LOG_DIR=/home/ubuntu export WORKER_LOGGING_LEVEL=DEBUG export WORK=/home/ubuntu/work export JOBSPERNODE=1 export JOBTHROTTLE=10 export SSH_TUNNELING=yes setup.sh went fine without errors. see the output below. Swift version is Swift 0.94.1 swift-r7114 cog-r3803 Assuming /home/ubuntu/swift-cloud-tutorial/bin:/home/ubuntu/swift-cloud-tutorial/app: is already at front of PATH Saving /home/ubuntu/.swift/swift.properties in /home/ubuntu/.swift/swift.properties.LTCg Start-coaster-service... Configuration: /home/ubuntu/swift-cloud-tutorial/scs/coaster-service.conf Service address: 192.168.100.2 Starting coaster-service Service port: 52871 Local port: 60572 Generating sites.xml Cannot find template for persistent-coasters Starting worker on 192.168.100.3 Starting worker on 192.168.100.4 However, the sites.xml file after part03 are empty. see the output below. ubuntu at uwe-vm1:~/swift-cloud-tutorial$ more part0*/sites.xml :::::::::::::: part01/sites.xml :::::::::::::: 0 10000 .swift/tmp local :::::::::::::: part02/sites.xml :::::::::::::: 0 10000 .swift/tmp local :::::::::::::: part03/sites.xml :::::::::::::: 0 10000 .swift/tmp local :::::::::::::: part04/sites.xml :::::::::::::: :::::::::::::: part05/sites.xml :::::::::::::: :::::::::::::: part06/sites.xml :::::::::::::: In the tutorial, part4 onwards is described as running on compute nodes and apps file has entry ?cloud sh /bin/bash?, however, I could not see how ?cloud? would be resolved. This is why, I could not run examples part03 onwards. Any help? Regards, Khawar On 15 Jan 2014, at 16:17, David Kelly wrote: > Hi Khawar, > > For running with the cloud tutorial, all the settings are defined in a file called scs/coaster-service.conf. The only things you should need to change there is the list of IP addresses, the remote username, and remote directories to use. > > When you run the command "source setup.sh", it should start the workers on remote sites and create a sites.xml file in each the part directories. Are you getting any errors when you run "source setup.sh"? > > Thanks, > David > > > On Wed, Jan 15, 2014 at 9:34 AM, Khawar Ahmad wrote: > Hi, > > I am trying to use SWIFT to run sample programs on my test Cloud setup. I am following a tutorial (http://swift-lang.org/tutorials/cloud/tutorial.html). > However, the sites.xml file for Part4 is empty and I get the following error from swift > "Execution failed: > Could not load file sites.xml: com.thoughtworks.xstream.io.StreamException: : input contained no data" > > Can you tell me what are the settings to run jobs in the Cloud environment? The Cloud is setup using OpenStack. > > Regards, > Khawar > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2720 bytes Desc: not available URL: From davidk at ci.uchicago.edu Wed Jan 15 13:53:52 2014 From: davidk at ci.uchicago.edu (David Kelly) Date: Wed, 15 Jan 2014 13:53:52 -0600 Subject: [Swift-user] sites.xml for cloud In-Reply-To: <5E046D241F2A1749918B990591AF54F7C8179254@CERNXCHG32.cern.ch> References: <5E046D241F2A1749918B990591AF54F7C81790D6@CERNXCHG32.cern.ch> <5E046D241F2A1749918B990591AF54F7C8179254@CERNXCHG32.cern.ch> Message-ID: Hi Khawaar, Sorry that you're running into problems. I think the issue is related to message "Cannot find template for persistent-coasters", which is strange because that file should be included with Swift. In the place where Swift is installed, do you see a file called etc/sites/persistent-coasters? On Wed, Jan 15, 2014 at 12:25 PM, Khawar Ahmad wrote: > Hi David, > > I have done these changes. See the conf file below. > > xport WORKER_LOCATION=/home/ubuntu > export WORKER_HOSTS="192.168.100.3 192.168.100.4" > export WORKER_MODE=ssh > export WORKER_USERNAME=ubuntu > export IPADDR="192.168.100.2" > export WORKER_LOG_DIR=/home/ubuntu > export WORKER_LOGGING_LEVEL=DEBUG > export WORK=/home/ubuntu/work > export JOBSPERNODE=1 > export JOBTHROTTLE=10 > export SSH_TUNNELING=yes > > setup.sh went fine without errors. see the output below. > > Swift version is Swift 0.94.1 swift-r7114 cog-r3803 > Assuming > /home/ubuntu/swift-cloud-tutorial/bin:/home/ubuntu/swift-cloud-tutorial/app: > is already at front of PATH > Saving /home/ubuntu/.swift/swift.properties in > /home/ubuntu/.swift/swift.properties.LTCg > Start-coaster-service... > Configuration: /home/ubuntu/swift-cloud-tutorial/scs/coaster-service.conf > Service address: 192.168.100.2 > Starting coaster-service > Service port: 52871 > Local port: 60572 > Generating sites.xml > Cannot find template for persistent-coasters > Starting worker on 192.168.100.3 > Starting worker on 192.168.100.4 > > > However, the sites.xml file after part03 are empty. see the output below. > > ubuntu at uwe-vm1:~/swift-cloud-tutorial$ more part0*/sites.xml > :::::::::::::: > part01/sites.xml > :::::::::::::: > > > > > 0 > 10000 > > .swift/tmp > local > > > :::::::::::::: > part02/sites.xml > :::::::::::::: > > > > > 0 > 10000 > > .swift/tmp > local > > > :::::::::::::: > part03/sites.xml > :::::::::::::: > > > > > 0 > 10000 > > .swift/tmp > local > > > :::::::::::::: > part04/sites.xml > :::::::::::::: > :::::::::::::: > part05/sites.xml > :::::::::::::: > :::::::::::::: > part06/sites.xml > :::::::::::::: > > > In the tutorial, part4 onwards is described as running on compute nodes > and apps file has entry ?cloud sh /bin/bash?, however, I could not see how > ?cloud? would be resolved. > This is why, I could not run examples part03 onwards. Any help? > > > Regards, > Khawar > On 15 Jan 2014, at 16:17, David Kelly wrote: > > Hi Khawar, > > For running with the cloud tutorial, all the settings are defined in a > file called scs/coaster-service.conf. The only things you should need to > change there is the list of IP addresses, the remote username, and remote > directories to use. > > When you run the command "source setup.sh", it should start the workers on > remote sites and create a sites.xml file in each the part directories. Are > you getting any errors when you run "source setup.sh"? > > Thanks, > David > > > On Wed, Jan 15, 2014 at 9:34 AM, Khawar Ahmad wrote: > >> Hi, >> >> I am trying to use SWIFT to run sample programs on my test Cloud setup. I >> am following a tutorial ( >> http://swift-lang.org/tutorials/cloud/tutorial.html). >> However, the sites.xml file for Part4 is empty and I get the following >> error from swift >> "Execution failed: >> Could not load file sites.xml: >> com.thoughtworks.xstream.io.StreamException: : input contained no data" >> >> Can you tell me what are the settings to run jobs in the Cloud >> environment? The Cloud is setup using OpenStack. >> >> Regards, >> Khawar >> >> _______________________________________________ >> Swift-user mailing list >> Swift-user at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From khawar.ahmad at cern.ch Wed Jan 15 14:56:23 2014 From: khawar.ahmad at cern.ch (Khawar Ahmad) Date: Wed, 15 Jan 2014 20:56:23 +0000 Subject: [Swift-user] sites.xml for cloud In-Reply-To: References: <5E046D241F2A1749918B990591AF54F7C81790D6@CERNXCHG32.cern.ch> <5E046D241F2A1749918B990591AF54F7C8179254@CERNXCHG32.cern.ch>, Message-ID: Hi David, Yes, I can see this file in swift installed directory. Regards, Khawar On 15 Jan 2014, at 19:53, "David Kelly" > wrote: Hi Khawaar, Sorry that you're running into problems. I think the issue is related to message "Cannot find template for persistent-coasters", which is strange because that file should be included with Swift. In the place where Swift is installed, do you see a file called etc/sites/persistent-coasters? On Wed, Jan 15, 2014 at 12:25 PM, Khawar Ahmad > wrote: Hi David, I have done these changes. See the conf file below. xport WORKER_LOCATION=/home/ubuntu export WORKER_HOSTS="192.168.100.3 192.168.100.4" export WORKER_MODE=ssh export WORKER_USERNAME=ubuntu export IPADDR="192.168.100.2" export WORKER_LOG_DIR=/home/ubuntu export WORKER_LOGGING_LEVEL=DEBUG export WORK=/home/ubuntu/work export JOBSPERNODE=1 export JOBTHROTTLE=10 export SSH_TUNNELING=yes setup.sh went fine without errors. see the output below. Swift version is Swift 0.94.1 swift-r7114 cog-r3803 Assuming /home/ubuntu/swift-cloud-tutorial/bin:/home/ubuntu/swift-cloud-tutorial/app: is already at front of PATH Saving /home/ubuntu/.swift/swift.properties in /home/ubuntu/.swift/swift.properties.LTCg Start-coaster-service... Configuration: /home/ubuntu/swift-cloud-tutorial/scs/coaster-service.conf Service address: 192.168.100.2 Starting coaster-service Service port: 52871 Local port: 60572 Generating sites.xml Cannot find template for persistent-coasters Starting worker on 192.168.100.3 Starting worker on 192.168.100.4 However, the sites.xml file after part03 are empty. see the output below. ubuntu at uwe-vm1:~/swift-cloud-tutorial$ more part0*/sites.xml :::::::::::::: part01/sites.xml :::::::::::::: 0 10000 .swift/tmp local :::::::::::::: part02/sites.xml :::::::::::::: 0 10000 .swift/tmp local :::::::::::::: part03/sites.xml :::::::::::::: 0 10000 .swift/tmp local :::::::::::::: part04/sites.xml :::::::::::::: :::::::::::::: part05/sites.xml :::::::::::::: :::::::::::::: part06/sites.xml :::::::::::::: In the tutorial, part4 onwards is described as running on compute nodes and apps file has entry ?cloud sh /bin/bash?, however, I could not see how ?cloud? would be resolved. This is why, I could not run examples part03 onwards. Any help? Regards, Khawar On 15 Jan 2014, at 16:17, David Kelly > wrote: Hi Khawar, For running with the cloud tutorial, all the settings are defined in a file called scs/coaster-service.conf. The only things you should need to change there is the list of IP addresses, the remote username, and remote directories to use. When you run the command "source setup.sh", it should start the workers on remote sites and create a sites.xml file in each the part directories. Are you getting any errors when you run "source setup.sh"? Thanks, David On Wed, Jan 15, 2014 at 9:34 AM, Khawar Ahmad > wrote: Hi, I am trying to use SWIFT to run sample programs on my test Cloud setup. I am following a tutorial (http://swift-lang.org/tutorials/cloud/tutorial.html). However, the sites.xml file for Part4 is empty and I get the following error from swift "Execution failed: Could not load file sites.xml: com.thoughtworks.xstream.io.StreamException: : input contained no data" Can you tell me what are the settings to run jobs in the Cloud environment? The Cloud is setup using OpenStack. Regards, Khawar _______________________________________________ Swift-user mailing list Swift-user at ci.uchicago.edu https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadudoc1729 at gmail.com Thu Jan 16 11:06:36 2014 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Thu, 16 Jan 2014 11:06:36 -0600 Subject: [Swift-user] sites.xml for cloud In-Reply-To: References: <5E046D241F2A1749918B990591AF54F7C81790D6@CERNXCHG32.cern.ch> <5E046D241F2A1749918B990591AF54F7C8179254@CERNXCHG32.cern.ch> Message-ID: Hi Khawar, Could you copy the /etc/sites/persistent-coasters file in your swift installation to your $HOME/.swift/sites folder ? As you noted earlier, the apps file from part04 onwards should reference "persistent-coasters" as defined in the sites.xml instead of "cloud". So, the apps file should look like this : persistent-coasters sh /bin/bash And the sites.xml generated should look like this (with some changes for your settings): passive 1 10 10000 /tmp/swiftwork Please let us know how this works out for you. Thanks, Yadu On Wed, Jan 15, 2014 at 2:56 PM, Khawar Ahmad wrote: > Hi David, > > Yes, I can see this file in swift installed directory. > > Regards, > Khawar > > On 15 Jan 2014, at 19:53, "David Kelly" wrote: > > Hi Khawaar, > > Sorry that you're running into problems. I think the issue is related to > message "Cannot find template for persistent-coasters", which is strange > because that file should be included with Swift. In the place where Swift > is installed, do you see a file called etc/sites/persistent-coasters? > > > On Wed, Jan 15, 2014 at 12:25 PM, Khawar Ahmad wrote: > >> Hi David, >> >> I have done these changes. See the conf file below. >> >> xport WORKER_LOCATION=/home/ubuntu >> export WORKER_HOSTS="192.168.100.3 192.168.100.4" >> export WORKER_MODE=ssh >> export WORKER_USERNAME=ubuntu >> export IPADDR="192.168.100.2" >> export WORKER_LOG_DIR=/home/ubuntu >> export WORKER_LOGGING_LEVEL=DEBUG >> export WORK=/home/ubuntu/work >> export JOBSPERNODE=1 >> export JOBTHROTTLE=10 >> export SSH_TUNNELING=yes >> >> setup.sh went fine without errors. see the output below. >> >> Swift version is Swift 0.94.1 swift-r7114 cog-r3803 >> Assuming >> /home/ubuntu/swift-cloud-tutorial/bin:/home/ubuntu/swift-cloud-tutorial/app: >> is already at front of PATH >> Saving /home/ubuntu/.swift/swift.properties in >> /home/ubuntu/.swift/swift.properties.LTCg >> Start-coaster-service... >> Configuration: /home/ubuntu/swift-cloud-tutorial/scs/coaster-service.conf >> Service address: 192.168.100.2 >> Starting coaster-service >> Service port: 52871 >> Local port: 60572 >> Generating sites.xml >> Cannot find template for persistent-coasters >> Starting worker on 192.168.100.3 >> Starting worker on 192.168.100.4 >> >> >> However, the sites.xml file after part03 are empty. see the output >> below. >> >> ubuntu at uwe-vm1:~/swift-cloud-tutorial$ more part0*/sites.xml >> :::::::::::::: >> part01/sites.xml >> :::::::::::::: >> >> >> >> >> 0 >> 10000 >> >> .swift/tmp >> local >> >> >> :::::::::::::: >> part02/sites.xml >> :::::::::::::: >> >> >> >> >> 0 >> 10000 >> >> .swift/tmp >> local >> >> >> :::::::::::::: >> part03/sites.xml >> :::::::::::::: >> >> >> >> >> 0 >> 10000 >> >> .swift/tmp >> local >> >> >> :::::::::::::: >> part04/sites.xml >> :::::::::::::: >> :::::::::::::: >> part05/sites.xml >> :::::::::::::: >> :::::::::::::: >> part06/sites.xml >> :::::::::::::: >> >> >> In the tutorial, part4 onwards is described as running on compute nodes >> and apps file has entry ?cloud sh /bin/bash?, however, I could not see how >> ?cloud? would be resolved. >> This is why, I could not run examples part03 onwards. Any help? >> >> >> Regards, >> Khawar >> On 15 Jan 2014, at 16:17, David Kelly wrote: >> >> Hi Khawar, >> >> For running with the cloud tutorial, all the settings are defined in a >> file called scs/coaster-service.conf. The only things you should need to >> change there is the list of IP addresses, the remote username, and remote >> directories to use. >> >> When you run the command "source setup.sh", it should start the workers >> on remote sites and create a sites.xml file in each the part directories. >> Are you getting any errors when you run "source setup.sh"? >> >> Thanks, >> David >> >> >> On Wed, Jan 15, 2014 at 9:34 AM, Khawar Ahmad wrote: >> >>> Hi, >>> >>> I am trying to use SWIFT to run sample programs on my test Cloud >>> setup. I am following a tutorial ( >>> http://swift-lang.org/tutorials/cloud/tutorial.html). >>> However, the sites.xml file for Part4 is empty and I get the following >>> error from swift >>> "Execution failed: >>> Could not load file sites.xml: >>> com.thoughtworks.xstream.io.StreamException: : input contained no data" >>> >>> Can you tell me what are the settings to run jobs in the Cloud >>> environment? The Cloud is setup using OpenStack. >>> >>> Regards, >>> Khawar >>> >>> _______________________________________________ >>> Swift-user mailing list >>> Swift-user at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user >>> >> >> >> > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From iraicu at cs.iit.edu Tue Jan 21 04:08:31 2014 From: iraicu at cs.iit.edu (Ioan Raicu) Date: Tue, 21 Jan 2014 04:08:31 -0600 Subject: [Swift-user] CFP: 5th Workshop on Scientific Cloud Computing (ScienceCloud) @ ACM HPDC 2014 Message-ID: <52DE471F.2010005@cs.iit.edu> Call for Papers: 5th Workshop on Scientific Cloud Computing (ScienceCloud) June 23/24, 2014. Vancouver, Canada (http://datasys.cs.iit.edu/events/ScienceCloud2014/) Co-Located with HPDC 2014 ------------------------------------------------------------------------------- IMPORTANT DATES Paper Submission: March 1, 2014 Acceptance Notification: April 4, 2014 Final Papers: April 11, 2014 Workshop: June 23/24, 2014 ------------------------------------------------------------------------------- OVERVIEW Computational and Data-Driven Sciences have become the third and fourth pillar of scientific discovery in addition to experimental and theoretical sciences. Scientific Computing has already begun to change how science is done, enabling scientific breakthroughs through new kinds of experiments that would have been impossible only a decade ago. Today?s ?Big Data? science is generating datasets that are increasing exponentially in both complexity and volume, making their analysis, archival, and sharing one of the grand challenges of the 21st century. The support for data intensive computing is critical to advance modern science as storage systems have exposed a widening gap between their capacity and their bandwidth by more than 10-fold over the last decade. There is a growing need for advanced techniques to manipulate, visualize and interpret large datasets. Scientific Computing is the key to solving ?grand challenges? in many domains and providing breakthroughs in new knowledge, and it comes in many shapes and forms: high-performance computing (HPC) which is heavily focused on compute-intensive applications; high-throughput computing (HTC) which focuses on using many computing resources over long periods of time to accomplish its computational tasks; many-task computing (MTC) which aims to bridge the gap between HPC and HTC by focusing on using many resources over short periods of time; and data-intensive computing which is heavily focused on data distribution, data-parallel execution, and harnessing data locality by scheduling of computations close to the data. The 5th workshop on Scientific Cloud Computing (ScienceCloud) will provide the scientific community a dedicated forum for discussing new research, development, and deployment efforts in running these kinds of scientific computing workloads on Cloud Computing infrastructures. The ScienceCloud workshop will focus on the use of cloud-based technologies to meet new compute-intensive and data-intensive scientific challenges that are not well served by the current supercomputers, grids and HPC clusters. The workshop will aim to address questions such as: What architectural changes to the current cloud frameworks (hardware, operating systems, networking and/or programming models) are needed to support science? Dynamic information derived from remote instruments and coupled simulation, and sensor ensembles that stream data for real-time analysis are important emerging techniques in scientific and cyber-physical engineering systems. How can cloud technologies enable and adapt to these new scientific approaches dealing with dynamism? How are scientists using clouds? Are there scientific HPC/HTC/MTC workloads that are suitable candidates to take advantage of emerging cloud computing resources with high efficiency? Commercial public clouds provide easy access to cloud infrastructure for scientists. What are the gaps in commercial cloud offerings and how can they be adapted for running existing and novel eScience applications? What benefits exist by adopting the cloud model, over clusters, grids, or supercomputers? What factors are limiting clouds use or would make them more usable/efficient? This workshop encourages interaction and cross-pollination between those developing applications, algorithms, software, hardware and networking, emphasizing scientific computing for such cloud platforms. We believe the workshop will be an excellent place to help the community define the current state, determine future goals, and define architectures and services for future science clouds. ------------------------------------------------------------------------------- WORKSHOP SCOPE We invite the submission of original work that is related to the topics below. The papers can be either short (4 pages) position papers, or long (8 pages) research papers. Topics of interest include (in the context of Cloud Computing): Scientific application cases studies on Cloud infrastructure Performance evaluation of Cloud environments and technologies Fault tolerance and reliability in cloud systems Data-intensive workloads and tools on Clouds Use of programming models such as Map-Reduce and its implementations Storage cloud architectures I/O and Data management in the Cloud Workflow and resource management in the Cloud Use of cloud technologies (e.g., NoSQL databases) for scientific applications Data streaming and dynamic applications on Clouds Dynamic resource provisioning Many-Task Computing in the Cloud Application of cloud concepts in HPC environments or vice versa High performance parallel file systems in virtual environments Virtualized high performance I/O network interconnects Virtualization Distributed Operating Systems Many-core computing and accelerators (e.g. GPUs, MIC) in the Cloud Cloud security ------------------------------------------------------------------------------- SUBMISSION INSTRUCTIONS Authors are invited to submit papers with unpublished, original work of not more than 8 pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages (including all text, figures, and references), as per ACM 8.5 x 11 manuscript guidelines (document templates can be found at http://www.acm.org/sigs/publications/proceedings-templates). A 250 word abstract and the final paper in PDF format must be submitted online at https://cmt.research.microsoft.com/SCIENCECLOUD2014/ before the deadline. Papers will be peer-reviewed, and accepted papers will be published in the workshop proceedings as part of the ACM digital library. Notifications of the paper decisions will be sent out by April 4th, 2014. Submission implies the willingness of at least one of the authors to register and present the paper. ------------------------------------------------------------------------------- JOURNAL SPECIAL ISSUE: IEEE Transaction on Cloud Computing Selected excellent work will be invited to submit extended versions of the workshop paper to the Special Issue on Scientific Cloud Computing in the IEEE Transactions on Cloud Computing (http://datasys.cs.iit.edu/events/ScienceCloud2014-TCC/). ------------------------------------------------------------------------------- GENERAL CHAIRS - Ioan Raicu, Illinois Institute of Technology & Argonne National Laboratory, USA - Kate Keahey, University of Chicago & Argonne National Laboratory, USA PROGRAM COMMITTEE CHAIRS - Kyle Chard, University of Chicago, USA - Bogdan Nicolae, IBM Research, Ireland STEERING COMMITTEE - Ian Foster, University of Chicago & Argonne National Laboratory, USA - Pete Beckman, University of Chicago & Argonne National Laboratory, USA - Carole Goble, University of Manchester, UK - Dennis Gannon, Microsoft Research, USA - Robert Grossman, University of Chicago, USA - Ed Lazowska, University of Washington & Computing Community Consortium, USA - David O'Hallaron, Carnegie Mellon University & Intel Labs, USA - Jack Dongarra, University of Tennessee, USA - Geoffrey Fox, Indiana University, USA - Yogesh Simmhan, University of Southern California, USA - Gabriel Antoniu, INRIA, France - Lavanya Ramakrishnan, Lawrence Berkeley National Lab, USA PROGRAM COMMITTEE - Samer Al-Kiswany, University of British Columbia - Roger Barga, Microsoft Research - Simon Caton, Karlsruhe Institute of Technology - Ake Edlund, Royal Institute of Technology - Chathura Herath, Indiana University - Neil Chue Hong, University of Edinburgh - Shantenu Jha, Rutgers - Carl Kesselman, University of Southern California - Thilo Kielmann, Vrije University - Shiyong Lu, Wayne State University - Wei Lu, Microsoft Research - David Martin, Argonne National Laboratory - Gabriel Mateescu, EURAC Research, Italy - Paolo Missier, University of Manchester - Ruben Montero, Universidad Complutense de Madrid - Reagan Moore, University of North Carolina - Pasquale Pagano, ISTI - Beth Plale, Indiana University - Omer Rana, Cardiff University - Matei Ripeanu, University of British Columbia - Josh Simons, VMWare - Douglas Thain, University of Notre Dame - Johan Tordsson, Ume University - Zhifeng Yun, Louisiana State University - Yong Zhao, University of Electronic and Science Technology of China -- ================================================================= Ioan Raicu, Ph.D. Assistant Professor, Illinois Institute of Technology (IIT) Guest Research Faculty, Argonne National Laboratory (ANL) ================================================================= Data-Intensive Distributed Systems Laboratory, CS/IIT Distributed Systems Laboratory, MCS/ANL ================================================================= Editor: IEEE TCC, Springer Cluster, Springer JoCCASA Chair: IEEE/ACM MTAGS, ACM ScienceCloud ================================================================= Cel: 1-847-722-0876 Office: 1-312-567-5704 Email: iraicu at cs.iit.edu Web: http://www.cs.iit.edu/~iraicu/ Web: http://datasys.cs.iit.edu/ LinkedIn: http://www.linkedin.com/in/ioanraicu Google: http://scholar.google.com/citations?user=jE73HYAAAAAJ ================================================================= ================================================================= From ketan at mcs.anl.gov Tue Jan 21 16:27:44 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Tue, 21 Jan 2014 16:27:44 -0600 Subject: [Swift-user] Swift retries despite execution.retries set to 0 Message-ID: Hi, It seems Swift latest trunk attempts to retry failed executions despite the execution.retries property set to 0. A tarred rundir is attached. Thanks, Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: run002.tgz Type: application/x-gzip Size: 6155 bytes Desc: not available URL: From ketan at mcs.anl.gov Tue Jan 21 16:40:54 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Tue, 21 Jan 2014 16:40:54 -0600 Subject: [Swift-user] swift submitting message for long time Message-ID: Hi, For the ssh-cl:xxx cases, with the trunk, I am getting "submitting" status message for a long time but seems nothing is being submitted. In log, I see the following messages which seems off: Submit: in: null command: ssh -p 22 blues.lcrc.anl.gov ls -1 /home/ketan/.globus/ssh*-*-* Copying proxy Submit: in: null command: scp -p -P 22 /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshproxy-1798790086-1391553457 blues.lcrc.anl.gov:/home/ketan/.globus/sshproxy-1798790086-1391553457 Copying certificate Submit: in: null command: scp -p -P 22 /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshCAcert-1798790086-1391553457.pem blues.lcrc.anl.gov:/home/ketan/.globus/sshCAcert-1798790086-1391553457.pem Copying signing policy file Submit: in: null command: scp -p -P 22 /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshCAcert-1798790086-1391553457.signing_policy blues.lcrc.anl.gov: /home/ketan/.globus/sshCAcert-1798790086-1391553457.signing_policy This is going from my laptop into blues. The rundir is attached. Thanks, Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: run007.tgz Type: application/x-gzip Size: 5068 bytes Desc: not available URL: From hategan at mcs.anl.gov Tue Jan 21 16:45:37 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 21 Jan 2014 14:45:37 -0800 Subject: [Swift-user] swift submitting message for long time In-Reply-To: References: Message-ID: <1390344337.27055.1.camel@echo> Yeah, that's "normal". Submitting means that the provider got the job. Before the coaster service is started, and if you're using the AutoCA, some proxy stuff needs to be set up. With SSH-CL this might take a while, since each invocation of an ssh command does some heavy crypto stuff. I can try to see if the number of operations can be reduced, but it's not exactly high on my list. Mihael On Tue, 2014-01-21 at 16:40 -0600, Ketan Maheshwari wrote: > Hi, > > For the ssh-cl:xxx cases, with the trunk, I am getting "submitting" status > message for a long time but seems nothing is being submitted. In log, I see > the following messages which seems off: > > Submit: in: null command: ssh -p 22 blues.lcrc.anl.gov ls -1 > /home/ketan/.globus/ssh*-*-* > Copying proxy > Submit: in: null command: scp -p -P 22 > /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshproxy-1798790086-1391553457 > blues.lcrc.anl.gov:/home/ketan/.globus/sshproxy-1798790086-1391553457 > Copying certificate > Submit: in: null command: scp -p -P 22 > /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshCAcert-1798790086-1391553457.pem > blues.lcrc.anl.gov:/home/ketan/.globus/sshCAcert-1798790086-1391553457.pem > Copying signing policy file > Submit: in: null command: scp -p -P 22 > /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshCAcert-1798790086-1391553457.signing_policy > blues.lcrc.anl.gov: > /home/ketan/.globus/sshCAcert-1798790086-1391553457.signing_policy > > This is going from my laptop into blues. The rundir is attached. > > Thanks, > Ketan > Hi, > > > For the ssh-cl:xxx cases, with the trunk, I am getting "submitting" > status message for a long time but seems nothing is being submitted. > In log, I see the following messages which seems off: > > > Submit: in: null command: ssh -p 22 blues.lcrc.anl.gov ls > -1 /home/ketan/.globus/ssh*-*-* > Copying proxy > Submit: in: null command: scp -p -P > 22 /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshproxy-1798790086-1391553457 blues.lcrc.anl.gov:/home/ketan/.globus/sshproxy-1798790086-1391553457 > Copying certificate > Submit: in: null command: scp -p -P > 22 /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshCAcert-1798790086-1391553457.pem blues.lcrc.anl.gov:/home/ketan/.globus/sshCAcert-1798790086-1391553457.pem > Copying signing policy file > Submit: in: null command: scp -p -P > 22 /var/folders/6k/9pg8w51j7wj_q1wjw15969400000gn/T/sshCAcert-1798790086-1391553457.signing_policy blues.lcrc.anl.gov:/home/ketan/.globus/sshCAcert-1798790086-1391553457.signing_policy > > > This is going from my laptop into blues. The rundir is attached. > > > Thanks, > Ketan > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user From davidkelly at uchicago.edu Tue Jan 21 17:44:53 2014 From: davidkelly at uchicago.edu (David Kelly) Date: Tue, 21 Jan 2014 17:44:53 -0600 Subject: [Swift-user] Swift retries despite execution.retries set to 0 In-Reply-To: References: Message-ID: I tried a local:slurm test with a failing app, and it stopped after one failure. The error you're seeing might be specific to ssh-cl. Also just wanted to mention that in your sites.xml you have {env.USER}. This no longer works in 0.95/trunk. You can do something similar though when/if you move to the new config. On Tue, Jan 21, 2014 at 4:27 PM, Ketan Maheshwari wrote: > Hi, > > It seems Swift latest trunk attempts to retry failed executions despite > the execution.retries property set to 0. A tarred rundir is attached. > > Thanks, > Ketan > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Tue Jan 21 18:46:20 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Tue, 21 Jan 2014 16:46:20 -0800 Subject: [Swift-user] Swift retries despite execution.retries set to 0 In-Reply-To: References: Message-ID: <1390351580.28488.3.camel@echo> I think this is a problem with how the job status is done combined with some genuine problem. All failing jobs are marked "Failed but can retry" at first. Eventually they get to "Failed", but there are some steps in-between that for some reason are not working properly. I need to check why. Mihael On Tue, 2014-01-21 at 17:44 -0600, David Kelly wrote: > I tried a local:slurm test with a failing app, and it stopped after one > failure. The error you're seeing might be specific to ssh-cl. > > Also just wanted to mention that in your sites.xml you have {env.USER}. > This no longer works in 0.95/trunk. You can do something similar though > when/if you move to the new config. > > > On Tue, Jan 21, 2014 at 4:27 PM, Ketan Maheshwari wrote: > > > Hi, > > > > It seems Swift latest trunk attempts to retry failed executions despite > > the execution.retries property set to 0. A tarred rundir is attached. > > > > Thanks, > > Ketan > > > > _______________________________________________ > > Swift-user mailing list > > Swift-user at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > > I tried a local:slurm test with a failing app, and it stopped after > one failure. The error you're seeing might be specific to ssh-cl. > > > Also just wanted to mention that in your sites.xml you have > {env.USER}. This no longer works in 0.95/trunk. You can do something > similar though when/if you move to the new config. > > > On Tue, Jan 21, 2014 at 4:27 PM, Ketan Maheshwari > wrote: > Hi, > > > It seems Swift latest trunk attempts to retry failed > executions despite the execution.retries property set to 0. A > tarred rundir is attached. > > > Thanks, > Ketan > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > > _______________________________________________ > Swift-user mailing list > Swift-user at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user From mickelso at ucar.edu Fri Jan 24 12:17:56 2014 From: mickelso at ucar.edu (Sheri Mickelson) Date: Fri, 24 Jan 2014 18:17:56 -0000 Subject: [Swift-user] Log plots Message-ID: <52E2AE52.4070103@ucar.edu> I'm currently running with Swift version 0.94.1-RC1 and I'd like to create some plots from the log file. I've tried following the directions on http://swift-lang.org/guides/release-0.94/userguide/userguide.html#_log_processing, but I keep ended up with empty files. I *think* the normalized file looks okay, but the files created in the next steps don't look correct or are empty. Is there a trick to creating the plots? Thanks, Sheri