From davidkelly at uchicago.edu Thu Apr 3 16:59:40 2014 From: davidkelly at uchicago.edu (David Kelly) Date: Thu, 3 Apr 2014 16:59:40 -0500 Subject: [Swift-devel] How can I reduce log file size? Message-ID: I'm running a script in 0.94.1 that generates a large log file. The script only runs 41 tasks and finishes in under 5 minutes, but the log file is 2.7 gigabytes. I have: provenance.log=false tracing.enabled=false I've tried "-minimal.logging" and "-reduced.logging" but it doesn't make much of a difference. What else can I do to reduce the log file size? I'm running Swift from a module so I can't easily edit any swift-wide log4j settings. The log is at http://web.ci.uchicago.edu/~davidk/logs/combine-20140403-2122-csdnssc1.log.gz(17mb compressed, 2.7 gigabytes uncompressed) Thanks, David -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Thu Apr 3 21:02:48 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 3 Apr 2014 19:02:48 -0700 Subject: [Swift-devel] How can I reduce log file size? In-Reply-To: References: Message-ID: <1396576968.10915.2.camel@echo> minimal.logging should be the one doing the trick, but if that fails... everything is ultimately configured through log4j. So it's between log4j.properties and Loader.java who overrides some logging options based on command line arguments. You could, for example, try to set a threshold of WARN for the file appender in log4j.properties and make sure that Loader doesn't mess with that. Mihael On Thu, 2014-04-03 at 16:59 -0500, David Kelly wrote: > I'm running a script in 0.94.1 that generates a large log file. The script > only runs 41 tasks and finishes in under 5 minutes, but the log file is 2.7 > gigabytes. > > I have: > provenance.log=false > tracing.enabled=false > > I've tried "-minimal.logging" and "-reduced.logging" but it doesn't make > much of a difference. > > What else can I do to reduce the log file size? I'm running Swift from a > module so I can't easily edit any swift-wide log4j settings. > > The log is at > http://web.ci.uchicago.edu/~davidk/logs/combine-20140403-2122-csdnssc1.log.gz(17mb > compressed, 2.7 gigabytes uncompressed) > > Thanks, > David > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From yadudoc1729 at gmail.com Fri Apr 4 12:08:50 2014 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Fri, 4 Apr 2014 12:08:50 -0500 Subject: [Swift-devel] New built-in system, request for comments Message-ID: Hi, I have just added a new builtin "system" to trunk. It takes a string, which is executed in the shell by java and the results are returned as an array. This is done to make mappings easier, for example, you can map an entire folder "foo" using the following : file folder[ ] < array_mapper; files = system ("find foo -type f") >; If the execution of the string provided fails, the logger would report the the exitcode and stderr to the swift stdout. Thanks, Yadu Nand B From hategan at mcs.anl.gov Fri Apr 4 17:47:17 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Fri, 4 Apr 2014 15:47:17 -0700 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: Message-ID: <1396651637.16842.11.camel@echo> I think that from a language design perspective there two important questions to ask: 1. Can you reasonably achieve the same with existing swift features? 2. Where else would system() be useful? Mihael On Fri, 2014-04-04 at 12:08 -0500, Yadu Nand wrote: > Hi, > > I have just added a new builtin "system" to trunk. It takes a string, > which is executed in the shell > by java and the results are returned as an array. This is done to make > mappings easier, for example, > you can map an entire folder "foo" using the following : > > file folder[ ] < array_mapper; files = system ("find foo -type f") >; > > If the execution of the string provided fails, the logger would report > the the exitcode and stderr to > the swift stdout. > > Thanks, > Yadu Nand B > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From tim.g.armstrong at gmail.com Fri Apr 4 20:18:06 2014 From: tim.g.armstrong at gmail.com (Tim Armstrong) Date: Fri, 4 Apr 2014 20:18:06 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <1396651637.16842.11.camel@echo> References: <1396651637.16842.11.camel@echo> Message-ID: I'm curious if there's a difference in Swift/K between how input mappers work and how function outputs work. Ie. could you have an equivalent function system_map() like file folder[]; folder = system_map("find foo -type f"); On Fri, Apr 4, 2014 at 5:47 PM, Mihael Hategan wrote: > I think that from a language design perspective there two important > questions to ask: > > 1. Can you reasonably achieve the same with existing swift features? > 2. Where else would system() be useful? > > Mihael > > On Fri, 2014-04-04 at 12:08 -0500, Yadu Nand wrote: > > Hi, > > > > I have just added a new builtin "system" to trunk. It takes a string, > > which is executed in the shell > > by java and the results are returned as an array. This is done to make > > mappings easier, for example, > > you can map an entire folder "foo" using the following : > > > > file folder[ ] < array_mapper; files = system ("find foo -type f") >; > > > > If the execution of the string provided fails, the logger would report > > the the exitcode and stderr to > > the swift stdout. > > > > Thanks, > > Yadu Nand B > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketan at mcs.anl.gov Fri Apr 4 22:58:27 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Fri, 4 Apr 2014 22:58:27 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: Message-ID: Is it possible to handle stdout and/or stderr of system("command") separately? That way, it might be useful in other places as well. On Fri, Apr 4, 2014 at 12:08 PM, Yadu Nand wrote: > Hi, > > I have just added a new builtin "system" to trunk. It takes a string, > which is executed in the shell > by java and the results are returned as an array. This is done to make > mappings easier, for example, > you can map an entire folder "foo" using the following : > > file folder[ ] < array_mapper; files = system ("find foo -type f") >; > > If the execution of the string provided fails, the logger would report > the the exitcode and stderr to > the swift stdout. > > Thanks, > Yadu Nand B > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Sat Apr 5 02:40:23 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sat, 5 Apr 2014 00:40:23 -0700 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: <1396651637.16842.11.camel@echo> Message-ID: <1396683623.24125.14.camel@echo> It's possible. But there is no notion of input mapper. A mapper is an isomorphism between a sub-set of the variable space and a sub-set of the file name space. It goes both ways. So when a variable is declared with a mapper, that mapper provides the information required to convert between the variable/structure fields and the file system and/or back. The basic advantage of that is that the "mapping" can be procedural, so that would be efficient for arbitrarily large data structures, since it would only require the algorithm to convert between a type path and a fie name (or back). That said, it's entirely possible to implement your example below, and it might actually make things easier. Plus there are other limitations that prevent us from having arbitrarily large data structures, so that argument only goes so far. Mihael On Fri, 2014-04-04 at 20:18 -0500, Tim Armstrong wrote: > I'm curious if there's a difference in Swift/K between how input mappers > work and how function outputs work. Ie. could you have an equivalent > function system_map() like > > file folder[]; > folder = system_map("find foo -type f"); > > > On Fri, Apr 4, 2014 at 5:47 PM, Mihael Hategan wrote: > > > I think that from a language design perspective there two important > > questions to ask: > > > > 1. Can you reasonably achieve the same with existing swift features? > > 2. Where else would system() be useful? > > > > Mihael > > > > On Fri, 2014-04-04 at 12:08 -0500, Yadu Nand wrote: > > > Hi, > > > > > > I have just added a new builtin "system" to trunk. It takes a string, > > > which is executed in the shell > > > by java and the results are returned as an array. This is done to make > > > mappings easier, for example, > > > you can map an entire folder "foo" using the following : > > > > > > file folder[ ] < array_mapper; files = system ("find foo -type f") >; > > > > > > If the execution of the string provided fails, the logger would report > > > the the exitcode and stderr to > > > the swift stdout. > > > > > > Thanks, > > > Yadu Nand B > > > _______________________________________________ > > > Swift-devel mailing list > > > Swift-devel at ci.uchicago.edu > > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > > _______________________________________________ > > Swift-devel mailing list > > Swift-devel at ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > From wilde at anl.gov Sat Apr 5 09:42:58 2014 From: wilde at anl.gov (Michael Wilde) Date: Sat, 5 Apr 2014 09:42:58 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: Message-ID: <53401672.5070207@anl.gov> Yadu and I discussed something similar (ie getting a separate return code as well). I agree, that would be good. Mihael, is it possible for a built-in function to provide multiple returns? How does one go about coding the built-in to behave context-sensitively to its return values? (Eg, I think as does readData?) (Thats more of a question for whoever codes this). Another question this raises is how one would write code that is sensitive to a failing function. This leads to the need for an exit() function that could shut down a swift run. And in the broader context it leads to the need for exception handling. - Mike On 4/4/14, 10:58 PM, Ketan Maheshwari wrote: > Is it possible to handle stdout and/or stderr of system("command") > separately? That way, it might be useful in other places as well. > > > On Fri, Apr 4, 2014 at 12:08 PM, Yadu Nand > wrote: > > Hi, > > I have just added a new builtin "system" to trunk. It takes a string, > which is executed in the shell > by java and the results are returned as an array. This is done to make > mappings easier, for example, > you can map an entire folder "foo" using the following : > > file folder[ ] < array_mapper; files = system ("find foo -type f") >; > > If the execution of the string provided fails, the logger would report > the the exitcode and stderr to > the swift stdout. > > Thanks, > Yadu Nand B > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Sat Apr 5 10:56:45 2014 From: wilde at anl.gov (Michael Wilde) Date: Sat, 5 Apr 2014 10:56:45 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <1396651637.16842.11.camel@echo> References: <1396651637.16842.11.camel@echo> Message-ID: <534027BD.1020500@anl.gov> On 4/4/14, 5:47 PM, Mihael Hategan wrote: > I think that from a language design perspective there two important > questions to ask: > > 1. Can you reasonably achieve the same with existing swift features? No, not "reasonably". We do this to date by calling sh as an app. You need to obtain the cwd first, then pass that to the app to get it to cd to the swift working dir. All told, a lot of code and cognitive dissonance for a simple task. > 2. Where else would system() be useful? Whenever a Swift script needs to perform shell-like operations relative to its current environment. Same reason C and virtually every other language provides a system( ) function. Eg, directory setup, cleanup, etc. But yes, a driving motivation here was to separate the task of finding the set of files that to be mapped from the semantics of mapping, and to provide the arbitrary flexibility to filter such lists that is often needed. And which should not in my opinion be hardwired into mappers. I feel that Swift users would be best served by a very simple set of minimal mapping constructs, that would be used with powerful flexible functions like system( ) and glob( ) to let users do anything they would need or imagine. - Mike From hategan at mcs.anl.gov Sat Apr 5 11:39:44 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sat, 5 Apr 2014 09:39:44 -0700 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <534027BD.1020500@anl.gov> References: <1396651637.16842.11.camel@echo> <534027BD.1020500@anl.gov> Message-ID: <1396715984.29002.8.camel@echo> On Sat, 2014-04-05 at 10:56 -0500, Michael Wilde wrote: > On 4/4/14, 5:47 PM, Mihael Hategan wrote: > > I think that from a language design perspective there two important > > questions to ask: > > > > 1. Can you reasonably achieve the same with existing swift features? > No, not "reasonably". We do this to date by calling sh as an app. You > need to obtain the cwd first, then pass that to the app to get it to cd > to the swift working dir. All told, a lot of code and cognitive > dissonance for a simple task. I see and agree. > > 2. Where else would system() be useful? > Whenever a Swift script needs to perform shell-like operations relative > to its current environment. Same reason C and virtually every other > language provides a system( ) function. Eg, directory setup, cleanup, etc. Right, but I'm not sure many of these make sense in swift. There is no directory setup and there is no cleanup. > > But yes, a driving motivation here was to separate the task of finding > the set of files that to be mapped from the semantics of mapping, and to > provide the arbitrary flexibility to filter such lists that is often > needed. And which should not in my opinion be hardwired into mappers. But then let's address the issue instead of providing an escape to semantics that we were trying to get away from in the first place. > > I feel that Swift users would be best served by a very simple set of > minimal mapping constructs, that would be used with powerful flexible > functions like system( ) and glob( ) to let users do anything they would > need or imagine. I'm worried that this might happen: file out <"out.txt">; out = someApp(); file files[] ; system("rm out.txt"); Which is broken in many ways. Mihael From yadunand at uchicago.edu Sat Apr 5 19:01:45 2014 From: yadunand at uchicago.edu (Yadu Nand Babuji) Date: Sat, 05 Apr 2014 19:01:45 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: Message-ID: <53409969.5080604@uchicago.edu> Ketan, Do we need that ? If you add stdout, stderr and say exitcode as returns, the user has no choice but to define variables to hold those return values. Mihael mentioned in a separate thread that the behavior should be to fail early in case the executed string returns a non-zero exitcode. -Yadu On 04/04/2014 10:58 PM, Ketan Maheshwari wrote: > Is it possible to handle stdout and/or stderr of system("command") > separately? That way, it might be useful in other places as well. > > > On Fri, Apr 4, 2014 at 12:08 PM, Yadu Nand > wrote: > > Hi, > > I have just added a new builtin "system" to trunk. It takes a string, > which is executed in the shell > by java and the results are returned as an array. This is done to make > mappings easier, for example, > you can map an entire folder "foo" using the following : > > file folder[ ] < array_mapper; files = system ("find foo -type f") >; > > If the execution of the string provided fails, the logger would report > the the exitcode and stderr to > the swift stdout. > > Thanks, > Yadu Nand B > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidkelly at uchicago.edu Sat Apr 5 19:58:22 2014 From: davidkelly at uchicago.edu (David Kelly) Date: Sat, 5 Apr 2014 19:58:22 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <53401672.5070207@anl.gov> References: <53401672.5070207@anl.gov> Message-ID: On Sat, Apr 5, 2014 at 9:42 AM, Michael Wilde wrote: Yadu and I discussed something similar (ie getting a separate return code > as well). I agree, that would be good. > How do you get the return code of an app() function in Swift? As far as I know you can't - it either runs successfully (perhaps after retries) or it fails and stops execution. I think system() should follow the same pattern. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketan at mcs.anl.gov Sat Apr 5 19:59:08 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Sat, 5 Apr 2014 19:59:08 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <53409969.5080604@uchicago.edu> References: <53409969.5080604@uchicago.edu> Message-ID: I agree this might not be needed in most cases. I was thinking of commands or external programs that emit both stdout and stderr and say the list of files that needs to be mapped are emitted only on one of the streams, in such case user might want to ignore the other stream. Exitcode in this context is a different issue. For instance, user might have a Swift script that produce user written trace() on stdout while progress log on stderr. In such case, this might be useful: file folder[ ] < array_mapper; files = system ("swift genfiles.swift", 1) >; where 1 would mean only put the stdout into folder array. Thanks, -Ketan On Sat, Apr 5, 2014 at 7:01 PM, Yadu Nand Babuji wrote: > Ketan, > > Do we need that ? If you add stdout, stderr and say exitcode as returns, > the user has no > choice but to define variables to hold those return values. > > Mihael mentioned in a separate thread that the behavior should be to fail > early in case the > executed string returns a non-zero exitcode. > > -Yadu > > > On 04/04/2014 10:58 PM, Ketan Maheshwari wrote: > > Is it possible to handle stdout and/or stderr of system("command") > separately? That way, it might be useful in other places as well. > > > On Fri, Apr 4, 2014 at 12:08 PM, Yadu Nand wrote: > >> Hi, >> >> I have just added a new builtin "system" to trunk. It takes a string, >> which is executed in the shell >> by java and the results are returned as an array. This is done to make >> mappings easier, for example, >> you can map an entire folder "foo" using the following : >> >> file folder[ ] < array_mapper; files = system ("find foo -type f") >; >> >> If the execution of the string provided fails, the logger would report >> the the exitcode and stderr to >> the swift stdout. >> >> Thanks, >> Yadu Nand B >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> > > > > _______________________________________________ > Swift-devel mailing listSwift-devel at ci.uchicago.eduhttps://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Sat Apr 5 20:11:21 2014 From: wilde at anl.gov (Michael Wilde) Date: Sat, 5 Apr 2014 20:11:21 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <53409969.5080604@uchicago.edu> References: <53409969.5080604@uchicago.edu> Message-ID: <5340A9B9.6000303@anl.gov> As I alluded to in a prior post (but didnt say clearly), Im in favor of giving system a varying number of returns, something like: output = system("command"); (output,error) = system("command"); (output,error,rc) = system("command"); Also letting output and error be either strings or string arrays. Not sure how much flexibility is best, though. Regarding treating function failures like app failures: I dont think thats good in the case of functions. Further, in the case of apps, we have at least one user request to provide more flexible error handling for app failures. So when the user requests an rc (from either), any non-zero RC should be returned to the user and not cause the function (app or built-in) to be considered failed. Null files can be returned to preserve dataflow semantics. This is not totally clear, and not clearly "best", but should should consider it. Yadu, regarding the current implementation: I was about to send a note to swift-user when I decided to test it first. Im getting strange errors when I try to do: string o[] = system("echo *.swift"); tracef("o=%s\n",o[0]); # produces: o=*.swift string o[] = system("sh -c 'echo *.swift'"); tracef("o=%s\n",o[0]); # produces: swift:system returned exitcode :1 swift:system stderr: *.swift': -c: line 0: unexpected EOF while looking for matching `''*.swift': -c: line 1: syntax error: unexpected end of file Execution failed: java.lang.IndexOutOfBoundsException: Invalid index [0] for o tracef @ system, line: 28 Caused by: java.lang.IndexOutOfBoundsException: Invalid index [0] for o --- Yet: string o[] = system("sh -c 'date'"); tracef("o=%s\n",o[0]); # produces: o=Sat Apr 5 20:08:29 CDT 2014 So something strange is happening when I try to pass an argument with shell wildcard chars to "sh -c". I think that system( ) by default should send its args to "sh -c" so that wildcards and all related shell features are available. Lastly, note that in above, "echo" is just an experiment; what I really want to do in this example is "/bin/ls -1 *.swift" for example. - Mike On 4/5/14, 7:01 PM, Yadu Nand Babuji wrote: > Ketan, > > Do we need that ? If you add stdout, stderr and say exitcode as > returns, the user has no > choice but to define variables to hold those return values. > > Mihael mentioned in a separate thread that the behavior should be to > fail early in case the > executed string returns a non-zero exitcode. > > -Yadu > > On 04/04/2014 10:58 PM, Ketan Maheshwari wrote: >> Is it possible to handle stdout and/or stderr of system("command") >> separately? That way, it might be useful in other places as well. >> >> >> On Fri, Apr 4, 2014 at 12:08 PM, Yadu Nand > > wrote: >> >> Hi, >> >> I have just added a new builtin "system" to trunk. It takes a string, >> which is executed in the shell >> by java and the results are returned as an array. This is done to >> make >> mappings easier, for example, >> you can map an entire folder "foo" using the following : >> >> file folder[ ] < array_mapper; files = system ("find foo -type f") >; >> >> If the execution of the string provided fails, the logger would >> report >> the the exitcode and stderr to >> the swift stdout. >> >> Thanks, >> Yadu Nand B >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> >> >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Sat Apr 5 20:47:48 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sat, 5 Apr 2014 18:47:48 -0700 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: <53409969.5080604@uchicago.edu> Message-ID: <1396748868.17484.0.camel@echo> On Sat, 2014-04-05 at 19:59 -0500, Ketan Maheshwari wrote: > I agree this might not be needed in most cases. I was thinking of commands > or external programs that emit both stdout and stderr and say the list of > files that needs to be mapped are emitted only on one of the streams, in > such case user might want to ignore the other stream. Exitcode in this > context is a different issue. > > For instance, user might have a Swift script that produce user written > trace() on stdout while progress log on stderr. In such case, this might be > useful: > > file folder[ ] < array_mapper; files = system ("swift genfiles.swift", 1) >; If we're trying to simulate shell commands, why re-invent the wheel and not use system("swift genfiles.swift 2>/dev/null")? From hategan at mcs.anl.gov Sat Apr 5 20:56:03 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sat, 5 Apr 2014 18:56:03 -0700 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <5340A9B9.6000303@anl.gov> References: <53409969.5080604@uchicago.edu> <5340A9B9.6000303@anl.gov> Message-ID: <1396749363.17484.8.camel@echo> On Sat, 2014-04-05 at 20:11 -0500, Michael Wilde wrote: > Regarding treating function failures like app failures: I dont think > thats good in the case of functions. I am of the relatively strong opinion that we should not make it easy to overlook errors. The c style errno or the bash style $? are not what I consider in line with best language design practices, and most languages past 1990-ish have dealt away with that scheme. The reason is that it is very rare that an ignored error is intentional. It is more likely the result of a problem that should be caught sooner rather than later. I do though think that we should have exceptions. I believe they are the clean way to address this issue. Also, I would like to note that all current functions (and pretty much everything in swift) fail on failure instead of returning some magic value. System() would be the only exception. Mihael From yadunand at uchicago.edu Sat Apr 5 22:51:24 2014 From: yadunand at uchicago.edu (Yadu Nand Babuji) Date: Sat, 05 Apr 2014 22:51:24 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <5340A9B9.6000303@anl.gov> References: <53409969.5080604@uchicago.edu> <5340A9B9.6000303@anl.gov> Message-ID: <5340CF3C.2090006@uchicago.edu> I've committed the change for running the strings under bash, and wildcards and pipes are working. I can now do bash one-liners neatly and I like that. I tested with this : string o[] = system("ls *.swift"); foreach t,i in o{ tracef("o=%s\n",o[i]); } RunID: run021 Progress: Sat, 05 Apr 2014 22:49:15-0500 o=fail.swift o=system.swift o=shell.swift o=t.swift Final status:Sat, 05 Apr 2014 22:49:15-0500 while system("echo *.swift"); returns a single line which goes to tracef. Like this : RunID: run022 Progress: Sat, 05 Apr 2014 22:50:12-0500 o=fail.swift shell.swift system.swift t.swift Final status:Sat, 05 Apr 2014 22:50:12-0500 -Yadu On 04/05/2014 08:11 PM, Michael Wilde wrote: > > As I alluded to in a prior post (but didnt say clearly), Im in favor > of giving system a varying number of returns, something like: > > output = system("command"); > (output,error) = system("command"); > (output,error,rc) = system("command"); > > Also letting output and error be either strings or string arrays. > > Not sure how much flexibility is best, though. > > Regarding treating function failures like app failures: I dont think > thats good in the case of functions. Further, in the case of apps, we > have at least one user request to provide more flexible error handling > for app failures. So when the user requests an rc (from either), any > non-zero RC should be returned to the user and not cause the function > (app or built-in) to be considered failed. Null files can be returned > to preserve dataflow semantics. This is not totally clear, and not > clearly "best", but should should consider it. > > Yadu, regarding the current implementation: I was about to send a note > to swift-user when I decided to test it first. Im getting strange > errors when I try to do: > > string o[] = system("echo *.swift"); > tracef("o=%s\n",o[0]); > # produces: > o=*.swift > > string o[] = system("sh -c 'echo *.swift'"); > tracef("o=%s\n",o[0]); > > # produces: > swift:system returned exitcode :1 > swift:system stderr: > *.swift': -c: line 0: unexpected EOF while looking for matching > `''*.swift': -c: line 1: syntax error: unexpected end of file > > Execution failed: > java.lang.IndexOutOfBoundsException: Invalid index [0] for o > tracef @ system, line: 28 > Caused by: java.lang.IndexOutOfBoundsException: Invalid index [0] for o > > --- > > Yet: > string o[] = system("sh -c 'date'"); > tracef("o=%s\n",o[0]); > # produces: > o=Sat Apr 5 20:08:29 CDT 2014 > > So something strange is happening when I try to pass an argument with > shell wildcard chars to "sh -c". > > I think that system( ) by default should send its args to "sh -c" so > that wildcards and all related shell features are available. > > Lastly, note that in above, "echo" is just an experiment; what I > really want to do in this example is "/bin/ls -1 *.swift" for example. > > - Mike > > > On 4/5/14, 7:01 PM, Yadu Nand Babuji wrote: >> Ketan, >> >> Do we need that ? If you add stdout, stderr and say exitcode as >> returns, the user has no >> choice but to define variables to hold those return values. >> >> Mihael mentioned in a separate thread that the behavior should be to >> fail early in case the >> executed string returns a non-zero exitcode. >> >> -Yadu >> >> On 04/04/2014 10:58 PM, Ketan Maheshwari wrote: >>> Is it possible to handle stdout and/or stderr of system("command") >>> separately? That way, it might be useful in other places as well. >>> >>> >>> On Fri, Apr 4, 2014 at 12:08 PM, Yadu Nand >> > wrote: >>> >>> Hi, >>> >>> I have just added a new builtin "system" to trunk. It takes a >>> string, >>> which is executed in the shell >>> by java and the results are returned as an array. This is done >>> to make >>> mappings easier, for example, >>> you can map an entire folder "foo" using the following : >>> >>> file folder[ ] < array_mapper; files = system ("find foo -type >>> f") >; >>> >>> If the execution of the string provided fails, the logger would >>> report >>> the the exitcode and stderr to >>> the swift stdout. >>> >>> Thanks, >>> Yadu Nand B >>> _______________________________________________ >>> Swift-devel mailing list >>> Swift-devel at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>> >>> >>> >>> >>> _______________________________________________ >>> Swift-devel mailing list >>> Swift-devel at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -- > Michael Wilde > Mathematics and Computer Science Computation Institute > Argonne National Laboratory The University of Chicago > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Sat Apr 5 23:54:45 2014 From: wilde at anl.gov (Michael Wilde) Date: Sat, 5 Apr 2014 23:54:45 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: <5340CF3C.2090006@uchicago.edu> References: <53409969.5080604@uchicago.edu> <5340A9B9.6000303@anl.gov> <5340CF3C.2090006@uchicago.edu> Message-ID: <5340DE15.2060206@anl.gov> Nice, thanks, Yadu - this works great. I see that "{ }" in the command line dont work yet - they are flagged as invalid in the kml file. But that's not a system( ) problem: you can't even put these characters in a Swift string literal ! However "[a-d].data" passes through the kml fine. - Mike type file; (string f[ ]) glob (string pat) { f = system(strcat("/bin/ls -1 ",pat)); } file dir[ ] ; foreach f in dir { tracef("file: %s\n", filename(f)); } # gives: file: data3.txt file: data.txt file: data4.txt file: f4.dat file: f1.dat file: data1.txt file: f3.dat file: data2.txt file: f2.dat On 4/5/14, 10:51 PM, Yadu Nand Babuji wrote: > I've committed the change for running the strings under bash, and > wildcards > and pipes are working. I can now do bash one-liners neatly and I like > that. > > I tested with this : > string o[] = system("ls *.swift"); > foreach t,i in o{ > tracef("o=%s\n",o[i]); > } > > RunID: run021 > Progress: Sat, 05 Apr 2014 22:49:15-0500 > o=fail.swift > o=system.swift > o=shell.swift > o=t.swift > Final status:Sat, 05 Apr 2014 22:49:15-0500 > > while system("echo *.swift"); returns a single line which goes to tracef. > Like this : > RunID: run022 > Progress: Sat, 05 Apr 2014 22:50:12-0500 > o=fail.swift shell.swift system.swift t.swift > Final status:Sat, 05 Apr 2014 22:50:12-0500 > > -Yadu > > On 04/05/2014 08:11 PM, Michael Wilde wrote: >> >> As I alluded to in a prior post (but didnt say clearly), Im in favor >> of giving system a varying number of returns, something like: >> >> output = system("command"); >> (output,error) = system("command"); >> (output,error,rc) = system("command"); >> >> Also letting output and error be either strings or string arrays. >> >> Not sure how much flexibility is best, though. >> >> Regarding treating function failures like app failures: I dont think >> thats good in the case of functions. Further, in the case of apps, >> we have at least one user request to provide more flexible error >> handling for app failures. So when the user requests an rc (from >> either), any non-zero RC should be returned to the user and not cause >> the function (app or built-in) to be considered failed. Null files >> can be returned to preserve dataflow semantics. This is not totally >> clear, and not clearly "best", but should should consider it. >> >> Yadu, regarding the current implementation: I was about to send a >> note to swift-user when I decided to test it first. Im getting >> strange errors when I try to do: >> >> string o[] = system("echo *.swift"); >> tracef("o=%s\n",o[0]); >> # produces: >> o=*.swift >> >> string o[] = system("sh -c 'echo *.swift'"); >> tracef("o=%s\n",o[0]); >> >> # produces: >> swift:system returned exitcode :1 >> swift:system stderr: >> *.swift': -c: line 0: unexpected EOF while looking for matching >> `''*.swift': -c: line 1: syntax error: unexpected end of file >> >> Execution failed: >> java.lang.IndexOutOfBoundsException: Invalid index [0] for o >> tracef @ system, line: 28 >> Caused by: java.lang.IndexOutOfBoundsException: Invalid index [0] for o >> >> --- >> >> Yet: >> string o[] = system("sh -c 'date'"); >> tracef("o=%s\n",o[0]); >> # produces: >> o=Sat Apr 5 20:08:29 CDT 2014 >> >> So something strange is happening when I try to pass an argument with >> shell wildcard chars to "sh -c". >> >> I think that system( ) by default should send its args to "sh -c" so >> that wildcards and all related shell features are available. >> >> Lastly, note that in above, "echo" is just an experiment; what I >> really want to do in this example is "/bin/ls -1 *.swift" for example. >> >> - Mike >> >> >> On 4/5/14, 7:01 PM, Yadu Nand Babuji wrote: >>> Ketan, >>> >>> Do we need that ? If you add stdout, stderr and say exitcode as >>> returns, the user has no >>> choice but to define variables to hold those return values. >>> >>> Mihael mentioned in a separate thread that the behavior should be to >>> fail early in case the >>> executed string returns a non-zero exitcode. >>> >>> -Yadu >>> >>> On 04/04/2014 10:58 PM, Ketan Maheshwari wrote: >>>> Is it possible to handle stdout and/or stderr of system("command") >>>> separately? That way, it might be useful in other places as well. >>>> >>>> >>>> On Fri, Apr 4, 2014 at 12:08 PM, Yadu Nand >>> > wrote: >>>> >>>> Hi, >>>> >>>> I have just added a new builtin "system" to trunk. It takes a >>>> string, >>>> which is executed in the shell >>>> by java and the results are returned as an array. This is done >>>> to make >>>> mappings easier, for example, >>>> you can map an entire folder "foo" using the following : >>>> >>>> file folder[ ] < array_mapper; files = system ("find foo -type >>>> f") >; >>>> >>>> If the execution of the string provided fails, the logger would >>>> report >>>> the the exitcode and stderr to >>>> the swift stdout. >>>> >>>> Thanks, >>>> Yadu Nand B >>>> _______________________________________________ >>>> Swift-devel mailing list >>>> Swift-devel at ci.uchicago.edu >>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Swift-devel mailing list >>>> Swift-devel at ci.uchicago.edu >>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >>> >>> >>> >>> _______________________________________________ >>> Swift-devel mailing list >>> Swift-devel at ci.uchicago.edu >>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel >> >> -- >> Michael Wilde >> Mathematics and Computer Science Computation Institute >> Argonne National Laboratory The University of Chicago >> >> >> _______________________________________________ >> Swift-devel mailing list >> Swift-devel at ci.uchicago.edu >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketan at mcs.anl.gov Sun Apr 6 13:30:31 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Sun, 6 Apr 2014 13:30:31 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: <53401672.5070207@anl.gov> Message-ID: One difference between app and non-app functions like system() is that the non-app functions will always run on local host whereas the app functions will almost always run on remote and/or compute nodes. It should thus be easier to handle exit conditions for system(). On Sat, Apr 5, 2014 at 7:58 PM, David Kelly wrote: > On Sat, Apr 5, 2014 at 9:42 AM, Michael Wilde wrote: > > Yadu and I discussed something similar (ie getting a separate return code >> as well). I agree, that would be good. >> > > How do you get the return code of an app() function in Swift? As far as I > know you can't - it either runs successfully (perhaps after retries) or it > fails and stops execution. I think system() should follow the same pattern. > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Sun Apr 6 18:20:09 2014 From: wilde at anl.gov (Michael Wilde) Date: Sun, 6 Apr 2014 18:20:09 -0500 Subject: [Swift-devel] Returning float from java( ) no longer works Message-ID: <5341E129.8020205@anl.gov> I'm dusting off some old Swift code and running into errors which seem to stem from changes in type checking and/or casting. This used to work in a prior release (will need to track down which): (float result) pow (float x, float y) { result = java("java.lang.Math", "pow", x, y); } In trunk, it now gives: Swift trunk swift-r7759 cog-r3890 RunID: run007 Could not start execution: Compile error in procedure invocation at line 50: You cannot assign value of type any to a variable of type float (50 is line before result=...) Any thoughts on why? A similar error occurs on string results. I'll file a ticket. Thanks, - Mike From davidkelly at uchicago.edu Sun Apr 6 21:31:42 2014 From: davidkelly at uchicago.edu (David Kelly) Date: Sun, 6 Apr 2014 21:31:42 -0500 Subject: [Swift-devel] Returning float from java( ) no longer works In-Reply-To: <5341E129.8020205@anl.gov> References: <5341E129.8020205@anl.gov> Message-ID: In 0.95, this fails: --- type file; (float result) pow (float x, float y) { result = java("java.lang.Math", "pow", x, y); } float f = pow(2.0, 2.0); tracef("%f\n", f); --- But when I replace java() with @java() it works for me... On Sun, Apr 6, 2014 at 6:20 PM, Michael Wilde wrote: > I'm dusting off some old Swift code and running into errors which seem > to stem from changes in type checking and/or casting. > > This used to work in a prior release (will need to track down which): > > (float result) pow (float x, float y) > { > result = java("java.lang.Math", "pow", x, y); > } > > In trunk, it now gives: > > Swift trunk swift-r7759 cog-r3890 > RunID: run007 > Could not start execution: > Compile error in procedure invocation at line 50: > You cannot assign value of type any to a variable of type float > > (50 is line before result=...) > > Any thoughts on why? > > A similar error occurs on string results. > > I'll file a ticket. > > Thanks, > > - Mike > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketan at mcs.anl.gov Mon Apr 7 12:45:52 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Mon, 7 Apr 2014 12:45:52 -0500 Subject: [Swift-devel] New built-in system, request for comments In-Reply-To: References: <53409969.5080604@uchicago.edu> Message-ID: Thanks, that makes sense. On a related note, currently, Swift sends version and runid to stdout and progress log to stderr, I think it would be better if everything Swift produces go to stderr and only user specified output go to stdout. On Sat, Apr 5, 2014 at 8:47 PM, Hategan-Marandiuc, Philip M. < hategan at mcs.anl.gov> wrote: > On Sat, 2014-04-05 at 19:59 -0500, Ketan Maheshwari wrote: > > I agree this might not be needed in most cases. I was thinking of > commands > > or external programs that emit both stdout and stderr and say the list of > > files that needs to be mapped are emitted only on one of the streams, in > > such case user might want to ignore the other stream. Exitcode in this > > context is a different issue. > > > > For instance, user might have a Swift script that produce user written > > trace() on stdout while progress log on stderr. In such case, this might > be > > useful: > > > > file folder[ ] < array_mapper; files = system ("swift genfiles.swift", > 1) >; > > If we're trying to simulate shell commands, why re-invent the wheel and > not use system("swift genfiles.swift 2>/dev/null")? > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Mon Apr 7 21:48:17 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Mon, 7 Apr 2014 19:48:17 -0700 Subject: [Swift-devel] me have flu... Message-ID: <1396925297.10042.0.camel@echo> ... will be back in business shortly. Mihael From yadunand at uchicago.edu Wed Apr 9 16:06:19 2014 From: yadunand at uchicago.edu (Yadu Nand Babuji) Date: Wed, 09 Apr 2014 16:06:19 -0500 Subject: [Swift-devel] Swift cloud tutorial not working with Swift 0.95 In-Reply-To: References: Message-ID: <5345B64B.60207@uchicago.edu> Hi, Anthony reports that the Swift cloud tutorial is not working with Swift-0.95. I tested the simple ones which use the local provider as well as the ones which use a coaster service and both seem to be failing. A few logs from part04 which fail are attached. The tutorial works fine with 0.94. Anthony needs 0.95 for the new log format so that he'd be able to extract run related information. Thanks, Yadu -------- Original Message -------- Subject: Log Date: Wed, 9 Apr 2014 15:53:46 -0500 From: Anthony SIMONET To: Yadu Nand Babuji -- Anthony SIMONET LIP - ENS Lyon Site Jaques Monod (Bureau 311 Sud) 46 All?e d'Italie, 69364 Lyon Cedex 07 - France Tel: +33 (0) 4 26 23 39 36 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: swift-tutorial.tar.gz Type: application/x-gzip Size: 6259 bytes Desc: not available URL: From anthony.simonet at inria.fr Thu Apr 10 15:44:08 2014 From: anthony.simonet at inria.fr (Anthony SIMONET) Date: Thu, 10 Apr 2014 15:44:08 -0500 Subject: [Swift-devel] Bug in start-coaster-service Message-ID: Hi everyone, I think I found a tiny bug in 0.95. 'start-coaster-service' will fail on OSX because 'mktemp' is called with no argument. While this is fine on most Linux, either a prefix or a pattern is required on OSX. I've attached a sample output from my machine and a patch demonstrating how I fixed it. Cheers, -- Anthony SIMONET LIP - ENS Lyon Site Jaques Monod (Bureau 311 Sud) 46 All?e d'Italie, 69364 Lyon Cedex 07 - France Tel: +33 (0) 4 26 23 39 36 -------------- next part -------------- A non-text attachment was scrubbed... Name: start-coaster-service.patch Type: application/octet-stream Size: 639 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: start-coaster-service.out Type: application/octet-stream Size: 2903 bytes Desc: not available URL: From benc at hawaga.org.uk Thu Apr 10 19:00:52 2014 From: benc at hawaga.org.uk (Ben Clifford) Date: Fri, 11 Apr 2014 00:00:52 +0000 (UTC) Subject: [Swift-devel] CREAM-CE support In-Reply-To: References: Message-ID: (Bruce was someone I worked with on my various visits to Johannesburg previously, as part of OSG outreach work. Hi Bruce! We did some hacking before with 2009-era Swift to get stuff running. The big deal, I think, was file staging in a way that worked the way the EMI stack wanted it - I think some of the way the coasters/provider staging work has gone would make this easier. Based on my occasional reading of this list and not much else. Good luck.) On Thu, 31 Oct 2013, Bruce Becker wrote: > Hi Swift-devel > I've got some users in Johannesburg who are very keen on swift, and have been using it for a while. However, > they cannot use the national grid infrastructure in South Africa, which is EMI-3 based. All of our clusters > expose CREAM Computing Elements. I would like to hack something together to > 1) allow swift to use VO SAGrid proxies for authentication and authorisation on our sites > 2) create a CREAM provider so that swift can understand how to submit to CREAM CE's. Either via the command > line or via the API. > > Where should I start ?? > Thanks !? > Bruce > > -- > Dr. Bruce Becker, > Coordinator, South African National Grid > Meraka Institute, CSIR > Meiring Naude Road > Pretoria 0001 South Africa > +27 12 841 3746 (w) | +27 12 841-4829 (f) | > +27 84 989 6169 (m)? | + 39 392 622 9279 > AIM/GTalk/Skype/MSN/Jabber : brucellino > http://www.sagrid.ac.za > > From wilde at anl.gov Fri Apr 11 09:19:38 2014 From: wilde at anl.gov (Michael Wilde) Date: Fri, 11 Apr 2014 09:19:38 -0500 Subject: [Swift-devel] Bug in start-coaster-service In-Reply-To: References: Message-ID: <5347F9FA.4030701@anl.gov> Thanks, Antony. Yadu, could you integrate Antony's fix? Regards, - Mike On 4/10/14, 3:44 PM, Anthony SIMONET wrote: > Hi everyone, > > I think I found a tiny bug in 0.95. > 'start-coaster-service' will fail on OSX because 'mktemp' is called > with no argument. > While this is fine on most Linux, either a prefix or a pattern is > required on OSX. > > I've attached a sample output from my machine and a patch > demonstrating how I fixed it. > > Cheers, > -- > > Anthony SIMONET > LIP - ENS Lyon > Site Jaques Monod (Bureau 311 Sud) > 46 All?e d'Italie, 69364 Lyon Cedex 07 - France > Tel: +33 (0) 4 26 23 39 36 > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From foster at anl.gov Fri Apr 11 09:22:02 2014 From: foster at anl.gov (Ian Foster) Date: Fri, 11 Apr 2014 09:22:02 -0500 Subject: [Swift-devel] White papers In-Reply-To: <9AB7792F032F864D93FEF34EF398060424D630CC@CFWEX01.americas.cray.com> References: <9AB7792F032F864D93FEF34EF398060424D630CC@CFWEX01.americas.cray.com> Message-ID: <5512F664-07DF-4460-94B7-4247A7A757A9@anl.gov> Dear Carlos: See www.swift-lang.org. Regards -- Ian. On Dec 23, 2013, at 8:54 PM, Carlos P. Sosa wrote: > Hello > > Are there any white papers that describe swift? > > Thanks > > carlos > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadudoc1729 at gmail.com Fri Apr 11 09:23:27 2014 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Fri, 11 Apr 2014 09:23:27 -0500 Subject: [Swift-devel] Bug in start-coaster-service In-Reply-To: <5347F9FA.4030701@anl.gov> References: <5347F9FA.4030701@anl.gov> Message-ID: Sure, I could test and merge Anthony's fix. Anthony, thanks for the fix. :) On Fri, Apr 11, 2014 at 9:19 AM, Michael Wilde wrote: > Thanks, Antony. Yadu, could you integrate Antony's fix? > > Regards, > > - Mike > > > On 4/10/14, 3:44 PM, Anthony SIMONET wrote: > > Hi everyone, > > I think I found a tiny bug in 0.95. > 'start-coaster-service' will fail on OSX because 'mktemp' is called with > no argument. > While this is fine on most Linux, either a prefix or a pattern is required > on OSX. > > I've attached a sample output from my machine and a patch demonstrating > how I fixed it. > > Cheers, > -- > > Anthony SIMONET > LIP - ENS Lyon > Site Jaques Monod (Bureau 311 Sud) > 46 All?e d'Italie, 69364 Lyon Cedex 07 - France > Tel: +33 (0) 4 26 23 39 36 > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > > -- > Michael Wilde > Mathematics and Computer Science Computation Institute > Argonne National Laboratory The University of Chicago > > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Sat Apr 12 13:17:46 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Sat, 12 Apr 2014 11:17:46 -0700 Subject: [Swift-devel] svn down? Message-ID: <1397326666.30255.1.camel@echo> Both svn and trac seem down. Is anybody else seeing this? Mihael From skrieder at iit.edu Sat Apr 12 13:26:06 2014 From: skrieder at iit.edu (Scott Krieder) Date: Sat, 12 Apr 2014 13:26:06 -0500 Subject: [Swift-devel] svn down? In-Reply-To: <1397326666.30255.1.camel@echo> References: <1397326666.30255.1.camel@echo> Message-ID: Yes, I'm unable to load trac. On Sat, Apr 12, 2014 at 1:17 PM, Mihael Hategan wrote: > Both svn and trac seem down. Is anybody else seeing this? > > Mihael > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -- Scott J. Krieder C: 419-685-0410 E: skrieder at iit.edu http://datasys.cs.iit.edu/~skrieder/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From yadunand at uchicago.edu Sat Apr 12 13:28:15 2014 From: yadunand at uchicago.edu (Yadu Nand Babuji) Date: Sat, 12 Apr 2014 13:28:15 -0500 Subject: [Swift-devel] svn down? In-Reply-To: <1397326666.30255.1.camel@echo> References: <1397326666.30255.1.camel@echo> Message-ID: <534985BF.7050305@uchicago.edu> I can confirm. This happened atleast twice this week for me (svn). -Yadu On 04/12/2014 01:17 PM, Mihael Hategan wrote: > Both svn and trac seem down. Is anybody else seeing this? > > Mihael > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel From ketan at mcs.anl.gov Sun Apr 13 17:23:57 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Sun, 13 Apr 2014 17:23:57 -0500 Subject: [Swift-devel] Idris Language Message-ID: Might be interesting wrt tryswift: http://www.tryidris.org/console >From the page: "Idris is a language for practical, dependently typed, functional programming." Thanks, Ketan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ketan at mcs.anl.gov Sun Apr 13 17:24:47 2014 From: ketan at mcs.anl.gov (Ketan Maheshwari) Date: Sun, 13 Apr 2014 17:24:47 -0500 Subject: [Swift-devel] Idris Language In-Reply-To: References: Message-ID: Main page is: http://www.idris-lang.org On Sun, Apr 13, 2014 at 5:23 PM, Ketan Maheshwari wrote: > Might be interesting wrt tryswift: > > http://www.tryidris.org/console > > From the page: > > "Idris is a language for practical, dependently typed, functional > programming." > > Thanks, > Ketan > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Tue Apr 15 15:56:02 2014 From: wilde at anl.gov (Michael Wilde) Date: Tue, 15 Apr 2014 15:56:02 -0500 Subject: [Swift-devel] Tryswift enhancements In-Reply-To: References: Message-ID: <534D9CE2.3040703@anl.gov> [Moving discussion to swift-devel] On 4/15/14, 12:06 PM, David Kelly wrote: > Hi Mike, > > One idea I had for tryswift was to change the way that file outputs > are displayed. Keeping them in a drop down list saves some space and I > think makes it look a little cleaner. To see what I mean, check out > http://swiftvm4.ci.uchicago.edu and try the foreach example (still > working on some CSS changes there as well). > > Right now it only handles html/text, but if the file output was images > or videos, they could be loaded in the same window. Just wanted to get > your initial thoughts before I did too much with it. Hi David, I think this is heading in a nice direction. It looks very nice and clean. Some thoughts: Top window: - Add text in front of menu: Example: [example-menu] [Explain] - [Explain] button should pop out in a new window, sized and placed "nicely" (not quite sure what nicely involves yet ;) Bottom window - its hard to tell when the script completes or how many files it produced - when you view a file it overwrites the progress window - (reset is broken, btw) How about: - dont show the output-files menu while the script is running - when script completes, show the menu with a label [N files produced:] [output-files-menu] - when you select a file, show it in a new popup window "placed nicely" with a [close] button top and bottom - allowing that window to be multi-media sounds nice Bigger project: - for status, show running tally of the status to the right of the control buttons - add button [Progress log] that shows the progress to date or after the fact, as a pop-up. With cleaner timestamp etc. WHich we should make the default for all Swift runs. So with this improved progress menu, you'd have a very clean look with the Swift text taking most of the window, bracketd by a script-selector stripe at the top and an execution control strip at the bottom. How do these sound to you? - Mike From davidkelly at uchicago.edu Tue Apr 15 16:30:46 2014 From: davidkelly at uchicago.edu (David Kelly) Date: Tue, 15 Apr 2014 16:30:46 -0500 Subject: [Swift-devel] Tryswift enhancements In-Reply-To: <534D9CE2.3040703@anl.gov> References: <534D9CE2.3040703@anl.gov> Message-ID: These sound pretty good. I'll implement most of these and get it to a more polished/styled state soon, just wanted to get some initial feedback if I was going in the right direction or not. Thanks! David On Tue, Apr 15, 2014 at 3:56 PM, Michael Wilde wrote: > [Moving discussion to swift-devel] > > On 4/15/14, 12:06 PM, David Kelly wrote: > >> Hi Mike, >> >> One idea I had for tryswift was to change the way that file outputs are >> displayed. Keeping them in a drop down list saves some space and I think >> makes it look a little cleaner. To see what I mean, check out >> http://swiftvm4.ci.uchicago.edu and try the foreach example (still >> working on some CSS changes there as well). >> >> Right now it only handles html/text, but if the file output was images or >> videos, they could be loaded in the same window. Just wanted to get your >> initial thoughts before I did too much with it. >> > Hi David, > > I think this is heading in a nice direction. It looks very nice and clean. > > Some thoughts: > > Top window: > - Add text in front of menu: Example: [example-menu] [Explain] > - [Explain] button should pop out in a new window, sized and placed > "nicely" (not quite sure what nicely involves yet ;) > > Bottom window > - its hard to tell when the script completes or how many files it produced > - when you view a file it overwrites the progress window > - (reset is broken, btw) > > How about: > - dont show the output-files menu while the script is running > - when script completes, show the menu with a label [N files produced:] > [output-files-menu] > - when you select a file, show it in a new popup window "placed nicely" > with a [close] button top and bottom > - allowing that window to be multi-media sounds nice > > Bigger project: > - for status, show running tally of the status to the right of the control > buttons > - add button [Progress log] that shows the progress to date or after the > fact, as a pop-up. With cleaner timestamp etc. WHich we should make the > default for all Swift runs. > > So with this improved progress menu, you'd have a very clean look with the > Swift text taking most of the window, bracketd by a script-selector stripe > at the top and an execution control strip at the bottom. > > How do these sound to you? > > - Mike > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Wed Apr 16 10:38:29 2014 From: wilde at anl.gov (Michael Wilde) Date: Wed, 16 Apr 2014 10:38:29 -0500 Subject: [Swift-devel] Tryswift enhancements In-Reply-To: <534E07C7.7040106@anl.gov> References: <534D9CE2.3040703@anl.gov> <534E07C7.7040106@anl.gov> Message-ID: <534EA3F5.7050408@anl.gov> For the progress output: Swift trunk swift-r7139 cog-r3806 RunID: 20140416-1029-alqxvvi1 Progress: time: Wed, 16 Apr 2014 10:29:54 -0500 Progress: time: Wed, 16 Apr 2014 10:29:55 -0500 Selecting site:7 Active:3 Progress: time: Wed, 16 Apr 2014 10:29:56 -0500 Selecting site:7 Active:3 Finished successfully:3 Progress: time: Wed, 16 Apr 2014 10:29:57 -0500 Selecting site:7 Active:3 Finished successfully:6 Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 - start with Swift run starting at 10:23:03 - suppress Swift version # and RunID lines - change: Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 to: 10:29:58 Ready:7 Active:3 Done:9 [Failed: 3] (show failures only if non-zero) Use spaces to make these line up "nicely" - add a line saying Swift run completed at 10:24:59 Mike On 4/15/14, 11:32 PM, Michael Wilde wrote: > David, just to clarify: by the [Explain] notation below I just meant > to make "Explain" a "button". > > - Mike > > On 4/15/14, 3:56 PM, Michael Wilde wrote: >> [Moving discussion to swift-devel] >> >> On 4/15/14, 12:06 PM, David Kelly wrote: >>> Hi Mike, >>> >>> One idea I had for tryswift was to change the way that file outputs >>> are displayed. Keeping them in a drop down list saves some space and >>> I think makes it look a little cleaner. To see what I mean, check >>> out http://swiftvm4.ci.uchicago.edu and try the foreach example >>> (still working on some CSS changes there as well). >>> >>> Right now it only handles html/text, but if the file output was >>> images or videos, they could be loaded in the same window. Just >>> wanted to get your initial thoughts before I did too much with it. >> Hi David, >> >> I think this is heading in a nice direction. It looks very nice and >> clean. >> >> Some thoughts: >> >> Top window: >> - Add text in front of menu: Example: [example-menu] [Explain] >> - [Explain] button should pop out in a new window, sized and placed >> "nicely" (not quite sure what nicely involves yet ;) >> >> Bottom window >> - its hard to tell when the script completes or how many files it >> produced >> - when you view a file it overwrites the progress window >> - (reset is broken, btw) >> >> How about: >> - dont show the output-files menu while the script is running >> - when script completes, show the menu with a label [N files >> produced:] [output-files-menu] >> - when you select a file, show it in a new popup window "placed >> nicely" with a [close] button top and bottom >> - allowing that window to be multi-media sounds nice >> >> Bigger project: >> - for status, show running tally of the status to the right of the >> control buttons >> - add button [Progress log] that shows the progress to date or after >> the fact, as a pop-up. With cleaner timestamp etc. WHich we should >> make the default for all Swift runs. >> >> So with this improved progress menu, you'd have a very clean look >> with the Swift text taking most of the window, bracketd by a >> script-selector stripe at the top and an execution control strip at >> the bottom. >> >> How do these sound to you? >> >> - Mike >> > -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidkelly at uchicago.edu Wed Apr 16 15:10:23 2014 From: davidkelly at uchicago.edu (David Kelly) Date: Wed, 16 Apr 2014 15:10:23 -0500 Subject: [Swift-devel] Tryswift enhancements In-Reply-To: <534EA3F5.7050408@anl.gov> References: <534D9CE2.3040703@anl.gov> <534E07C7.7040106@anl.gov> <534EA3F5.7050408@anl.gov> Message-ID: Maybe we should take a look at these and see if we should change them in Swift as well? If the ticker naming is not clear in tryswift, it may not clear from the command line either. Here is the list of ticker values I see: Initializing Selecting site Stage in Submitting Submitted Active Stage out Failed Replicating Finished in previous run Finished successfully I don't think most people are interested in many of these, or understand what they mean. I'd be in favor of a cleaner ticker, perhaps with just timestamp, "pending", "active", and "completed" nicely formatted and aligned. On Wed, Apr 16, 2014 at 10:38 AM, Michael Wilde wrote: > For the progress output: > > Swift trunk swift-r7139 cog-r3806 > RunID: 20140416-1029-alqxvvi1 > Progress: time: Wed, 16 Apr 2014 10:29:54 -0500 > Progress: time: Wed, 16 Apr 2014 10:29:55 -0500 Selecting site:7 Active:3 > Progress: time: Wed, 16 Apr 2014 10:29:56 -0500 Selecting site:7 Active:3 Finished successfully:3 > Progress: time: Wed, 16 Apr 2014 10:29:57 -0500 Selecting site:7 Active:3 Finished successfully:6 > Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 > > - start with Swift run starting at 10:23:03 > - suppress Swift version # and RunID lines > - change: Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 > to: 10:29:58 Ready:7 Active:3 Done:9 [Failed: 3] (show failures only if non-zero) > > Use spaces to make these line up "nicely" > > - add a line saying Swift run completed at 10:24:59 > > Mike > > > On 4/15/14, 11:32 PM, Michael Wilde wrote: > > David, just to clarify: by the [Explain] notation below I just meant to > make "Explain" a "button". > > - Mike > > On 4/15/14, 3:56 PM, Michael Wilde wrote: > > [Moving discussion to swift-devel] > > On 4/15/14, 12:06 PM, David Kelly wrote: > > Hi Mike, > > One idea I had for tryswift was to change the way that file outputs are > displayed. Keeping them in a drop down list saves some space and I think > makes it look a little cleaner. To see what I mean, check out > http://swiftvm4.ci.uchicago.edu and try the foreach example (still > working on some CSS changes there as well). > > Right now it only handles html/text, but if the file output was images or > videos, they could be loaded in the same window. Just wanted to get your > initial thoughts before I did too much with it. > > Hi David, > > I think this is heading in a nice direction. It looks very nice and clean. > > Some thoughts: > > Top window: > - Add text in front of menu: Example: [example-menu] [Explain] > - [Explain] button should pop out in a new window, sized and placed > "nicely" (not quite sure what nicely involves yet ;) > > Bottom window > - its hard to tell when the script completes or how many files it produced > - when you view a file it overwrites the progress window > - (reset is broken, btw) > > How about: > - dont show the output-files menu while the script is running > - when script completes, show the menu with a label [N files produced:] > [output-files-menu] > - when you select a file, show it in a new popup window "placed nicely" > with a [close] button top and bottom > - allowing that window to be multi-media sounds nice > > Bigger project: > - for status, show running tally of the status to the right of the control > buttons > - add button [Progress log] that shows the progress to date or after the > fact, as a pop-up. With cleaner timestamp etc. WHich we should make the > default for all Swift runs. > > So with this improved progress menu, you'd have a very clean look with the > Swift text taking most of the window, bracketd by a script-selector stripe > at the top and an execution control strip at the bottom. > > How do these sound to you? > > - Mike > > > > -- > Michael Wilde > Mathematics and Computer Science Computation Institute > Argonne National Laboratory The University of Chicago > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Wed Apr 16 16:56:28 2014 From: wilde at anl.gov (Michael Wilde) Date: Wed, 16 Apr 2014 16:56:28 -0500 Subject: [Swift-devel] Tryswift enhancements In-Reply-To: References: <534D9CE2.3040703@anl.gov> <534E07C7.7040106@anl.gov> <534EA3F5.7050408@anl.gov> Message-ID: <534EFC8C.80904@anl.gov> Indeed - I strongly agree, David. The basic ticker should be simpler, the state-names simpler, and clear precise explanations of their meanings should be documented. I suspect all the states below make sense, but perhaps some could be merged into a smaller simpler set, and the full details only exposed when requested by a property. - Mike On 4/16/14, 3:10 PM, David Kelly wrote: > Maybe we should take a look at these and see if we should change them > in Swift as well? If the ticker naming is not clear in tryswift, it > may not clear from the command line either. Here is the list of ticker > values I see: > > Initializing > Selecting site > Stage in > Submitting > Submitted > Active > Stage out > Failed > Replicating > Finished in previous run > Finished successfully > > I don't think most people are interested in many of these, or > understand what they mean. I'd be in favor of a cleaner ticker, > perhaps with just timestamp, "pending", "active", and "completed" > nicely formatted and aligned. > > > On Wed, Apr 16, 2014 at 10:38 AM, Michael Wilde > wrote: > > For the progress output: > > Swift trunk swift-r7139 cog-r3806 > RunID: 20140416-1029-alqxvvi1 > Progress: time: Wed, 16 Apr 2014 10:29:54 -0500 > Progress: time: Wed, 16 Apr 2014 10:29:55 -0500 Selecting site:7 Active:3 > Progress: time: Wed, 16 Apr 2014 10:29:56 -0500 Selecting site:7 Active:3 Finished successfully:3 > Progress: time: Wed, 16 Apr 2014 10:29:57 -0500 Selecting site:7 Active:3 Finished successfully:6 > Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 > > - start with Swift run starting at 10:23:03 > - suppress Swift version # and RunID lines > - change: Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 > to: 10:29:58 Ready:7 Active:3 Done:9 [Failed: 3] (show failures only if non-zero) > > Use spaces to make these line up "nicely" > > - add a line saying Swift run completed at 10:24:59 > > Mike > > > On 4/15/14, 11:32 PM, Michael Wilde wrote: >> David, just to clarify: by the [Explain] notation below I just >> meant to make "Explain" a "button". >> >> - Mike >> >> On 4/15/14, 3:56 PM, Michael Wilde wrote: >>> [Moving discussion to swift-devel] >>> >>> On 4/15/14, 12:06 PM, David Kelly wrote: >>>> Hi Mike, >>>> >>>> One idea I had for tryswift was to change the way that file >>>> outputs are displayed. Keeping them in a drop down list saves >>>> some space and I think makes it look a little cleaner. To see >>>> what I mean, check out http://swiftvm4.ci.uchicago.edu and try >>>> the foreach example (still working on some CSS changes there as >>>> well). >>>> >>>> Right now it only handles html/text, but if the file output was >>>> images or videos, they could be loaded in the same window. Just >>>> wanted to get your initial thoughts before I did too much with it. >>> Hi David, >>> >>> I think this is heading in a nice direction. It looks very nice >>> and clean. >>> >>> Some thoughts: >>> >>> Top window: >>> - Add text in front of menu: Example: [example-menu] [Explain] >>> - [Explain] button should pop out in a new window, sized and >>> placed "nicely" (not quite sure what nicely involves yet ;) >>> >>> Bottom window >>> - its hard to tell when the script completes or how many files >>> it produced >>> - when you view a file it overwrites the progress window >>> - (reset is broken, btw) >>> >>> How about: >>> - dont show the output-files menu while the script is running >>> - when script completes, show the menu with a label [N files >>> produced:] [output-files-menu] >>> - when you select a file, show it in a new popup window "placed >>> nicely" with a [close] button top and bottom >>> - allowing that window to be multi-media sounds nice >>> >>> Bigger project: >>> - for status, show running tally of the status to the right of >>> the control buttons >>> - add button [Progress log] that shows the progress to date or >>> after the fact, as a pop-up. With cleaner timestamp etc. WHich >>> we should make the default for all Swift runs. >>> >>> So with this improved progress menu, you'd have a very clean >>> look with the Swift text taking most of the window, bracketd by >>> a script-selector stripe at the top and an execution control >>> strip at the bottom. >>> >>> How do these sound to you? >>> >>> - Mike >>> >> > > -- > Michael Wilde > Mathematics and Computer Science Computation Institute > Argonne National Laboratory The University of Chicago > > -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Thu Apr 17 08:35:47 2014 From: wilde at anl.gov (Michael Wilde) Date: Thu, 17 Apr 2014 08:35:47 -0500 Subject: [Swift-devel] Tryswift enhancements In-Reply-To: References: <534D9CE2.3040703@anl.gov> <534E07C7.7040106@anl.gov> <534EA3F5.7050408@anl.gov> Message-ID: <534FD8B3.2030500@anl.gov> I like the current status display of Tryswift, and think it merits thought for the swift command as well: Swift run starting at 08:21:55 08:21:59 08:22:00 Ready:7 Active:3 08:22:10 Ready:7 Active:3 Done:3 08:22:20 Ready:7 Active:3 Done:6 08:22:30 Ready:7 Active:3 Done:9 08:22:40 Ready:5 Active:3 Done:12 08:22:50 Ready:2 Active:3 Done:15 08:23:00 Active:2 Done:18 Final status: 08:23:10 Done:21 Generated 21 file(s) Swift completed I suspect its a matter of continuing to maintain the detailed state as is, but abstracting that to a simpler state (as an option) Doing this well and documenting it well is non trivial, though. - Mike On 4/16/14, 3:10 PM, David Kelly wrote: > Maybe we should take a look at these and see if we should change them > in Swift as well? If the ticker naming is not clear in tryswift, it > may not clear from the command line either. Here is the list of ticker > values I see: > > Initializing > Selecting site > Stage in > Submitting > Submitted > Active > Stage out > Failed > Replicating > Finished in previous run > Finished successfully > > I don't think most people are interested in many of these, or > understand what they mean. I'd be in favor of a cleaner ticker, > perhaps with just timestamp, "pending", "active", and "completed" > nicely formatted and aligned. > > > On Wed, Apr 16, 2014 at 10:38 AM, Michael Wilde > wrote: > > For the progress output: > > Swift trunk swift-r7139 cog-r3806 > RunID: 20140416-1029-alqxvvi1 > Progress: time: Wed, 16 Apr 2014 10:29:54 -0500 > Progress: time: Wed, 16 Apr 2014 10:29:55 -0500 Selecting site:7 Active:3 > Progress: time: Wed, 16 Apr 2014 10:29:56 -0500 Selecting site:7 Active:3 Finished successfully:3 > Progress: time: Wed, 16 Apr 2014 10:29:57 -0500 Selecting site:7 Active:3 Finished successfully:6 > Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 > > - start with Swift run starting at 10:23:03 > - suppress Swift version # and RunID lines > - change: Progress: time: Wed, 16 Apr 2014 10:29:58 -0500 Selecting site:7 Active:3 Finished successfully:9 > to: 10:29:58 Ready:7 Active:3 Done:9 [Failed: 3] (show failures only if non-zero) > > Use spaces to make these line up "nicely" > > - add a line saying Swift run completed at 10:24:59 > > Mike > > > On 4/15/14, 11:32 PM, Michael Wilde wrote: >> David, just to clarify: by the [Explain] notation below I just >> meant to make "Explain" a "button". >> >> - Mike >> >> On 4/15/14, 3:56 PM, Michael Wilde wrote: >>> [Moving discussion to swift-devel] >>> >>> On 4/15/14, 12:06 PM, David Kelly wrote: >>>> Hi Mike, >>>> >>>> One idea I had for tryswift was to change the way that file >>>> outputs are displayed. Keeping them in a drop down list saves >>>> some space and I think makes it look a little cleaner. To see >>>> what I mean, check out http://swiftvm4.ci.uchicago.edu and try >>>> the foreach example (still working on some CSS changes there as >>>> well). >>>> >>>> Right now it only handles html/text, but if the file output was >>>> images or videos, they could be loaded in the same window. Just >>>> wanted to get your initial thoughts before I did too much with it. >>> Hi David, >>> >>> I think this is heading in a nice direction. It looks very nice >>> and clean. >>> >>> Some thoughts: >>> >>> Top window: >>> - Add text in front of menu: Example: [example-menu] [Explain] >>> - [Explain] button should pop out in a new window, sized and >>> placed "nicely" (not quite sure what nicely involves yet ;) >>> >>> Bottom window >>> - its hard to tell when the script completes or how many files >>> it produced >>> - when you view a file it overwrites the progress window >>> - (reset is broken, btw) >>> >>> How about: >>> - dont show the output-files menu while the script is running >>> - when script completes, show the menu with a label [N files >>> produced:] [output-files-menu] >>> - when you select a file, show it in a new popup window "placed >>> nicely" with a [close] button top and bottom >>> - allowing that window to be multi-media sounds nice >>> >>> Bigger project: >>> - for status, show running tally of the status to the right of >>> the control buttons >>> - add button [Progress log] that shows the progress to date or >>> after the fact, as a pop-up. With cleaner timestamp etc. WHich >>> we should make the default for all Swift runs. >>> >>> So with this improved progress menu, you'd have a very clean >>> look with the Swift text taking most of the window, bracketd by >>> a script-selector stripe at the top and an execution control >>> strip at the bottom. >>> >>> How do these sound to you? >>> >>> - Mike >>> >> > > -- > Michael Wilde > Mathematics and Computer Science Computation Institute > Argonne National Laboratory The University of Chicago > > -- Michael Wilde Mathematics and Computer Science Computation Institute Argonne National Laboratory The University of Chicago -------------- next part -------------- An HTML attachment was scrubbed... URL: From wilde at anl.gov Thu Apr 17 08:41:51 2014 From: wilde at anl.gov (Michael Wilde) Date: Thu, 17 Apr 2014 08:41:51 -0500 Subject: [Swift-devel] Tryswift enhancements In-Reply-To: <534FD8B3.2030500@anl.gov> References: <534D9CE2.3040703@anl.gov> <534E07C7.7040106@anl.gov> <534EA3F5.7050408@anl.gov> <534FD8B3.2030500@anl.gov> Message-ID: <534FDA1F.5030605@anl.gov> I see that long outputs overflow the output panel rather than causing it to start scrolling. (see attached image) -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2014-04-17 at 8.41.02 AM.png Type: image/png Size: 226266 bytes Desc: not available URL: From hategan at mcs.anl.gov Thu Apr 17 19:03:31 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Thu, 17 Apr 2014 17:03:31 -0700 Subject: [Swift-devel] what's the cog branch for 0.95? Message-ID: <1397779411.26039.8.camel@echo> I'm assuming 4.1.11, but wanted to double-check. Mihael From davidkelly at uchicago.edu Thu Apr 17 19:17:36 2014 From: davidkelly at uchicago.edu (David Kelly) Date: Thu, 17 Apr 2014 19:17:36 -0500 Subject: [Swift-devel] what's the cog branch for 0.95? In-Reply-To: <1397779411.26039.8.camel@echo> References: <1397779411.26039.8.camel@echo> Message-ID: Yep, it's at https://svn.code.sf.net/p/cogkit/svn/branches/4.1.11/src/cog On Thu, Apr 17, 2014 at 7:03 PM, Mihael Hategan wrote: > I'm assuming 4.1.11, but wanted to double-check. > > Mihael > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wozniak at mcs.anl.gov Wed Apr 23 16:43:54 2014 From: wozniak at mcs.anl.gov (Justin M Wozniak) Date: Wed, 23 Apr 2014 16:43:54 -0500 Subject: [Swift-devel] SSH-CL with port forwarding Message-ID: <5358341A.3020704@mcs.anl.gov> Hi all I'm trying to get SSH-CL going with port forwarding. I think I just need to tell Coasters which port to use for the bootstrap connection. I'm doing: sites.xml: > ssh -L 50001:localhost:50001 grind > export GLOBUS_HOSTNAME=localhost > export GLOBUS_TCP_PORT_RANGE=50000,50002 I either get: Failed to download bootstrap jar from http://localhost:50002 if the ports do not line up or: java.io.IOException: java.io.IOException: No free ports available. if I make the range too restrictive. How should I set this up? I cannot change the firewall rules. Or should I consider doing this with a local coaster service and manual coasters? Thanks Justin -- Justin M Wozniak From yadudoc1729 at gmail.com Wed Apr 23 19:59:22 2014 From: yadudoc1729 at gmail.com (Yadu Nand) Date: Wed, 23 Apr 2014 19:59:22 -0500 Subject: [Swift-devel] SSH-CL with port forwarding In-Reply-To: <5358341A.3020704@mcs.anl.gov> References: <5358341A.3020704@mcs.anl.gov> Message-ID: Hi Justin, I'm not sure why you are using ssh-cl, but I'm guessing that you want to get around firewall / auth issues. The configuration I use is coaster-service running on remote machine, with an ssh-tunnel between the remote machine and the local machine. Here are the steps. Assume you are on localmachine and you want to run on remotemachine # start ssh tunnel to forward the port swift is going to use to connect to the coaster-service ssh -L 51001:remotemachine:51001 user at remotemachine # start the coaster-service in passive mode # Make sure you use a recent swift-0.95 version coaster-service -p 51001 -localport 51005 -nosec -passive &> coaster-service.logs & # Start the worker , once connected worker would appear to be stuck worker.pl http://localhost:51005 0099 ~/workerlog # At this point you could try running a swift script on the remotemachine to check if the # setup is done right, with the following sites.xml file passive 4 10000 00:05:00 8.10 ./swiftwork The same sites.xml should work for you on localmachine. I tested this between crush and grind, with the latest swift-0.95 version. Thanks, ?Yadu On Wed, Apr 23, 2014 at 4:43 PM, Justin M Wozniak wrote: > Hi all > I'm trying to get SSH-CL going with port forwarding. I think I > just need to tell Coasters which port to use for the bootstrap > connection. I'm doing: > > sites.xml: > > > url="grind.mcs.anl.gov"/> > > > ssh -L 50001:localhost:50001 grind > > export GLOBUS_HOSTNAME=localhost > > export GLOBUS_TCP_PORT_RANGE=50000,50002 > > I either get: > > Failed to download bootstrap jar from http://localhost:50002 > > if the ports do not line up or: > > java.io.IOException: java.io.IOException: No free ports available. > > if I make the range too restrictive. > > How should I set this up? I cannot change the firewall rules. > > Or should I consider doing this with a local coaster service and manual > coasters? > > Thanks > Justin > > -- > Justin M Wozniak > > _______________________________________________ > Swift-devel mailing list > Swift-devel at ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel > -- Yadu Nand B -------------- next part -------------- An HTML attachment was scrubbed... URL: From hategan at mcs.anl.gov Fri Apr 25 14:56:22 2014 From: hategan at mcs.anl.gov (Mihael Hategan) Date: Fri, 25 Apr 2014 12:56:22 -0700 Subject: [Swift-devel] swift function signature definition Message-ID: <1398455782.24063.1.camel@echo> This is a follow-up to a discussion on the telecon. org.griphyn.vdl.engine.ProcedureSignature Mihael