[Swift-devel] user guide work

Tue May 19 10:53:12 CDT 2015

Thanks Mike.

I am also going to start planning the workflow lectures on Beagle, goes without saying that swift will be part of it ;-)

I will start sending draft when I have something. If you have any ideas or pointers, please send them over.

Lorenzo

> On May 19, 2015, at 10:49 AM, Michael Wilde <wilde at anl.gov> wrote:
> 
> Thanks, Lorenzo - this is very useful feedback.
> 
> We have some good documentation for 0.96.0 in progress, and will welcome 
> further feedback, testing and improvements on it once its posted for 
> comment.
> 
> - Mike
> 
> On 5/19/15 9:25 AM, Lorenzo Pesce wrote:
>> I will throw in my useless 2 cents. I have been part of five or six swift based projects
>> all involving what I would call large computations (at least a million core hours, usually tens of terabytes of data).
>> 
>> In my experience:
>> 1) The basic variables are all we ever needed.
>> 2) The basic control loops is all we ever needed even if some more complex ones could have been handy (they just weren’t there when I started).
>> 3) All the file system trickeries known to the devil and his friends are absolutely necessary. I never found a project that didn’t end up chocking on its data.
>> 4) All the tricks for dealing intelligently with staging of data and code and “offloading” control are essential to avoid the hatred of the system administrators and other users.
>> 5) A very clear understanding of where Swift is robust and where it isn’t are very important because nobody likes to have a 300 node run come crashing down on you creating an infestation of zombies that eventually might have forced Beagle’s admin to reboot. This is not criticism, I participated heavily in writing that script, so I am on the accused team.
>> 
>> I can try to be more helpful, if you want me to.
>> 
>> I have worked on a number of parallel projects, where essentially the same workflow was implemented with and without swift and both teams played to win. That is one paper I plan to write some day…
>> 
>> 
>>> On May 18, 2015, at 3:27 PM, Ketan Maheshwari <ketan at mcs.anl.gov> wrote:
>>> 
>>> I think a little bit of both.  A brief overview of important things
>>> with examples in chapter 1. Then, topics like "Task Parallel
>>> Computing" or "Many Task Computing" may cover foreach, futures and
>>> task parallelism as implemented in Swift. Then, a chapter like "File
>>> I/O" may cover the mappers, etc. Then a topic like operators and
>>> variables may bring in more of the language reference in that
>>> describing types, structs, etc. Next may be builtin functions.
>>> Followed by a chapter on Compute sites which covers Resource and data
>>> managers, remote access and providers and such.
>>> 
>>> So, to elaborate here is a rough outline I have in mind:
>>> 
>>> Chap 1. Swift Overview
>>> -- Importance of Swift
>>> -- Swift example 1
>>>   -- hello world, description, concept show in this example
>>> -- Swift example 2
>>>   -- Run an app functions, describe importance of app functions
>>> -- Swift example 3
>>>   -- Do file mappings, describe mappers
>>> -- Swift example 4
>>>  -- Do foreach loop, describe foreach
>>> -- Swift example 5 (if needed)
>>> 
>>> Chap 2. Many Task Computing in Swift
>>> -- Parallelism in DAGs
>>> -- Futures
>>> -- foreach loop and variants
>>> -- iterate
>>> 
>>> Chap 3. File I/O
>>> -- File types
>>> -- Simple file def
>>> -- Complex Mappers
>>> -- Read and Write to Files
>>> 
>>> Chap 4. Variables and Operators
>>> -- types
>>> -- structs
>>> -- import
>>> 
>>> Chap 5. User defined and Builtin functions
>>> -- string
>>> -- numerical
>>> -- stat
>>> -- math
>>> 
>>> Chap 6. Swift providers
>>> -- Coasters
>>> -- Data providers
>>> -- Compute Providers
>>> 
>>> Chap 6. Remote compute sites
>>> -- Modes of running
>>> 
>>> Appendix A. Swift Grammar in BNF
>>> Appendix B. A table of Swift conf options
>>> Appendix C. Swift cheatsheet
>>> 
>>> 
>>> On Mon, May 18, 2015 at 2:20 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
>>>> On Mon, 2015-05-18 at 13:27 -0500, Ketan Maheshwari wrote:
>>>>> I think Swift is a special purpose language where ~20% of features are
>>>>> used for ~80% of cases. Consequently, providing a K&R style language
>>>>> manual might not be the most valuable thing for a user to read. It
>>>>> also runs a risk of coming across as a general purpose language which
>>>>> may mislead readers.
>>>>> 
>>>>> I think, a document structure where the important features (app and
>>>>> builtin functions, variables and mappers (with examples)) are
>>>>> highlighted in the early parts and details such as variable scope,
>>>>> expressions, operators etc. are described in the middle and concluded
>>>>> with sites specific details might serve well.
>>>> You mean like a brief overview of important things / tutorial in the
>>>> beginning?
>>>> 
>>>> Or do you mean that the language reference should not have a bottom-up
>>>> structure, starting with the simple/primary concepts and building up
>>>> from that?
>>>> 
>>>> Mihael
>>>> 
>>>>> --
>>>>> Ketan
>>>>> 
>>>>> On Sat, May 16, 2015 at 4:04 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> Mike pointed out that I should probably post a link to what I'm trying
>>>>>> to do with the user guide just in case things are going awfully wrong.
>>>>>> So here it is:
>>>>>> 
>>>>>> http://www.mcs.anl.gov/~hategan/ug/ug/ug.html
>>>>>> 
>>>>>> Mike mention K&R a number of times and I think I like that general idea.
>>>>>> The above, so far, is a mix between K&R Chapter 2 and Appendix A.
>>>>>> 
>>>>>> The main idea is to keep things concise by separating parallelism
>>>>>> behaviour from what a program actually does. In that sense, one would
>>>>>> first describe the language, and then go into the details of how a
>>>>>> particular implementation goes about running programs.
>>>>>> 
>>>>>> I'm not particularly opposed to starting with a brief tutorial in the
>>>>>> spirit of K&R Chapter 1.
>>>>>> 
>>>>>> Mihael
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Swift-devel mailing list
>>>>>> Swift-devel at ci.uchicago.edu
>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>> 
>>>> _______________________________________________
>>>> Swift-devel mailing list
>>>> Swift-devel at ci.uchicago.edu
>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>> _______________________________________________
>>> Swift-devel mailing list
>>> Swift-devel at ci.uchicago.edu
>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> -- 
> Michael Wilde
> Mathematics and Computer Science          Computation Institute
> Argonne National Laboratory               The University of Chicago
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel