[Swift-user] XDTM

J A jamalphd at gmail.com
Fri Aug 7 15:40:03 CDT 2009 

Hi Michael:

   1. After running a .swift or .dtm code, two files gets created:  .xml and
   .klm.  What do they represent?
   2. Correct me if I am wrong:
      - Datasets are mapped to physical presentation using mapping
      algorithms.  Some mapping algorithms already created part of
swift and the
      user can add/create others and use the existing once as the base.
      - Currently, the physical representation are files.
   3. In the fMRI example, I see volume, Image, etc declared as a type?
   who defines them as a type?
   4. In one of your emails, you stated that Swift functions can take accept
   files, int, string, float and boolean values as arguments. They return
   files, or scalar values inside files. My question is:  if the output is a
   string that is inside a file, how can I use this output in another program
   that takes it as an input?  doesn't call the file name and should have a
   code to read from the file?
   5. I am still confused when talk about XML Data Type and Mapping.  Where
   is the XML representation?  Is it the .xml that gets generated when run the
   swift code?
   6. Let's look at this example:

 type messagefile {}

(messagefile t) greeting (string s[]) {
    app {
        echo s[0] s[1] s[2] stdout=@filename(t);
    }
}

messagefile outfile <"q5out.txt">;

string words[] = ["how","are","you"];

outfile = greeting(words);
===

So we have messagefile as a data type.  outfile and words are datasets.
what will be the physical representation for these 2 datasets?  is thee
system parsing the swift code, identifying the data types and datasets and
based on that choosea the proper mapping algorithm needed?


Thanks,
Jamal






On Sun, Jul 26, 2009 at 9:53 PM, Michael Wilde <wilde at mcs.anl.gov> wrote:

> Hi Jamal,
>
> A lot of this is covered in the Swift user guide and tutorial. Have you
> read through those yet?
>
> All the docs are at: http://www.ci.uchicago.edu/swift/docs/index.php
>
> If so, and the clarifications below don't help, please ask again on the
> list, OK?
>
> - Mike
>
>
> On 7/26/09 7:27 PM, J A wrote:
>
>> Hi Michael:
>>  First, thank you for your reply and information provided.
>>  I am trying to understand more how it handles the input/output parameters
>> and make them available for other functions.
>>
>
> All functions in Swift are either atomic interfaces to application programs
> (ie, how o exec the program) or composite higher level functions.
>
>>  To illustrate, I will give this example for the sake of discussion:
>>  I have a C program called test.c that contains 4 functions ( main(), F1,
>> F2, and F3).  each function takes some parameters such as int, string, name
>> of a file that is in the same directory, and each one produced some output
>> (string, int, and a file).  Of course i am using global variables.  Now,
>> main calls F1, F1 passes its output to F2, and F2 passes its output to F3.
>>
>
> Swift doesnt look at the functions inside an application. It invokes the
> application as a program (think fork/exec) just like a shell would, but
> distributed and in parallel if so specified.
>
>>  Overall, the test.c takes an int, string, and file, and output several
>> files.  the output files contains output produced by the internal functions
>> (tasks).
>>
>
> Swift functions can take accept files, int, string, float and boolean
> values as arguments. They return files, or scalar values inside files.
> (Again, think shell scripts).  Composite structures - structs and arrays -
> of the above can be passed.
>
>>  I would like to understand more when i transfer my code to Swift how it
>> handles the input/output data, where it stores them, etc.  I read couple of
>> papers about XDTM and still have some confusion about the terms:  dataset,
>> typed, how/where its physical representation is located at, and how the
>> input/output is used within the internal functions.
>>
>
> Files are by default named ("mapped") relative to the directory in which
> you run the Swift command. Many flexible extensions to that model are
> provided for (eg, URIs).  Swift sends the data to the site chosen for
> execution (thats yet another topic) and returns results back to the same
> submission host.
>
> Mapping declarations in the Swift script specify how files and directory
> structures are mapped to Swift variables (scalars, arrays, structures).
> These are used in the specification of the Swift code. When Swift runs
> programs, it takes files that were mapped and knows how to send them to grid
> sites or clusters and get data back.
>
>>   I am new to this area and trying to understand how the DTM works.
>>  Any help from your side on this area is really appreciated.
>>  Thanks,
>> Jamal
>>
>>   On Sun, Jul 26, 2009 at 7:09 PM, Michael Wilde <wilde at mcs.anl.gov<mailto:
>> wilde at mcs.anl.gov>> wrote:
>>
>>    Jamal,
>>
>>    As Swift evolved from its early prototypes to a more mature system,
>>    the notion of XDTM evolved to one of mapping between
>>    filesystem-based structures and Swift in-memory data structures (ie,
>>    scalars, arrays, and structures, which can be nested and typed).
>>
>>    This is best seen by looking at the "external" mapper, which allows
>>    a user to map a dataset using any external program (typically a
>>    script) that returns the members of the dataset as a two-column
>>    list: the Swift variable reference, and the external file or URI.
>>
>>    See the user guide section on the external mapper:
>>
>>
>> http://www.ci.uchicago.edu/swift/guides/userguide.php#mapper.ext_mapper
>>    (but the example in the user guide doesn't show the power of mapping
>>    to nested structures).
>>
>>    In other words, it still has the flavor of XDTM, but without any XML
>>    being visible to the user. It meets the same need but is easier to
>>    use and explain.
>>
>>    - Mike
>>
>>
>>    On 7/26/09 2:50 PM, J A wrote:
>>
>>        Hi All:
>>         Can any one direct me to a source with more
>>        examples/explanation on how XDTM is working/implemented?
>>         Thanks,
>>        Jamal
>>
>>
>>  ------------------------------------------------------------------------
>>
>>        _______________________________________________
>>        Swift-user mailing list
>>        Swift-user at ci.uchicago.edu <mailto:Swift-user at ci.uchicago.edu>
>>        http://mail.ci.uchicago.edu/mailman/listinfo/swift-user
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20090807/a1a89fc0/attachment.html>

More information about the Swift-user mailing list