[Swift-devel] Re: Question and update on swift script

[Michael Wilde]

Jim, the Swift tutorial I gave last week has a few version of a system 
diagram. Let me know if this meets your needs:

https://twiki.grid.iu.edu/pub/Education/FloridaInternationalGridSchool2008Syllabus/6_SwiftWorkflow.pdf

https://twiki.grid.iu.edu/pub/Education/FloridaInternationalGridSchool2008Syllabus/6_SwiftWorkflow.ppt

slides 7, 22, 28.  Feel free to use anything else.

- Mike


On 1/30/08 11:30 AM, James Simone wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dear swift developers,
> 
> Do you have a system diagram for swift?
> I'd like to show one as part of my talk on Friday.
> 
>         Thanks,
>         --jim
> 
> James Simone wrote:
> | Thanks Mihael!
> |
> |
> | Mihael Hategan wrote:
> | | Mike is probably on or around a plane.
> | |
> | | Here's my feeble attempt at it.
> | |
> | | Mihael
> | |
> | | On Fri, 2008-01-25 at 10:20 -0600, James Simone wrote:
> | |> Hi Mike,
> | |>
> | |> Could you please send us your target version of the 2-pt QCD workflow?
> | |> The target swiftscipt would not necessarily be able to run with 
> current
> | |> swift, but it would include syntax changes that are/will be under
> | development.
> | |>
> | |>     Thanks,
> | |>     --jim
> | |>
> | |> Michael Wilde wrote:
> | |>> Hi all,
> | |>>
> | |>> I made some good progress yesterday in understanding the swift
> code for
> | |>> the 2ptHL workflow, and started revising it.
> | |>>
> | |>> Im doing the following:
> | |>> - giving all files a mnemonic type name
> | |>> - creating compound types that encapsulate each data/info file pair
> | |>> - putting each set of nested foreach loops into a procedure (for
> better
> | |>> abstraction)
> | |>> - changing the mapper to tie it to each data type
> | |>>
> | |>> For now, Im also pulling the two cache-loading functions out of the
> | |>> workflow, as it seems like these should be part of the runtime
> | |>> environment, rather than in the workflow script. Do you feel the 
> same?
> | |>>
> | |>> I *thought* that you had a set of wrappers that were python stubs for
> | |>> simulated execution, but the wrappers Don sent me looked like 
> wrappers
> | |>> that call the *real* code. So for now I'll create my own simple
> | stubs to
> | |>> test the data flow with.
> | |>>
> | |>> Ive got many more questions that Im collecting, but for now I only
> need
> | |>> the answer to this one:
> | |>>
> | |>> In the nested loops that call the function Onia() (lines 136-148
> in the
> | |>> attached numbered listing), is the actual call at line 145 correct?
> | |>> This call is passing the same "HeavyQuarkConverted" as both the
> | |>> anti-quark and quark1. Im assuming that is the correct intent.
> | |>> Also, its passing only the wave[1] wave file (1S) each time 
> (second of
> | |>> the three wave files d, 1S, 2S). (Same is true for BStaggered).
> | |>>
> | |>> Lastly, Onia seems to be getting called with successive pairs of
> | |>> converted heavy quark files, but it seems like the final call is 
> going
> | |>> to get a null file for the second file, as the value of "bindex"
> | will be
> | |>> one past the last converted heavy quark file computed.
> | |>>
> | |>> Im looking for ways to avoid the way the current script needs to
> | compute
> | |>> the various array indices, but Im not likely to find something that
> | |>> doesnt require a language change. Was this approach of computing the
> | |>> indices something that your team liked, or did not like, about this
> | |>> swift script?
> | |>>
> | |>> I'll try to send more details and questions as I proceed.
> | |>> A few of these are below. If you have a moment to consider these, 
> that
> | |>> will help me get a better understanding of the environment.
> | |>>
> | |>> Thanks,
> | |>>
> | |>> - Mike
> | |>>
> | |>> qcopy: I need to learn more about how you manage data and use dcache,
> | |>> but for now, I get the essence if this. Can you point me to a
> qcopy doc
> | |>> page though? I couldnt find one.
> | |>>
> | |>> tag_array_mapper: I can guess how this works, but can you send me the
> | |>> code for it? It seems like the order in which it fills the array must
> | |>> match the index computations in your swift script. Looks to me
> like the
> | |>> leftmost tag in the format script is varied fastest (ie, is the 
> "least
> | |>> significant index"). Is this correct?
> | |>>
> | |>> kappa value passing bug: you allude to this in a comment.  Was this a
> | |>> swift problem or a problem in the py wrapper?  Resolved or still 
> open?
> | |>> If Swift, I can test to see if I can reproduce it. But I suspect you
> | |>> were testing against a pretty old version of swift?
> | |>>
> | |>> Is the notion of an "info" file paired with most data files a 
> standard
> | |>> metadata convention for LQCD? (Ie, Im assuming that this is done
> | |>> throughout your apps, not just in this swift example, right?  If
> so, it
> | |>> seems to justify a mapping convention so that you can simply pass a
> | |>> "Quark" object, and have the data and info files passed 
> automatically,
> | |>> together. You can then dereference the object top extract each field
> | |>> (file).
> | |>>
> | |>> Are the file naming conventions set by the tag mapper the ones 
> already
> | |>> in use by the current workflow system? I.e., the order of the foreach
> | |>> loops and hence of the mappings was chosen carefully to match the
> | |>> existing file-naming conventions?
> | |>>
> | |>> How is the name of the ensemble chosen? Does it have a relation to 
> the
> | |>> phyarams? Is it a database key? (It seems opaque to the current swift
> | |>> example. Is that by preference or was there a desire to expose its
> | |>> structure? Its its contents related to the phyparams?  Or when
> | looked up
> | |>> in a DB, does it yield the phyparams?
> | |>>
> | |>>
> | |> _______________________________________________
> | |> Swift-devel mailing list
> | |> Swift-devel at ci.uchicago.edu
> | |> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> | |>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.8 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iEYEARECAAYFAkegtCYACgkQCdIhGvtLXB0YXACfWs5Va2U23eTxzCiSnjPVcRqe
> Jp0AnjeQ9ONnE0H9FHp97RIcOm80bENd
> =CIo3
> -----END PGP SIGNATURE-----
> 
>