[Swift-devel] Re: passing types and variables in swift

[Mihael Hategan]

On Thu, 2007-10-04 at 23:06 -0500, Mihael Hategan wrote:
> On Thu, 2007-10-04 at 22:53 -0500, Yong Zhao wrote:
> > Mike
> > 
> > > while we're discussing mappers, could you or Ben clarify:
> > >
> > > - can a mapper be only used on a single variable declaration? ie you can
> > > not map a dataset to a variable within a struct?  this would be useful
> > > eg to create a struct that has both file and scalar valued members, to
> > > set up a parameter sweep where each file needs specific parameters.
> > 
> > This would be the nested mapper scenario, in order to make the scripts
> > cleaner, I was think about a mapping descriptor (which could be an XML
> > file to describe each layer of the mapping), but this is kind of far
> > reaching.
> 
> I think C pointers provide a very good model for this. We should perhaps
> use that as a reference.

So let me elaborate on that.

All non-primitive types are pointers. For simplicity, we should have all
types as pointers, but currently we distinguish a bit between primitive
and non-primitive.

A complex type variable in Swift is like a pointer to a struct of
pointers.

In the beginning, Swift needs to figure out the exact addresses of those
pointers. That's what the existing() mapper method does. It goes through
the struct and recursively initializes the pointer addresses based on
some scheme which depends on the mapper implementation. This is like
doing a recursive field = malloc(sizeof(field)).

The map method of a mapper takes a struct path (a.b.c) and returns the
address.

Currently, nothing ever uses a value of a pointer. Grid applications are
passed the address (file name).

Getting/setting values amounts to pointer dereferencing. Both read and
write.

The addresses of these pointers are abstract. They don't represent
anything in particular. Currently we assume that they are specific
addresses (files), but desire is to extends this to databases and
things. Let's say that we have a segmented address space and our
pointers can be in different segments.

The segment issue only applies to how we read/write values.

There is one plain memory segment. Currently only primitive types can be
here.

There may be multiple memory-mapped I/O segments. Say one for files, one
for databases, etc. We need to implement what actually happens when you
read/write from/to a I/O segment. Basically mappers supporting
read/write of values are the I/O hardware drivers.

Whenever moving data between segments, we are either efficient and use
DMA transfers, or we read to the plain memory segment and then write
back to the other I/O segment. This is the analogy of, for example,
getting some value from a database to a file so it can be passed to an
application. The DMA case is when there is some entity that knows how to
convert directly between the two. The non-DMA case is when we convert
first to a common format and then back to another format.

> 
> > 
> > >
> > > - are mappers processed before the program (or procedure) starts
> > > executing? so you cant use a mapper like an assignment statement to set
> > > or reset a variable?
> > >
> > Mappers are evaluated before procedure execution, the only exception is
> > when a mapper depends on some intermediate data, and it will wait for that
> > data to be available (as in the montage workflow). However, mapping is
> > different from assignment (in assignment, it is more like calling a
> > mapping function such as @extractint). A variable currently can not be
> > reset or reassigned, unless it is an interation variable (it gets
> > re-assigned implicitly).
> > 
> > Yong.
> > 
> > > - mike
> > >
> > > On 10/4/07 10:21 PM, Yong Zhao wrote:
> > > > Mappers should ideally be able to map primitive types such as string and
> > > > int, as well as file names. The CSVMapper can read all the values in the
> > > > file, it is just that it needs to interprete the values according to their
> > > > types. So If we convey the type info 'RGIparams' to the csv mapper, it
> > > > should have no problem reading the values.
> > > >
> > > > I actually added getType and setType to the mapper interface, that did
> > > > not get into the release, but I think it is what it should have.
> > > >
> > > > Yong.
> > > >
> > > > On Thu, 4 Oct 2007 andrewj at uchicago.edu wrote:
> > > >
> > > >> All right, thanks.
> > > >>
> > > >> I am just trying to figure out the best way to setup parameter
> > > >> sweeps.  Ideally, I would like to have some file of directory
> > > >> of files containing experiments to be run on the Grid
> > > >> environment and Swift picks them up and automatically sets up
> > > >> the appropriate WF based on the parameter information in the
> > > >> files.
> > > >>
> > > >>
> > > >>> On Wed, 3 Oct 2007, andrewj at uchicago.edu wrote:
> > > >>>
> > > >>>> could I do something like this:
> > > >>>>
> > > >>>> RGIparams RGIinput[] <csv_mapper;file="rgi_runs.txt">;
> > > >>>>
> > > >>>> where rgi_runs would contain say
> > > >>>>
> > > >>>> 1,2,true
> > > >>>> 3,4,false
> > > >>>> 18,20,true
> > > >>>>
> > > >>>> and so on..so that the parameters are passed into the workflow?
> > > >>> no.
> > > >>>
> > > >>> mappers only map filenames (or URIs now); they don't map
> > > >> variable values.
> > > >>> --
> > > >> _______________________________________________
> > > >> Swift-devel mailing list
> > > >> Swift-devel at ci.uchicago.edu
> > > >> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > >>
> > > > _______________________________________________
> > > > Swift-devel mailing list
> > > > Swift-devel at ci.uchicago.edu
> > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > >
> > > >
> > >
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>