[Swift-devel] mapper problem or ...?

[Mihael Hategan]

On Tue, 2007-03-13 at 18:07 -0500, Veronika V. Nefedova wrote:
> ok, here is in short what I need to do:
> 
> at some point in the workflow N files are produced (in my case its 68, but 
> it could be any number). These files produced each by a separate job (i.e. 
> N jobs produce N files).
> The next job in the workflow needs to take those N files as an input.
> 
> Question: how do I pass these unknown number of files as an input to an 
> application ? The array_mapper didn't work (or i didn't use it correctly).

In this case you need some other kind of mapper that can deal with
unknown numbers of items. The default mapper (i.e. specifying no mapper)
should work.

So you need to do:

file whamfiles_002[];

foreach v,k in someinput {
  whamfiles_002[k] = job(v);	
}

... = GENERATOR(whamfiles_002);

> 
> Thanks!
> 
> Nika
> 
> At 05:56 PM 3/13/2007, Mihael Hategan wrote:
> >On a third thought. This looks like, eventually, you are trying to do
> >the same thing that Yong did with the dependent mappers earlier. I think
> >he would have more insight on the topic.
> >
> >On Tue, 2007-03-13 at 17:23 -0500, Veronika V. Nefedova wrote:
> > > I think I am confused. Sorry!
> > > what will be the type of 'whamfiles' ? If its a string - will the swift
> > > know to brake it down to filenames and stage them all in ?
> > > Also - is there a mapper (or whatever) that can map the list of *logical*
> > > file names to an array ? (thats what I was trying to do).
> > >
> > > Thanks!
> > >
> > > Nika
> > >
> > >
> > > At 04:54 PM 3/13/2007, Mihael Hategan wrote:
> > > >Oh my :)
> > > >@whamfiles_m002 is known by the system at all times. That means
> > > >GENERATOR does not need to wait for the actual files to be there since
> > > >it knows very well what @whamfiles_m002 is (the list of names).
> > > >
> > > >You should try this instead:
> > > >...
> > > >... GENERATOR(whamfiles, str) {
> > > >    app {
> > > >      generator @whamfiles, str;
> > > >    }
> > > >}
> > > >
> > > >... = GENERATOR(whamfiles_m002, "m002")
> > > >
> > > >Mihael
> > > >
> > > >On Tue, 2007-03-13 at 16:46 -0500, Veronika V. Nefedova wrote:
> > > > > Hi,
> > > > >
> > > > > I have a question:
> > > > >
> > > > > I am using a fixed_array_mapper to pass some 68 files as an input to my
> > > > > application called GENERATOR. I need to use the mapper since the 
> > number of
> > > > > input files is unknown before the workflow starts. Here is how I 
> > use it:
> > > > > file whamfiles_m002[] <fixed_array_mapper;files=" 
> > solv_chg_a0_m002_wham,
> > > > > solv_chg_a1_m002_wham, solv_chg_a10_m002_wham, <snip -- many files, 
> > you
> > > > get
> > > > > the idea>, solv_repu_0_0DOT2_b1_m002_wham">;
> > > > >
> > > > > These files are all generated by stage four of my workflow, each 
> > file is
> > > > > mapped to a physical filename, for example:
> > > > >
> > > > > file solv_chg_a0_m002_wham  <"solv_chg_a0_m002.wham">;
> > > > > and this particular file is produced this way:
> > > > > (solv_chg_a0_m002_wham, solv_chg_a0_m002_crd, solv_chg_a0_m002_out,
> > > > > solv_chg_a0_m002_done) = CHARMM2 (standn, gaff_prm, gaff_rft,
> > > > > rtf_file_m002, prm_file_m002, psf_file_m002, crd_eq_file_m002,
> > > > > solv_chg_a0_m002_prt, "prtfile:solv_chg_a0", "system:solv_m002",
> > > > > "stitle:m002", "rtffile:parm03_gaff_all.rtf",
> > > > > "paramfile:parm03_gaffnb_all.prm", "gaff:m002_am1", "stage:chg",
> > > > > "urandseed:5395098", "dirname:solv_chg_a0_m002");
> > > > >
> > > > > Then I call my application (the last stage of my workflow, stage five)
> > > > >
> > > > > (solv_chg_m002, solv_disp_m002, solv_repu_0DOT2_0DOT3_m002DOTwham,
> > > > > solv_repu_0DOT3_0DOT4_m002DOTwham, solv_repu_0DOT4_0DOT5_m002DOTwham,
> > > > > solv_repu_0DOT5_0DOT6_m002DOTwham, solv_repu_0DOT6_0DOT7_m002DOTwham,
> > > > > solv_repu_0DOT7_0DOT8_m002DOTwham, solv_repu_0DOT8_0DOT9_m002DOTwham,
> > > > > solv_repu_0DOT9_1_m002DOTwham, solv_repu_0_0DOT2_m002DOTwham ) = 
> > GENERATOR
> > > > > (@whamfiles_m002, "m002");
> > > > >
> > > > > And then when I start my workflow, the GENERATOR starts right away. 
> > I am
> > > > > not sure why. Does the mapper look for the physical files on the 
> > disk and
> > > > > when finds them - starts right away ? I do have the needed files in the
> > > > > directory from my previous runs. Or there is something else wrong 
> > here ?
> > > > >
> > > > > 109] wiggum /sandbox/ydeng/alamines > Swift V 0.0405
> > > > > RunID: b0n2liektep92
> > > > > pre_ch started              <---------- thats the first stage
> > > > > generator_cat started    <----------- not supposed to start now!
> > > > > generator_cat started
> > > > >
> > > > > My complete dtm file is in /home/nefedova/swift.dtm on
> > > > > terminable.ci.uchicago, but its pretty big...
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Nika
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Swift-devel mailing list
> > > > > Swift-devel at ci.uchicago.edu
> > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > >
> > >
> > >
> 
>