[Swift-devel] mapper problem or ...?
Yong Zhao
yongzh at cs.uchicago.edu
Tue Mar 13 18:35:29 CDT 2007
I think that should do, you put the output from an individual app and put
it into an array. We have the same pattern in the fmri workflow:
apply a reorient to a set of images, and then average the produced image
set.
Yong.
On Tue, 13 Mar 2007, Veronika V. Nefedova wrote:
> Hmmm. So here is how my files are produced (inside double loop over $s and
> $name):
>
> file $s9prt <"$name.prt">;
> file $s9wham <"$s9.wham">;
> file $s9crd <"$s9.crd">;
> file $s9out <"$s9.out">;
> file $s9done <"$s9donefile">;
>
> ($s9wham, $s9crd, $s9out, $s9done) = CHARMM3 (standn, gaff_prm, gaff_rft,
> rtf_file_$s, prm_file_$s, psf_file_$s, crd_eq_file_$s, $s9prt, "$ss1",
> "$s1", "$s2", "$s3", "$s4", "$s5", "$s7", "$s8", "$sprt", "$rcut1", "$rcut2");
>
> so if I change the mapping of the needed output file ($s9wham), everything
> should work?
>
> file whamfiles_$s[$i] <"$s9.wham">;
> i=`expr $i + 1`
>
> and call the function:
> (whamfiles_$s[$i], $s9crd, $s9out, $s9done) = CHARMM3 (standn, gaff_prm,
> gaff_rft, rtf_file_$s, prm_file_$s, psf_file_$s, crd_eq_file_$s, $s9prt,
> "$ss1", "$s1", "$s2", "$s3", "$s4", "$s5", "$s7", "$s8", "$sprt", "$rcut1",
> "$rcut2");
>
> Nika
>
> At 06:12 PM 3/13/2007, Mihael Hategan wrote:
> >On Tue, 2007-03-13 at 18:07 -0500, Veronika V. Nefedova wrote:
> > > ok, here is in short what I need to do:
> > >
> > > at some point in the workflow N files are produced (in my case its 68, but
> > > it could be any number). These files produced each by a separate job (i.e.
> > > N jobs produce N files).
> > > The next job in the workflow needs to take those N files as an input.
> > >
> > > Question: how do I pass these unknown number of files as an input to an
> > > application ? The array_mapper didn't work (or i didn't use it correctly).
> >
> >In this case you need some other kind of mapper that can deal with
> >unknown numbers of items. The default mapper (i.e. specifying no mapper)
> >should work.
> >
> >So you need to do:
> >
> >file whamfiles_002[];
> >
> >foreach v,k in someinput {
> > whamfiles_002[k] = job(v);
> >}
> >
> >... = GENERATOR(whamfiles_002);
> >
> > >
> > > Thanks!
> > >
> > > Nika
> > >
> > > At 05:56 PM 3/13/2007, Mihael Hategan wrote:
> > > >On a third thought. This looks like, eventually, you are trying to do
> > > >the same thing that Yong did with the dependent mappers earlier. I think
> > > >he would have more insight on the topic.
> > > >
> > > >On Tue, 2007-03-13 at 17:23 -0500, Veronika V. Nefedova wrote:
> > > > > I think I am confused. Sorry!
> > > > > what will be the type of 'whamfiles' ? If its a string - will the swift
> > > > > know to brake it down to filenames and stage them all in ?
> > > > > Also - is there a mapper (or whatever) that can map the list of
> > *logical*
> > > > > file names to an array ? (thats what I was trying to do).
> > > > >
> > > > > Thanks!
> > > > >
> > > > > Nika
> > > > >
> > > > >
> > > > > At 04:54 PM 3/13/2007, Mihael Hategan wrote:
> > > > > >Oh my :)
> > > > > >@whamfiles_m002 is known by the system at all times. That means
> > > > > >GENERATOR does not need to wait for the actual files to be there since
> > > > > >it knows very well what @whamfiles_m002 is (the list of names).
> > > > > >
> > > > > >You should try this instead:
> > > > > >...
> > > > > >... GENERATOR(whamfiles, str) {
> > > > > > app {
> > > > > > generator @whamfiles, str;
> > > > > > }
> > > > > >}
> > > > > >
> > > > > >... = GENERATOR(whamfiles_m002, "m002")
> > > > > >
> > > > > >Mihael
> > > > > >
> > > > > >On Tue, 2007-03-13 at 16:46 -0500, Veronika V. Nefedova wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > I have a question:
> > > > > > >
> > > > > > > I am using a fixed_array_mapper to pass some 68 files as an
> > input to my
> > > > > > > application called GENERATOR. I need to use the mapper since the
> > > > number of
> > > > > > > input files is unknown before the workflow starts. Here is how I
> > > > use it:
> > > > > > > file whamfiles_m002[] <fixed_array_mapper;files="
> > > > solv_chg_a0_m002_wham,
> > > > > > > solv_chg_a1_m002_wham, solv_chg_a10_m002_wham, <snip -- many
> > files,
> > > > you
> > > > > > get
> > > > > > > the idea>, solv_repu_0_0DOT2_b1_m002_wham">;
> > > > > > >
> > > > > > > These files are all generated by stage four of my workflow, each
> > > > file is
> > > > > > > mapped to a physical filename, for example:
> > > > > > >
> > > > > > > file solv_chg_a0_m002_wham <"solv_chg_a0_m002.wham">;
> > > > > > > and this particular file is produced this way:
> > > > > > > (solv_chg_a0_m002_wham, solv_chg_a0_m002_crd, solv_chg_a0_m002_out,
> > > > > > > solv_chg_a0_m002_done) = CHARMM2 (standn, gaff_prm, gaff_rft,
> > > > > > > rtf_file_m002, prm_file_m002, psf_file_m002, crd_eq_file_m002,
> > > > > > > solv_chg_a0_m002_prt, "prtfile:solv_chg_a0", "system:solv_m002",
> > > > > > > "stitle:m002", "rtffile:parm03_gaff_all.rtf",
> > > > > > > "paramfile:parm03_gaffnb_all.prm", "gaff:m002_am1", "stage:chg",
> > > > > > > "urandseed:5395098", "dirname:solv_chg_a0_m002");
> > > > > > >
> > > > > > > Then I call my application (the last stage of my workflow,
> > stage five)
> > > > > > >
> > > > > > > (solv_chg_m002, solv_disp_m002, solv_repu_0DOT2_0DOT3_m002DOTwham,
> > > > > > > solv_repu_0DOT3_0DOT4_m002DOTwham,
> > solv_repu_0DOT4_0DOT5_m002DOTwham,
> > > > > > > solv_repu_0DOT5_0DOT6_m002DOTwham,
> > solv_repu_0DOT6_0DOT7_m002DOTwham,
> > > > > > > solv_repu_0DOT7_0DOT8_m002DOTwham,
> > solv_repu_0DOT8_0DOT9_m002DOTwham,
> > > > > > > solv_repu_0DOT9_1_m002DOTwham, solv_repu_0_0DOT2_m002DOTwham ) =
> > > > GENERATOR
> > > > > > > (@whamfiles_m002, "m002");
> > > > > > >
> > > > > > > And then when I start my workflow, the GENERATOR starts right
> > away.
> > > > I am
> > > > > > > not sure why. Does the mapper look for the physical files on the
> > > > disk and
> > > > > > > when finds them - starts right away ? I do have the needed
> > files in the
> > > > > > > directory from my previous runs. Or there is something else wrong
> > > > here ?
> > > > > > >
> > > > > > > 109] wiggum /sandbox/ydeng/alamines > Swift V 0.0405
> > > > > > > RunID: b0n2liektep92
> > > > > > > pre_ch started <---------- thats the first stage
> > > > > > > generator_cat started <----------- not supposed to start now!
> > > > > > > generator_cat started
> > > > > > >
> > > > > > > My complete dtm file is in /home/nefedova/swift.dtm on
> > > > > > > terminable.ci.uchicago, but its pretty big...
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Nika
> > > > > > >
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Swift-devel mailing list
> > > > > > > Swift-devel at ci.uchicago.edu
> > > > > > > http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> > > > > > >
> > > > >
> > > > >
> > >
> > >
>
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
More information about the Swift-devel
mailing list