[Swift-user] Question about mappers

[Michael Wilde]

OK, I think I'll need to look at the code more closely and maybe get more info.

I think youre on the right track, in terms of looking at the run directory issues. I suspect this can be further simplified in the process of debugging it.  Thats all I can say for now; I will need to spend more time studying your example later.

- Mike

----- Original Message -----
> From: "Lorenzo Pesce" <lpesce at uchicago.edu>
> To: "Michael Wilde" <wilde at mcs.anl.gov>
> Cc: swift-user at ci.uchicago.edu
> Sent: Friday, March 30, 2012 8:56:35 AM
> Subject: Re: [Swift-user] Question about mappers
> > file result <single_file_mapper; file=@strcat("result_neuron_", chn,
> > "_ht_",s,".mat")>;
> > file LogDemoReal <single_file_mapper; file=@strcat("demo_real", chn,
> > "_ht_",s,".log")>;
> >
> > If you want these files to get created in one of the WinDirs
> > directories, you should prepend win,"/" in front:
> >
> > file result <single_file_mapper;
> > file=@strcat(win,"/"."result_neuron_",...
> >
> > Was that what you intended?
> 
> It would be, but if I add it, it does this
> 
> Exception in runDemoReal:
> Arguments: [/soft/matlab/7.13, 1.0, 3.0,
> /lustre/beagle/GCNet/grasping/200ms_5windows_new/Win7/X.mat]
> Host: pbs
> Directory:
> demo_real-20120330-1352-71yp4ivb/jobs/x/runDemoReal-xe0j37pk
> stderr.txt:
> stdout.txt:
> 
> ----
> 
> Caused by: The following output files were not created by the
> application: Win7/result_neuron_1_ht_3.mat
> 
> because (I think) the script generates the files in its own run
> directory.
> Originally the script has a cd to Win# and then ran the code.
> 
> 
> 
> > Also, there should be no need to generate the Swift script with
> > shell variable substitution. You can pass in values from the shell
> > script via swift command line and access them from the Swift script
> > with the @arg() function.
> >
> > You can make htvec and chnvec string vectors instead of ints, and
> > pass them like so:
> >
> > swift -tc.file etc etc myscript.swift -htvec=1,3,5 -chnvec=a,b,c
> > -currdir=$PWD
> >
> > and in the script do:
> >
> > string htvec[]=@strsplit(@arg("htvec"));
> >
> > You can use the strng htvec[] in your later @strcat() call in the
> > same way you used the int version of that vector.
> 
> Thanks.
> 
> 
> >
> > - Mike
> >
> >
> >
> >
> >
> > ----- Original Message -----
> >> From: "Lorenzo Pesce" <lpesce at uchicago.edu>
> >> To: swift-user at ci.uchicago.edu
> >> Sent: Thursday, March 29, 2012 9:03:48 PM
> >> Subject: [Swift-user] Question about mappers
> >> Hi all--
> >> Thanks a lot for your help. I have another question.
> >>
> >> This is what my problem looks like. There is a set of input files,
> >> all
> >> with the same name (say X.mat), in a set of subdirectories of say
> >> <root_dir>:
> >> Win1
> >> Win2
> >> Win3
> >> .
> >> .
> >> .
> >> WinN
> >>
> >>
> >>
> >> The applications takes the input file as *filename* (string) and
> >> not a
> >> handle, then some other parameters and produces a couple of output
> >> files (see below for app demoreal).
> >>
> >> For each directory, say Win1, each file X.mat is mapped to many
> >> output
> >> files result_neuron_<i>_ht_<j>.mat, where i and j are a parameter
> >> sweep (same for all windows, but the X.mat files are different).
> >>
> >> My script pokes around, takes some input and then spews up a swift
> >> script. My mapping misfires and all the output files end up in
> >> <root_dir>, which doesn't work because many should have the same
> >> name.
> >> If the input file were a handle, I would use a file type and map
> >> input
> >> and output and I think that I would know how to do that. Here I
> >> have
> >> no idea.
> >>
> >> // file to run the Margic square example from the matlab web site
> >> type file;
> >>
> >> string MCRPath = "/soft/matlab/7.13";
> >> string spikefilename = "$infile";
> >>
> >> app (file outdata,file result) demoreal (string mcr, int chn, int
> >> htin, string datafile)
> >> {
> >> runDemoReal mcr chn htin datafile stdout=@outdata;
> >> }
> >>
> >> string WinDirs[] = ${swiftWinDirs};
> >> int htvec[] = ${htvec};
> >> int chnvec[] = ${chnvec};
> >>
> >>
> >> foreach win in WinDirs {
> >> string spikefile = @strcat("${currdir}/",win,spikefilename);
> >> foreach chn in chnvec {
> >> foreach s, i in htvec {
> >> file result <single_file_mapper; file=@strcat("result_neuron_",
> >> chn,
> >> "_ht_",s,".mat")>;
> >> file LogDemoReal <single_file_mapper; file=@strcat("demo_real",
> >> chn,
> >> "_ht_",s,".log")>;
> >> (LogDemoReal,result) = demoreal (MCRPath, chn, s, spikefile);
> >> }
> >> }
> >> }
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Swift-user mailing list
> >> Swift-user at ci.uchicago.edu
> >> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >
> > --
> > Michael Wilde
> > Computation Institute, University of Chicago
> > Mathematics and Computer Science Division
> > Argonne National Laboratory
> >

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory