[Swift-user] How does swift mapping to files that do not yet exist work?

[Michael Wilde]

Jonathan, from a cursory read of your script, I suspect that the Swift runtime is complaining that the app cce_ua did not produce the file new_point1_1.mat, which your output file mapping specified.

Given your output mapping, Swift is expecting new_point1_1.mat to exist in the temporary job directory in which your app was launched. I suspect your app (matlab script) did not create a file of this exact name.

You can specify the property "sitedir.keep=true" in your config file (e.g., -config cf on the command line) to tell Swift to retain these job directories on failure. They should be below the "workdirectory" that you specified in your sites.xml file.
(I think...)

- Mike


----- Original Message -----
> From: "Jonathan Margoliash" <jmargolpeople at gmail.com>
> To: swift-user at ci.uchicago.edu
> Sent: Tuesday, July 3, 2012 4:48:16 PM
> Subject: [Swift-user] How does swift mapping to files that do not yet exist work?
> Hello swift user base,
> 
> I have been having trouble with a bit of swift code for a while now,
> and despite my hesitancy to ask other people to debug my code, I
> believe my issue may be more of a lack of understanding of swift than
> a simple syntax error, so I have decided to send out this email.
> Specifically, I have a single_file_mapper mapping to a file which does
> not yet exist when my code is run, in hopes that the file being mapped
> to will be written to by a an app call. However, when I try running my
> code, I get the error back from swift
> 
> Execution failed:
> File not found: filename (in this case new_point1_1.mat)
> 
> This paradigm of referencing files that do not yet exist has worked
> before in swift, so I do not know what the problem is here. Hopefully
> it is not a simple syntax bug. Anyway, thank you for your help,
> 
> Jonathan
> 
> ----- My Code: ----------
> 
> #cce_ua.swift
> type file;
> 
> int num_complexes = @toint(@arg("num_complexes"));
> int num_simplexes_per_complex =
> @toint(@arg("num_simplexes_per_complex"));
> 
> app (file output, file log) cce_ua (file input1, file input2, int
> number, file input3)
> {
> cce_ua @input1 @input2 @output number @input3 stdout=@filename(log)
> stderr=@filename(log);
> #located at _RUNDIR_/swat/cce_ua.sh
> }
> 
> file inSCE<single_file_mapper;file="SCE_Par.mat">;
> file inrunswat<single_file_mapper;file="RunSwat.m">;
> 
> foreach i in [1:num_complexes]
> {
> foreach j in [1:num_simplexes_per_complex]
> {
> # These files already exist in the local directory
> file input<single_file_mapper; file=@strcat("simplex", i, "_", j,
> ".mat")>;
> # These files do not exist in the local directory at runtime
> file output<single_file_mapper; file=@strcat("new_point", i, "_", j,
> ".mat")>;
> file log<single_file_mapper; file=@strcat("cce_ua", i, "_", j,
> "_log.txt")>;
> 
> (output, log) = cce_ua(inSCE, input, i*num_simplexes_per_complex + j,
> inrunswat);
> }
> }
> 
> 
> ---- The command line call to swift (broken up over multiple lines)
> ---
> swift -tc.file swift_files/executable_locations -sites.file
> swift_files/sites.xml cce_ua.swift -num_complexes=2
> -num_simplexes_per_complex=1
> 
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory