[Swift-user] Swift stopping because of problematic foreach iteration

[Guilherme Gall]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi everyone,

I'm using Swift to automate and parallelize the potential distribution
modeling of some species'.

Follows the script:

- ------------------------------------------------------------------------
type request_file;
type environment_layers;
type occurrences_file;
type serialized_model;
type projected_map;

type modeling_results {
        serialized_model model;
        projected_map map;
}

request_file requests[]
<ext;exec="species_mapper.sh",o=@filename(occurrences)>;
environment_layers env_layers[]
<filesys_mapper;location="workshop/Brasil_ASC/", suffix=".asc">;

occurrences_file occurrences <single_file_mapper;file=@arg("o")>;
modeling_results
results[]<ext;exec="modeling_results_mapper.sh",o=@filename(occurrences)>;

app (request_file out[]) generate_requests(occurrences_file i) {
        generate_requests @filename(i);
}

app (modeling_results out) do_modeling(request_file r,
environment_layers e[], occurrences_file o) {
        om_console @filename(r);
}

requests = generate_requests(occurrences);

foreach request,index in requests {
        results[index] = do_modeling(request, env_layers, occurrences);
}
- ------------------------------------------------------------------------

The app do_modeling consumes a set of environment files, a file with
occurrence points from several species and a request file with
parameters for the chosen modeling algorithm (these request files are
generated by the generate_requests app).

The results of the modeling are 2 files per specie: one XML file and a
bitmap file.

The script works perfectly if the om_console command ends without
errors for **all** species. But if one of the species makes the
om_console command generate an error, the output files aren't created
and the script stops. Even if there are other species that would not
generate errors when modeled individually (out of Swift).

Follows the error reported by Swift in these cases:

- ------------------------------------------------------------------------
RunID: 20140206-0924-7dacf5h9
 (input): found 20 files
Progress:  time: Qui, 06 Fev 2014 09:24:34 -0200
Progress:  time: Qui, 06 Fev 2014 09:24:36 -0200  Checking status:1
Progress:  time: Qui, 06 Fev 2014 09:24:38 -0200  Stage in:11
Submitting:1  Finished successfully:1
Progress:  time: Qui, 06 Fev 2014 09:24:40 -0200  Active:11  Checking
status:1  Finished successfully:1
Progress:  time: Qui, 06 Fev 2014 09:24:42 -0200  Active:11  Checking
status:1  Finished successfully:1
Progress:  time: Qui, 06 Fev 2014 09:24:45 -0200  Active:11  Checking
status:1  Finished successfully:1
Execution failed:
        File not found:
/var/tmp/gmgall/swiftwork/workflow-openmodeller-20140206-0924-7dacf5h9/shared/output_Abarema
arenaria.xml
- ------------------------------------------------------------------------

The problem is that the script is stopping in arbitrary points because
of just one problematic iteration of the foreach loop in the end of
the script.

How can I address this issue to not leave behind species that would be
modeled otherwise?

Thanks in advance,
- -- 
Guilherme Gall
CSR/LNCC
GPG Public Key ID: A65ED0D5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJS84mFAAoJEG9WBlOmXtDVViMH/3WNaBG0OUPNJ97Fhc1odRwf
4JZPzuHKiVpvBh16lOcmph2kHehhzdlVbOGmIDRfBMCKWKDajh3k5DVxz08A17wZ
kfUnfq5DR+Ein6wr/XWDZ3WJPZBzdnHdtOY5VlIK+1K/71VVDorzQW5ReVAg2QDw
cFhvIhfvgepvo9IXyR6D91AaxUXdf44X4yi88XOLdNFjIyR6kxUBmCgf45RWSaeV
AwAfADQe+s82Q8qXPBE7iChdD1hmcj9V6Usfi43G1wPYZfKC6xTIFwmiS5nlHLFG
Ck1fo/dKMHxzR12zh72lG4yf6X2nhnTORWGA+LZa6Sd6lWlRl0YlifPHYrXnz3M=
=+obS
-----END PGP SIGNATURE-----