[Swift-user] Swift stopping because of problematic foreach iteration

[Luiz Gadelha]

Hi Guilherme,

It seems to be an openmodeller problem, it's returning exit status 0 
even when it reports an error. Swift must be interpreting that the 
execution was successful but then it reports an error because it's not 
able to find the files that should have been generated by a successful 
om_console execution. It seems also that in addition to the presence 
points we are giving as input, the Maxent model requires some absence 
points as well, it should fail anyway since it doesn't have enough 
information.

$ /usr/bin/time -f exit_status:%x om_console request_Abarema\ 
brachystachya.txt
[Info] openModeller version 1.4.0
[Error] Cannot create model without any presence or absence point.
[Info] Exception occurred: Cannot create model without any presence or 
absence point.

exit_status:0

Regards,

Luiz

On 02/06/2014 07:09 AM, Guilherme Gall wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi everyone,
>
> I'm using Swift to automate and parallelize the potential distribution
> modeling of some species'.
>
> Follows the script:
>
> - ------------------------------------------------------------------------
> type request_file;
> type environment_layers;
> type occurrences_file;
> type serialized_model;
> type projected_map;
>
> type modeling_results {
>          serialized_model model;
>          projected_map map;
> }
>
> request_file requests[]
> <ext;exec="species_mapper.sh",o=@filename(occurrences)>;
> environment_layers env_layers[]
> <filesys_mapper;location="workshop/Brasil_ASC/", suffix=".asc">;
>
> occurrences_file occurrences <single_file_mapper;file=@arg("o")>;
> modeling_results
> results[]<ext;exec="modeling_results_mapper.sh",o=@filename(occurrences)>;
>
> app (request_file out[]) generate_requests(occurrences_file i) {
>          generate_requests @filename(i);
> }
>
> app (modeling_results out) do_modeling(request_file r,
> environment_layers e[], occurrences_file o) {
>          om_console @filename(r);
> }
>
> requests = generate_requests(occurrences);
>
> foreach request,index in requests {
>          results[index] = do_modeling(request, env_layers, occurrences);
> }
> - ------------------------------------------------------------------------
>
> The app do_modeling consumes a set of environment files, a file with
> occurrence points from several species and a request file with
> parameters for the chosen modeling algorithm (these request files are
> generated by the generate_requests app).
>
> The results of the modeling are 2 files per specie: one XML file and a
> bitmap file.
>
> The script works perfectly if the om_console command ends without
> errors for **all** species. But if one of the species makes the
> om_console command generate an error, the output files aren't created
> and the script stops. Even if there are other species that would not
> generate errors when modeled individually (out of Swift).
>
> Follows the error reported by Swift in these cases:
>
> - ------------------------------------------------------------------------
> RunID: 20140206-0924-7dacf5h9
>   (input): found 20 files
> Progress:  time: Qui, 06 Fev 2014 09:24:34 -0200
> Progress:  time: Qui, 06 Fev 2014 09:24:36 -0200  Checking status:1
> Progress:  time: Qui, 06 Fev 2014 09:24:38 -0200  Stage in:11
> Submitting:1  Finished successfully:1
> Progress:  time: Qui, 06 Fev 2014 09:24:40 -0200  Active:11  Checking
> status:1  Finished successfully:1
> Progress:  time: Qui, 06 Fev 2014 09:24:42 -0200  Active:11  Checking
> status:1  Finished successfully:1
> Progress:  time: Qui, 06 Fev 2014 09:24:45 -0200  Active:11  Checking
> status:1  Finished successfully:1
> Execution failed:
>          File not found:
> /var/tmp/gmgall/swiftwork/workflow-openmodeller-20140206-0924-7dacf5h9/shared/output_Abarema
> arenaria.xml
> - ------------------------------------------------------------------------
>
> The problem is that the script is stopping in arbitrary points because
> of just one problematic iteration of the foreach loop in the end of
> the script.
>
> How can I address this issue to not leave behind species that would be
> modeled otherwise?
>
> Thanks in advance,
> - --
> Guilherme Gall
> CSR/LNCC
> GPG Public Key ID: A65ED0D5
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.14 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJS84mFAAoJEG9WBlOmXtDVViMH/3WNaBG0OUPNJ97Fhc1odRwf
> 4JZPzuHKiVpvBh16lOcmph2kHehhzdlVbOGmIDRfBMCKWKDajh3k5DVxz08A17wZ
> kfUnfq5DR+Ein6wr/XWDZ3WJPZBzdnHdtOY5VlIK+1K/71VVDorzQW5ReVAg2QDw
> cFhvIhfvgepvo9IXyR6D91AaxUXdf44X4yi88XOLdNFjIyR6kxUBmCgf45RWSaeV
> AwAfADQe+s82Q8qXPBE7iChdD1hmcj9V6Usfi43G1wPYZfKC6xTIFwmiS5nlHLFG
> Ck1fo/dKMHxzR12zh72lG4yf6X2nhnTORWGA+LZa6Sd6lWlRl0YlifPHYrXnz3M=
> =+obS
> -----END PGP SIGNATURE-----
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>


-- 
Luiz Gadelha
http://www.lncc.br/~lgadelha