<div dir="ltr"><div>Sorry, original response was cut short.<br><br>The unfilled subscribe message occurs when there's an unfilled
dependency on an array entry, e.g. if X[0] is read but not written.<br><br>The
debug output has the logical names and array keys, e.g.
"ens_priors_transposed[0]", along with the source file/line number, e.g.
"aidi_EnKF:main():169:4". Variables with names like __t:61 are compiler-generated temporary variables that store an intermediate result. The line number can help narrow down the location.<br><br>So I'd start with investigating why these array keys aren't assigned:<br><br>float_var_names[-13] (aidi_EnKF:main():169:4)<br>loat_var_names[-14] (aidi_EnKF:main():169:4)<br>ens_priors_transposed[0] (aidi_EnKF:params_to_priors():77:4)<br><br></div>- Tim<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Sep 28, 2014 at 3:55 PM, Tim Armstrong <span dir="ltr"><<a href="mailto:tim.g.armstrong@gmail.com" target="_blank">tim.g.armstrong@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The unfilled subscribe message occurs when there's an unfilled dependency on an array entry, e.g. if X[0] is read but not written.<br><br>The debug output has the logical names and array keys, e.g. "ens_priors_transposed[0]", along with the source file/line number, e.g. "aidi_EnKF:main():169:4". Those are the ones to look at:<br><br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Sep 28, 2014 at 2:01 PM, Ozik, Jonathan <span dir="ltr"><<a href="mailto:jozik@anl.gov" target="_blank">jozik@anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello all,<br>
<br>
I’m getting UNFILLED CONTAINER SUBSCRIBE messages when running my script. Based on the error output below, is there a (relatively) easy way to track this down that you’d recommend?<br>
<br>
Jonathan<br>
<br>
WAITING WORK: 34<br>
{16} params_to_priors-foreach-wait3 (/<37>:upf_lines (aidi_EnKF:params_to_priors():46:4)/)<br>
{17} params_to_priors-foreach-wait5 (/<38>:aggNewIs (aidi_EnKF:params_to_priors():59:4)/)<br>
{18} params_to_priors-call-size0 (/<39>:ens_priors_transposed[0] (aidi_EnKF:params_to_priors():77:4)/)<br>
{19} params_to_priors-call_foreign-size-1 (/<39>:ens_priors_transposed (aidi_EnKF:params_to_priors():77:4)/)<br>
{22} posteriors_to_params-call-size0 (/<27>:__t:67[0] (aidi_EnKF:main():186:5)/)<br>
{27} priors_to_posteriors-call-size0 (/<25>:__t:61[0] (aidi_EnKF:main():184:8)/)<br>
{28} priors_to_posteriors-call_foreign-size-1 (/<25>:__t:61 (aidi_EnKF:main():184:8)/)<br>
{29} priors_to_posteriors-call_sync-blob_from_floats (/<46>:one_dim_ens_priors (aidi_EnKF:priors_to_posteriors():123:1)/)<br>
{30} shape_one_dim-async_op-minus_int (/<44>:rows (aidi_EnKF:priors_to_posteriors():121:3)/)<br>
{31} shape_one_dim-async_op-minus_int-1 (/<45>:cols (aidi_EnKF:priors_to_posteriors():122:1)/)<br>
{32} flatten_two_dim-call-size0 (/<25>:__t:61[0] (aidi_EnKF:main():184:8)/)<br>
{33} flatten_two_dim-call_foreign-size-1 (/<25>:__t:61 (aidi_EnKF:main():184:8)/)<br>
{34} posteriors_to_params-wait-range3 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{36} posteriors_to_params-wait-range3 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{44} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{45} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{46} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{47} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{48} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{49} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{50} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{51} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{52} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{53} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{54} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{55} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{56} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{57} posteriors_to_params-wait-range1 (/<43>:__t:6 (aidi_EnKF:posteriors_to_params():138:30)/)<br>
{535} params_to_priors-call_sync-string_join (/<215>:ips (aidi_EnKF:params_to_priors():48:8)/)<br>
{538} params_to_priors-call_sync-string_join (/<217>:ips (aidi_EnKF:params_to_priors():48:8)/)<br>
{551} params_to_priors-async_op-plus_int (<218>:offset (aidi_EnKF:params_to_priors():52:8) /<14>:float_var_names[-14] (aidi_EnKF:main():169:4)/)<br>
{552} params_to_priors-async_op-plus_int (<218>:offset (aidi_EnKF:params_to_priors():52:8) /<14>:float_var_names[-13] (aidi_EnKF:main():169:4)/)<br>
{553} params_to_priors-async_op-plus_int (<216>:offset (aidi_EnKF:params_to_priors():52:8) /<14>:float_var_names[-14] (aidi_EnKF:main():169:4)/)<br>
{554} params_to_priors-async_op-plus_int (<216>:offset (aidi_EnKF:params_to_priors():52:8) /<14>:float_var_names[-13] (aidi_EnKF:main():169:4)/)<br>
UNFILLED CONTAINER SUBSCRIBE <27>:__t:67[0] (aidi_EnKF:main():186:5) rank: 2 work_type: 0<br>
UNFILLED CONTAINER SUBSCRIBE <14>:float_var_names[-13] (aidi_EnKF:main():169:4) rank: 2 work_type: 0<br>
UNFILLED CONTAINER SUBSCRIBE <25>:__t:61[0] (aidi_EnKF:main():184:8) rank: 2 work_type: 0<br>
UNFILLED CONTAINER SUBSCRIBE <14>:float_var_names[-14] (aidi_EnKF:main():169:4) rank: 2 work_type: 0<br>
UNFILLED CONTAINER SUBSCRIBE <39>:ens_priors_transposed[0] (aidi_EnKF:params_to_priors():77:4) rank: 2 work_type: 0<br>
FAILED: EXIT(1)<br>
<br>
===================================================================================<br>
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES<br>
= PID 66444 RUNNING AT XioMBP.local<br>
= EXIT CODE: 1<br>
= CLEANING UP REMAINING PROCESSES<br>
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES<br>
===================================================================================<br>
_______________________________________________<br>
ExM-user mailing list<br>
<a href="mailto:ExM-user@lists.mcs.anl.gov" target="_blank">ExM-user@lists.mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>