[Swift-user] Coordination between app calls in foreach loop
Mihael Hategan
hategan at mcs.anl.gov
Sat Sep 21 00:02:45 CDT 2013
Wait, I take it back. This works as designed.
There is the assumption that the assignment art[i] = i must happen after
A("astring") completes, but this is not true.
What happens is that all the iterations in the first foreach are
launched in parallel, and the invocation of A() and the assignment
art[i] = i are also launched in parallel. This effectively means that
all the art[i] assignments will happen very quickly, and before any
long-running A invocations complete.
If there are hidden dependencies between apps that are not exposed to
swift as files, then use the external type:
type file;
app (file out, external ex) A (string in1){
dummyA in1 stdout=@out;
}
app (file out) B (string in1, external[] _art){
dummyB in1 stdout=@out;
}
file outfileA[]<simple_mapper; location="outdir",
prefix="a.",suffix=".out">;
file outfileB[]<simple_mapper; location="outdir",
prefix="b.",suffix=".out">;
external art[];
foreach i in [0:3]{
(outfileA[i], art[i]) = A ("astring");
}
foreach j in [0:3]{
outfileB[j] = B ("bstring", art);
}
Mihael
On Fri, 2013-09-20 at 17:10 -0500, Ketan Maheshwari wrote:
> I have filed this as Swift bug:
>
> https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1099
>
>
> On Fri, Sep 20, 2013 at 4:37 PM, Mihael Hategan <hategan at mcs.anl.gov> wrote:
>
> > I can confirm this bug. Apps don't properly wait for arrays.
> >
> > Mihael
> >
> > On Fri, 2013-09-20 at 14:08 -0500, Ketan Maheshwari wrote:
> > > Hi,
> > >
> > > I am working with APS user Hemant Sharma (in CC) on one of his projects.
> > > Hemant wrote a Swift script which has about three foreach invocations
> > which
> > > he wants to run one after other.
> > >
> > > In order to achieve this coordination, Hemant is using an int[] in one
> > loop
> > > which is passed as input to the app call of another loop. He observes
> > that
> > > the coordination is not maintained and both app calls happen
> > simultaneously
> > > resulting in application error.
> > >
> > > Here is a quick reproduction I tried and I observe indeed that
> > coordination
> > > is not happening:
> > >
> > > $ cat dummy.swift
> > > type file;
> > >
> > > app (file out) A (string in1){
> > > dummyA in1 stdout=@out;
> > > }
> > >
> > > app (file out) B (string in1, int[] _art){
> > > dummyB in1 stdout=@out;
> > > }
> > >
> > > file outfileA[]<simple_mapper; location="outdir",
> > > prefix="a.",suffix=".out">;
> > > file outfileB[]<simple_mapper; location="outdir",
> > > prefix="b.",suffix=".out">;
> > >
> > > int art[];
> > > foreach i in [0:3]{
> > > outfileA[i] = A ("astring");
> > > art[i]=i;
> > > }
> > >
> > > foreach j in [0:3]{
> > > outfileB[j] = B ("bstring", art);
> > > }
> > >
> > > Since the art array is wholly required in app B, one would expect the
> > first
> > > foreach to finish befor the call to B gets invoked. This is not happening
> > > in my simple test where I ask the executable called by A to sleep for 5
> > sec
> > > and that called by B to print timestamp and quit.
> > >
> > > I see in the resulting files that the timestamps in B precede those in A.
> > >
> > > This looks like a bug to me. Swift is 0.94 swift-r7091 cog-r3789.
> > >
> > > Thanks,
> > > Hi,
> > >
> > >
> > > I am working with APS user Hemant Sharma (in CC) on one of his
> > > projects. Hemant wrote a Swift script which has about three foreach
> > > invocations which he wants to run one after other.
> > >
> > >
> > > In order to achieve this coordination, Hemant is using an int[] in one
> > > loop which is passed as input to the app call of another loop. He
> > > observes that the coordination is not maintained and both app calls
> > > happen simultaneously resulting in application error.
> > >
> > >
> > > Here is a quick reproduction I tried and I observe indeed that
> > > coordination is not happening:
> > >
> > > $ cat dummy.swift
> > > type file;
> > >
> > > app (file out) A (string in1){
> > > dummyA in1 stdout=@out;
> > > }
> > >
> > > app (file out) B (string in1, int[] _art){
> > > dummyB in1 stdout=@out;
> > > }
> > >
> > > file outfileA[]<simple_mapper; location="outdir",
> > > prefix="a.",suffix=".out">;
> > > file outfileB[]<simple_mapper; location="outdir",
> > > prefix="b.",suffix=".out">;
> > >
> > > int art[];
> > > foreach i in [0:3]{
> > > outfileA[i] = A ("astring");
> > > art[i]=i;
> > > }
> > >
> > > foreach j in [0:3]{
> > > outfileB[j] = B ("bstring", art);
> > > }
> > >
> > >
> > > Since the art array is wholly required in app B, one would expect the
> > > first foreach to finish befor the call to B gets invoked. This is not
> > > happening in my simple test where I ask the executable called by A to
> > > sleep for 5 sec and that called by B to print timestamp and quit.
> > >
> > >
> > > I see in the resulting files that the timestamps in B precede those in
> > > A.
> > >
> > >
> > > This looks like a bug to me. Swift is 0.94 swift-r7091 cog-r3789.
> > >
> > >
> > >
> > > Thanks,
> > >
> > > _______________________________________________
> > > Swift-user mailing list
> > > Swift-user at ci.uchicago.edu
> > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
> >
> >
> >
>
>
> I have filed this as Swift bug:
>
> https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1099
>
>
>
> On Fri, Sep 20, 2013 at 4:37 PM, Mihael Hategan <hategan at mcs.anl.gov>
> wrote:
> I can confirm this bug. Apps don't properly wait for arrays.
>
> Mihael
>
> On Fri, 2013-09-20 at 14:08 -0500, Ketan Maheshwari wrote:
> > Hi,
> >
> > I am working with APS user Hemant Sharma (in CC) on one of
> his projects.
> > Hemant wrote a Swift script which has about three foreach
> invocations which
> > he wants to run one after other.
> >
> > In order to achieve this coordination, Hemant is using an
> int[] in one loop
> > which is passed as input to the app call of another loop. He
> observes that
> > the coordination is not maintained and both app calls happen
> simultaneously
> > resulting in application error.
> >
> > Here is a quick reproduction I tried and I observe indeed
> that coordination
> > is not happening:
> >
> > $ cat dummy.swift
> > type file;
> >
> > app (file out) A (string in1){
> > dummyA in1 stdout=@out;
> > }
> >
> > app (file out) B (string in1, int[] _art){
> > dummyB in1 stdout=@out;
> > }
> >
> > file outfileA[]<simple_mapper; location="outdir",
> > prefix="a.",suffix=".out">;
> > file outfileB[]<simple_mapper; location="outdir",
> > prefix="b.",suffix=".out">;
> >
> > int art[];
> > foreach i in [0:3]{
> > outfileA[i] = A ("astring");
> > art[i]=i;
> > }
> >
> > foreach j in [0:3]{
> > outfileB[j] = B ("bstring", art);
> > }
> >
> > Since the art array is wholly required in app B, one would
> expect the first
> > foreach to finish befor the call to B gets invoked. This is
> not happening
> > in my simple test where I ask the executable called by A to
> sleep for 5 sec
> > and that called by B to print timestamp and quit.
> >
> > I see in the resulting files that the timestamps in B
> precede those in A.
> >
> > This looks like a bug to me. Swift is 0.94 swift-r7091
> cog-r3789.
> >
> > Thanks,
> > Hi,
> >
> >
> > I am working with APS user Hemant Sharma (in CC) on one of
> his
> > projects. Hemant wrote a Swift script which has about three
> foreach
> > invocations which he wants to run one after other.
> >
> >
> > In order to achieve this coordination, Hemant is using an
> int[] in one
> > loop which is passed as input to the app call of another
> loop. He
> > observes that the coordination is not maintained and both
> app calls
> > happen simultaneously resulting in application error.
> >
> >
> > Here is a quick reproduction I tried and I observe indeed
> that
> > coordination is not happening:
> >
> > $ cat dummy.swift
> > type file;
> >
> > app (file out) A (string in1){
> > dummyA in1 stdout=@out;
> > }
> >
> > app (file out) B (string in1, int[] _art){
> > dummyB in1 stdout=@out;
> > }
> >
> > file outfileA[]<simple_mapper; location="outdir",
> > prefix="a.",suffix=".out">;
> > file outfileB[]<simple_mapper; location="outdir",
> > prefix="b.",suffix=".out">;
> >
> > int art[];
> > foreach i in [0:3]{
> > outfileA[i] = A ("astring");
> > art[i]=i;
> > }
> >
> > foreach j in [0:3]{
> > outfileB[j] = B ("bstring", art);
> > }
> >
> >
> > Since the art array is wholly required in app B, one would
> expect the
> > first foreach to finish befor the call to B gets invoked.
> This is not
> > happening in my simple test where I ask the executable
> called by A to
> > sleep for 5 sec and that called by B to print timestamp and
> quit.
> >
> >
> > I see in the resulting files that the timestamps in B
> precede those in
> > A.
> >
> >
> > This looks like a bug to me. Swift is 0.94 swift-r7091
> cog-r3789.
> >
> >
> >
> > Thanks,
> >
>
> > _______________________________________________
> > Swift-user mailing list
> > Swift-user at ci.uchicago.edu
> >
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user
>
>
>
>
>
>
> --
> Ketan
>
>
More information about the Swift-user
mailing list