[ExM Users] Implicit blocking on foreach construct?

[Timothy Stitt (TGAC)]

Tim,

Thanks. Following your suggestions/example I was able to massage my own code to do what I need now. It took a couple of days working with Swift/T but the whole data-flow mindset has now kicked in and  briefly banished the typical sequential instruction thinking. It all makes sense now.

Cheers.

---

Timothy Stitt PhD / Head of Scientific Computing

The Genome Analysis Centre (TGAC)

http://www.tgac.ac.uk/


p: +44 1603 450378

e: timothy.stitt at tgac.ac.uk<mailto:timothy.stitt at tgac.ac.uk>

From: Tim Armstrong <tim.g.armstrong at gmail.com<mailto:tim.g.armstrong at gmail.com>>
Date: Tuesday, 17 March 2015 21:20
To: Timothy Stitt <timothy.stitt at tgac.ac.uk<mailto:timothy.stitt at tgac.ac.uk>>
Cc: "exm-user at lists.mcs.anl.gov<mailto:exm-user at lists.mcs.anl.gov>" <exm-user at lists.mcs.anl.gov<mailto:exm-user at lists.mcs.anl.gov>>
Subject: Re: [ExM Users] Implicit blocking on foreach construct?

Hi Tim,
  It's possible if you reformulate the problem in terms of data flow- there's no direct way in Swift to ask "is the loop finished" - but there is a way to ask "is the data being computed by the loop finished".

The "wait" construct - it lets you execute a block of code after a bit of data is assigned.

E.g. a trivial example is

int x = do_some_computation();
wait (x) {
  trace("x is done!");
}

If you're building an array of files or similar, it's a little more complicated:

import io;
import files;

file x[];

wait deep (x) {
  trace("Done");
}

foreach i in [1:10] {
  file tmp<"/tmp/swift-tmp-" + i + ".txt">;

  tmp = write("");

  x[i] = tmp;
}

This only prints "Done" once all of the files have been written.  The wait deep ensures that everything in the array is completely built.

I'm also not sure about your use case, but Swift has some built-in support for generating temporary files - Justin Wozniak was recently working on improving that.

- Tim

On 17 March 2015 at 15:08, Timothy Stitt (TGAC) <Timothy.Stitt at tgac.ac.uk<mailto:Timothy.Stitt at tgac.ac.uk>> wrote:
Hi all,

Is there a way to add explicit blocking to the end of the foreach construct? I have the following logic which doesn't work right now as the delete_files() routine is assumed to be concurrent with the foreach block when I want it to execute after the foreach block is completed for all workers:

foreach { do some work creating temp files}
delete_files() // delete temp files using an app function

Thanks,

Tim.

---

Timothy Stitt PhD / Head of Scientific Computing

The Genome Analysis Centre (TGAC)

http://www.tgac.ac.uk/


p: +44 1603 450378<tel:%2B44%201603%20450378>

e: timothy.stitt at tgac.ac.uk<mailto:timothy.stitt at tgac.ac.uk>

_______________________________________________
ExM-user mailing list
ExM-user at lists.mcs.anl.gov<mailto:ExM-user at lists.mcs.anl.gov>
https://lists.mcs.anl.gov/mailman/listinfo/exm-user


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/exm-user/attachments/20150319/40bd4f33/attachment.html>