[Swift-devel] [Fwd: re: IO overheads of swift wrapper scripts on BlueGene/P (fwd)]

[Mihael Hategan]

This from Ben.

-------- Forwarded Message --------
From: Ben Clifford <benc at hawaga.org.uk>
To: hategan at mcs.anl.gov
Subject: re: IO overheads of swift wrapper scripts on BlueGene/P (fwd)
Date: Mon, 19 Oct 2009 03:28:07 +0000 (GMT)

i sent this earlier then i remembered that no one seems to 
moderate-approve my swift devel postings ever...

---------- Forwarded message ----------
Date: Mon, 19 Oct 2009 01:59:44 +0000 (GMT)
From: Ben Clifford <benc at hawaga.org.uk>
To: swift-devel at ci.uchicago.edu
Subject: re: IO overheads of swift wrapper scripts on BlueGene/P


swift r2970 changed copying outputs to a hard-link of outputs to improve 
performance at the output stage. you might be interested to see how that 
works.

(also, don't use the same runid for different runs. you'll get into 
trouble somehow sooner or later. )


r2970 | benc | 2009-06-22 23:40:53 +1000 (Mon, 22 Jun 2009) | 6 lines

Make wrapper use mv to stage out files to shared directory rather than cp.
This should be faster when the job directories are on the same filesystem
as the site shared cache.

(suggested by tuecke)