[Swift-user] Optimizing Swift overhead of file copy - reg.

Karthikeyan Balasubramanian karthikeyanb at uchicago.edu
Wed Feb 11 11:34:22 CST 2015


Hi,

When jobs are run in Swift, it copies the data from the original location to the work folder for processing. Since our data are in the order of 15-30 GB, it adds unwanted overhead for the process. I recall from my early discussions with Michael Wilde that this could be optimized so that the files can be processed from their original location.

I am wondering as how should I proceed to achieve this optimization. Your inputs are greatly appreciated.

Thanks.
B.K.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20150211/18d67ddb/attachment.html>


More information about the Swift-user mailing list