[Swift-devel] Work directory on a non-shared filesystem

[Michael Wilde]

This sounds great. Provide staging with no actual data staging at least allows local fs to be used on both the submit-node and worker-node, and gets _swiftwrap transported with node code changes.

David, how does the scaling look? Do you have any plots for full 55K-cell DSSAT runs?

- Mike

----- Original Message -----
> From: "David Kelly" <davidk at ci.uchicago.edu>
> To: "Mihael Hategan" <hategan at mcs.anl.gov>
> Cc: "swift-devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, February 7, 2013 10:20:50 PM
> Subject: Re: [Swift-devel] Work directory on a non-shared filesystem
> 
> 
> 
> Thanks for the info Mihael. Right now, I am seeing really good
> performance using /tmp and provider staging, so I'm happy to stick
> with that. It's been pretty consistent as I scale up in my tests.
> I'll send you all the plots/benchmarks when I am finished.
> 
> 
> 
> 
> 
> From: "Mihael Hategan" <hategan at mcs.anl.gov>
> To: "David Kelly" <davidk at ci.uchicago.edu>
> Cc: "swift-devel" <swift-devel at ci.uchicago.edu>
> Sent: Thursday, February 7, 2013 9:43:02 PM
> Subject: Re: [Swift-devel] Work directory on a non-shared filesystem
> 
> _swiftwrap needs to make it to the compute nodes somehow.
> 
> The scenarios you list below make sense:
> 1. work dir on shared fs, scratch on /tmp: _swiftwrap is accessed
> through the shared fs so it works.
> 2. provider staging with workdir and scratch on /tmp: _swiftwrap is
> staged in by coasters, so that works too.
> 3. no provider staging, workdir and scratch on /tmp: _swiftwrap has
> no
> way of making it to the compute node, so 127 (command/function not
> found
> - see COMMAND EXECUTION section of man bash).
> 
> That's standard stuff. A hack would be to install _swiftwrap along
> with
> the app and convince swift to run that instead of its own swiftwrap.
> That might require hacking vdl-int.k, around line 502. Let me know if
> you want to go that route, and we can chat on how to get it done.
> 
> Mihael
> 
> On Thu, 2013-02-07 at 20:10 -0600, David Kelly wrote:
> > Is it possible to have a work directory on a non-shared filesystem
> > (/tmp) without using provider staging? I'm not staging any files in
> > or
> > out with swift, just passing strings. When I try this, I get an 127
> > app failure for some reason (the app is /bin/bash).
> > 
> > 
> > I've been able to set the work directory to a shared filesystem,
> > and
> > set the scratch tag to /tmp. That combination seems to work fine.
> > But
> > ideally I'd like to avoid the shared filesystem as much as
> > possible.
> > 
> > 
> > It also seems to work fine with /tmp as the work directory and with
> > provider staging enabled.
> > 
> > 
> > Just curious if this configuration is possible somehow. Thanks!
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > David Michael Kelly
> > Systems Programmer
> > University of Chicago Computation Institute
> > 5735 S. Ellis Ave. Chicago, IL 60637
> > Is it possible to have a work directory on a non-shared filesystem
> > (/tmp) without using provider staging? I'm not staging any files in
> > or
> > out with swift, just passing strings. When I try this, I get an 127
> > app failure for some reason (the app is /bin/bash).
> > 
> > 
> > I've been able to set the work directory to a shared filesystem,
> > and
> > set the scratch tag to /tmp. That combination seems to work fine.
> > But
> > ideally I'd like to avoid the shared filesystem as much as
> > possible.
> > 
> > 
> > It also seems to work fine with /tmp as the work directory and with
> > provider staging enabled.
> > 
> > 
> > Just curious if this configuration is possible somehow. Thanks!
> > 
> > 
> > 
> > 
> > 
> > David Michael Kelly
> > Systems Programmer
> > University of Chicago Computation Institute
> > 5735 S. Ellis Ave. Chicago, IL 60637
> > 
> > 
> > _______________________________________________
> > Swift-devel mailing list
> > Swift-devel at ci.uchicago.edu
> > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
> 
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>