[Swift-devel] disk space requirement

[Mike Wilde]

Nika, regarding where you can get 244GB, we should send in a request 
to CI Support to get 1TB or so from the CI SAN mounted on a Swift 
submit host.

In the interim:

- you can get 100GB or so on NFS on terminable

- we can try to quickly add a 300GB drive (or two) to terminable

- One of the gridlab hosts (4 I think) should be available with that 
storage now. I am not sure if we feel this lab host is stable yet 
but we should make sure.

We need to be gearing up quickly to this scale, so the lab 
environment that Ive been pushing to get set up is going to be 
increasingly important.

Regarding the core limitation that Nika points out - this seems to 
dictate that we take the step of allowing workflow inputs, outputs 
and intermediate results to be located on any gridftp server and 
tracked via a replica catalog, as VDS did.

Ben, Mihael, do you have a feel for where we can slot this into 
development? v3 - say circa September?

Mike


Veronika Nefedova wrote, On 6/1/2007 8:19 AM:
> Hi,
> 
> I know I raised this question many times before, but I think I need a 
> solution to it very soon (=now). I have a machinery in place to do a 
> really big runs for MolDyn. Currently, the workflow produces about 0.9GB 
> of data for each molecule, but this is all the intermediate data, all I 
> need is *one* 300K file as a result (per molecule). The rest is 
> intermediate data that I do not need.  So my questions are:
> 
> 1. How can I eliminate the intermediate results staging back to my 
> submit host? I do not need it in case of one remote compute pool. VDS 
> had this feature...
> 
> 2. If implementing this feature is very hard and time-consuming -- what 
> submit host would you recommend for a 244-molecule run ? Roughly 244 GB 
> of space is needed.
> 
> A 20-molecule run is all I can do on wiggum (where the application code 
> is currently compiled).
> 
> Thanks!
> 
> Nika
> 
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
> 

-- 
Mike Wilde
Computation Institute, University of Chicago
Math & Computer Science Division
Argonne National Laboratory
Argonne, IL   60439    USA
tel 630-252-7497 fax 630-252-1997