[Swift-devel] Need help using Data Capacitor in Swift Workflow

[Michael Wilde]

Mike, did you already do a space estimate for the upcoming workflows?
If not, can we help?

Once we have DC access (via certs), its just a GridFTP server, so its 
not much more work than just adding a new storage site.

Files in the DC have "term limits" of either 2 or 4 weeks of non-use 
before they are removed (depending on whether we use scratch or 
persistent directory areas).

The only extra work will be watching the space and deciding when things 
need to move to a tape archive (also via GridFTP, possibly using tgcp)

We will look into getting more space in the CI SAN, but need to find 
funds and place orders.  This will take some time. Space in the CI SAN 
is not reserved, so if we buy mre disks, we need to use placeholder 
files to hold our space. Thats yet more management work.

The DC space will always be handy to have access to, so I think we're 
doing the right thing by pushing it forward.

Turns out my app to TG Help for DC space got bounced back a week agon 
and I didnt notice. I just refiled it directly with the DC group.
Im hoping to hear in a day or two.  I *think* we can test against the DC 
scratch space with an ordinary TG account. Thats what I was testing a 
week ago before I started traveling. Im hoping Mihael can take over on this.


- Mike










On 1/30/08 10:32 AM, Mike Kubal wrote:
> Yes, definitely need the space and assistance modifying scripts to 
> utilize data capacitor at UI (  if for no other reason that it sounds 
> cools to say "I'm using a data capacitor" : )  ).
> 
> 
> ----- Original Message ----
> From: Michael Wilde <wilde at mcs.anl.gov>
> To: Mike Kubal <mikekubal at yahoo.com>; Mihael Hategan <hategan at mcs.anl.gov>
> Cc: swift-devel <swift-devel at ci.uchicago.edu>
> Sent: Wednesday, January 30, 2008 10:23:00 AM
> Subject: [Swift-devel] Need help using Data Capacitor in Swift Workflow
> 
> [was: Re: Store files in IU Data Capacitor?]
> 
> A week ago I started to help MikeK find several TB of free space to
> support his next round of workflows.
> 
> Currently the CI SAN has only 700GB free.
> 
> I started looking at how to use the 500TB data capacitor server at IU.
> 
> MikeK, should we assume you still need this, test access to it, and give
> you the parameters for your sites file, and swift examples of mapping
> data to/from GridFTP URIs?
> 
> If so, Mihael can you do this? (URL for the DC is below. Can use scratch
> space for 2-week stretches. Ive applied for an allocation on it).
> 
> - Mike
> 
> 
> -------- Original Message --------
> Subject: Re: Store files in IU Data Capacitor?
> Date: Mon, 21 Jan 2008 14:42:48 -0800 (PST)
> From: Mike Kubal <mikekubal at yahoo.com <mailto:mikekubal at yahoo.com>>
> To: Michael Wilde <wilde at mcs.anl.gov <mailto:wilde at mcs.anl.gov>>,  
> "Binkowski, Thomas Andrew"
> <abinkowski at anl.gov <mailto:abinkowski at anl.gov>>
> 
> 10TB would be plenty.
> 
> I've been using Benoit's allocation.
> 
> The TG project ID I've been using is:
> 
> TG-MCA01S018
> 
> Thanks MikeW,
> 
> MikeK
> 
> --- Michael Wilde <wilde at mcs.anl.gov <mailto:wilde at mcs.anl.gov>> wrote:
> 
>  > Hi Andrew and Mike,
>  >
>  > One place we might be able to keep our multi-GB data
>  > files is in the
>  > IndianaU Data Capacitor:
>  > 
>  >
> http://datacapacitor.researchtechnologies.uits.iu.edu/
>  >
>  > Do you have a rough estimate of what we need?  the
>  > DC allocates storage
>  > in 10TB chunks, which I assume will meet our needs
>  > for the moment?
>  >
>  > Can you send me the TG project # for the LigandAtlas
>  > project, and I will
>  > try to get an allocation of space there?
>  >
>  > Are we running on our own TG DAC, or on Benoit's
>  > large allocation?
>  >
>  > Thanks,
>  >
>  > Mike
>  >
> 
> 
> 
> 
> ____________________________________________________________________________________
> Never miss a thing.  Make Yahoo your home page.
> http://www.yahoo.com/r/hs
> 
> 
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu <mailto:Swift-devel at ci.uchicago.edu>
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
> 
> 
> 
> ------------------------------------------------------------------------
> Never miss a thing. Make Yahoo your homepage. 
> <http://us.rd.yahoo.com/evt=51438/*http://www.yahoo.com/r/hs>