[Swift-devel] ram disk with swift

[David Kelly]

Here is a portion of a script that that copies some frequently used data to a node's shared memory. I think it is pretty similar to the script that Glen sent earlier.

dssatdir=/dev/shm/dssat
dssattmp=/dev/shm/dssat.tmp

if [ ! -d $dssatdir ]; then
  if mkdir $dssattmp 2>/dev/null; then
    echo $$ copying
    savewd=$PWD
    cd $dssattmp

    mkdir -p refdata
    cp $refdata/* refdata/

    mkdir -p campaign
    cp $campaign/*.MZX campaign/

    mkdir -p binpath
    cp $binpath/*.EXE binpath/

    cd $savewd
    mv $dssattmp $dssatdir
  else
    while [ ! -d $dssatdir ]; do
     # echo $$ sleeping
      sleep 1;
    done
  fi
fi



----- Original Message -----
> From: "Lorenzo Pesce" <lpesce at uchicago.edu>
> To: "Michael Wilde" <wilde at mcs.anl.gov>, "Swift Devel" <swift-devel at ci.uchicago.edu>, swift-user at ci.uchicago.edu
> Cc: "Kazutaka Takahashi" <kazutaka at gmail.com>
> Sent: Friday, June 8, 2012 2:39:19 PM
> Subject: [Swift-devel] ram disk with swift
> Mike,
> We were talking about putting the input files for Taka's calculations
> (the X.mat files), which are rather small ~ 60Kb or so, in ram disk
> and try to use them for all the apps (that use the same file). giving
> the size, we could even load up all of them for each node
> (7*3*60=1.2MB, that is zero) to reduce traffic towards /lustre and
> thus avoid the meta-data slow downs that Beagle seems to have.
> 
> If we copy files in /tmp (as far as I understand it would be ramdisk)
> on the each node through swift, would it be there for all the apps?
> 
> Thanks,
> 
> Lorenzo
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel