[Swift-devel] hi

[Vipul Kumar Singh]

hi,

I am trying this little experiment .


Note 1) A List is maintained that contains FileName - PathWhereFileIsStored
- Count
    (When count is zero record is removed from list)

Note 2) Every site has a shared_directory

For all the Stage_In files required for a job DO
check if file is in List
IF file in List then
DO NOT copy file, rather in the temporary directory of this run, create Link
to the file in shared  directory AND increment count
ELSE if file is NOT in List
Copy the file to shared directory, create Link, create entry in List and set
COUNT to 1


During Stage_out for a job
IF COUNT value of the file is 1
remove file from the list
ELSE
decrement COUNT for the file


I still don't understand a large part of the work flow.

Going through the code i am not able to figure out where exactly the files
required to be staged_in for a particular job is determined.

The above case requires the data files to be present in shared_directory
only when a job in need of those files is being executed.The above logic
should work best for cases when a series of job dependent on same files are
submitted to the scheduler.

Though this experiment is not very useful in this form, i will be trying to
run several swiftscripts from different terminals simultaneously and see if
the logic is working and then later try to extend it to case where the
swiftscripts are run by different users.

And instead of  LIST  can we use LRUFileCache ?

Thank You
Vipul Kumar Singh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20100404/a0303b66/attachment.html>