[Swift-devel] Coaster provider staging data xfer problem

[wilde at mcs.anl.gov]

Hard to say; probably not on individual filesize; perhaps on cumulative size?

The file sizes are in a fixed directory, generated once (for all the tests Ive done)
sizes range from about 1K to 300K, randomly generated using $RANDOM:

10 smallest:

911
976
1752
1856
2236
2380
2612
2695
3087

10 largest:

355783
356035
356368
356402
356543
356601
357104
357117
358513

The files were generated with:

dd if=/home/wilde/swift/lab/48M of=data bs=1M
for i in $(seq -f %04g 0 9999) ; do
  dd if=data of=f.$i.in bs=$(($RANDOM*10+$RANDOM)) count=1
done


- Mike

----- "Mihael Hategan" <hategan at mcs.anl.gov> wrote:

> On Sun, 2010-10-03 at 20:20 -0600, Michael Wilde wrote:
> > Interesting: at 5000 jobs, the run completes normally; at 10,000 it
> > fails again, as before. (I adjusted the script slightly to give
> 10,000
> > jobs instead of 9999)
> 
> Strange indeed. Does it depend on file size?

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory