[Swift-devel] staging time to sites from log

[David Kelly]

I was just looking more closely through a log I just created that used
provider staging. I think there is some info about timings there that look
like this:

2014-01-22 14:11:50,249-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
source: ./hostname-run000/jobs/a/hostname-azak6dll/hostfile.out
2014-01-22 14:11:50,249-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
destination:
proxy://u2086f356-143bb93f2a2--7fff-u2086f356-143bb93f2a2--8000S//home/davidk/test/./hostfile.out
2014-01-22 14:11:50,251-0600 INFO  PutFileCommand Sending Command(tag: 10,
PUT) (t) on cpipe://1
2014-01-22 14:11:50,251-0600 INFO  PutFileCommand Command(tag: 10, PUT) (t)
sending data
2014-01-22 14:11:50,251-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
-> 10
2014-01-22 14:11:50,251-0600 INFO  PutFileHandler Handler(tag: 10, PUT)
source: ./hostname-run000/jobs/a/hostname-azak6dll/hostfile.out
2014-01-22 14:11:50,251-0600 INFO  PutFileHandler Handler(tag: 10, PUT)
destination: file://localhost//home/davidk/test/hostfile.out
2014-01-22 14:11:50,252-0600 INFO  PutFileHandler Handler(tag: 10, PUT)
Transfer done
2014-01-22 14:11:50,252-0600 INFO  RequestHandler Handler(tag: 10, PUT)
unregistering (send)
2014-01-22 14:11:50,252-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
Transfer done
2014-01-22 14:11:50,253-0600 INFO  RequestHandler Handler(tag: 7644, PUT)
unregistering (send)

Maybe you could keep track of the tag number and record the difference from
"sending data" to "Transfer done" to get an estimate. I'm not sure how to
correlate the file transfers to a jobid though.


On Wed, Jan 22, 2014 at 2:54 PM, Ketan Maheshwari <ketan at mcs.anl.gov> wrote:

> Hi,
>
> Is it possible to find staging in and out times per job with Swift logs
> with provider staging?
>
> Speaking with David about this, he mentioned it is not currently possible
> with provider staging but there may be some options in log4j that could
> achieve it.
>
> Does anyone know about any such options or any indirect methods?
>
> Thanks,
> Ketan
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20140122/411d2ba2/attachment.html>