[Swift-devel] staging time to sites from log

[David Kelly]

My suspicion is that since you're using ssh-cl, the relevant lines related
to staging in and staging out are getting buried in
~/.globus/coasters/coasters.log on the SSH site. In the test I did was
running with local:condor staging in and out directly to OSG nodes.


On Wed, Jan 22, 2014 at 3:39 PM, Ketan Maheshwari <ketan at mcs.anl.gov> wrote:

> In the log I have from Swift trunk swift-r7228 cog-r3817, I do not see
> 'sending data' messages.
>
>
> On Wed, Jan 22, 2014 at 3:09 PM, David Kelly <davidkelly at uchicago.edu>wrote:
>
>> I was just looking more closely through a log I just created that used
>> provider staging. I think there is some info about timings there that look
>> like this:
>>
>> 2014-01-22 14:11:50,249-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
>> source: ./hostname-run000/jobs/a/hostname-azak6dll/hostfile.out
>> 2014-01-22 14:11:50,249-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
>> destination:
>> proxy://u2086f356-143bb93f2a2--7fff-u2086f356-143bb93f2a2--8000S//home/davidk/test/./hostfile.out
>> 2014-01-22 14:11:50,251-0600 INFO  PutFileCommand Sending Command(tag:
>> 10, PUT) (t) on cpipe://1
>> 2014-01-22 14:11:50,251-0600 INFO  PutFileCommand Command(tag: 10, PUT)
>> (t) sending data
>> 2014-01-22 14:11:50,251-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
>> -> 10
>> 2014-01-22 14:11:50,251-0600 INFO  PutFileHandler Handler(tag: 10, PUT)
>> source: ./hostname-run000/jobs/a/hostname-azak6dll/hostfile.out
>> 2014-01-22 14:11:50,251-0600 INFO  PutFileHandler Handler(tag: 10, PUT)
>> destination: file://localhost//home/davidk/test/hostfile.out
>> 2014-01-22 14:11:50,252-0600 INFO  PutFileHandler Handler(tag: 10, PUT)
>> Transfer done
>> 2014-01-22 14:11:50,252-0600 INFO  RequestHandler Handler(tag: 10, PUT)
>> unregistering (send)
>> 2014-01-22 14:11:50,252-0600 INFO  PutFileHandler Handler(tag: 7644, PUT)
>> Transfer done
>> 2014-01-22 14:11:50,253-0600 INFO  RequestHandler Handler(tag: 7644, PUT)
>> unregistering (send)
>>
>> Maybe you could keep track of the tag number and record the difference
>> from "sending data" to "Transfer done" to get an estimate. I'm not sure how
>> to correlate the file transfers to a jobid though.
>>
>>
>> On Wed, Jan 22, 2014 at 2:54 PM, Ketan Maheshwari <ketan at mcs.anl.gov>wrote:
>>
>>> Hi,
>>>
>>> Is it possible to find staging in and out times per job with Swift logs
>>> with provider staging?
>>>
>>> Speaking with David about this, he mentioned it is not currently
>>> possible with provider staging but there may be some options in log4j that
>>> could achieve it.
>>>
>>> Does anyone know about any such options or any indirect methods?
>>>
>>> Thanks,
>>> Ketan
>>>
>>> _______________________________________________
>>> Swift-devel mailing list
>>> Swift-devel at ci.uchicago.edu
>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>
>>>
>>
>> _______________________________________________
>> Swift-devel mailing list
>> Swift-devel at ci.uchicago.edu
>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20140122/88059410/attachment.html>