[Swift-devel] Swift and BGP plots

[Ioan Raicu]

How about this:

iraicu at diablo:~/falkon/logs/swift/dc-4000> cat dc-4000.log | grep 
"JOB_SUBMISSION" | grep "Active"
iraicu at diablo:~/falkon/logs/swift/dc-4000> cat dc-4000.log | grep 
"JOB_SUBMISSION" | grep "Completed"

This seems to look OK to me, is this what I am looking for?

Ioan

Ioan Raicu wrote:
> OK, so this looks to be a bit more complex then. Can you suggest a way 
> to extract the Active and Complete timestamps (along with the task 
> ID)? It seems that if I grep for JOB_SUBMISSION, the status Active is 
> not found on the same line, so I can't filter according to those two 
> keywords. Also, the Completed keyword also comes up with many other 
> hits. Any ideas on how to get the info I need (with cat, grep, etc) 
> without writing some custom script/program that actually understands 
> in depth the logging of the Swift log?
>
> Thanks,
> Ioan
>
> Mihael Hategan wrote:
>> On Tue, 2009-10-27 at 14:17 -0500, Ioan Raicu wrote:
>>   
>>> OK, I converted the logs, and here is what I got.
>>>
>>> The thing that bugs me is that things ran faster (per task) at the
>>> larger scale (a bit counter intuitive).
>>>     
>>
>> Nope. Same workload but more processors. 
>>
>>   
>>>  Also, the maximum number of concurrent tasks I found was 20K, in both
>>> experiments. This doesn't seem right, as the experiments should have
>>> 4K or 6K active tasks at any time.
>>>     
>>
>> Right. You should see around 4k or 6k max active tasks.
>>
>>   
>>>  Could I be looking at the wrong entries from the Swift logs?
>>>
>>> I did this to get the data I needed from the Swift logs:
>>> cat dc-4000.log | grep "JOB_START" > dc-4000-start-end.txt
>>> cat dc-4000.log | grep "JOB_END" >> dc-4000-start-end.txt
>>>
>>> Is this correct? Or should I be looking for other job events?
>>>     
>>
>> You should probably be looking for "Task(type=JOB_SUBMISSION...) setting
>> status to Active" and "Task(type=JOB_SUBMISSION...) setting status to
>> Completed". Or something like that.
>>
>> JOB_START and JOB_END are statuses for the Swift execute2 processes.
>>
>>
>>   
>
> -- 
> =================================================================
> Ioan Raicu, Ph.D.
> NSF/CRA Computing Innovation Fellow
> =================================================================
> Center for Ultra-scale Computing and Information Security (CUCIS)
> Department of Electrical Engineering and Computer Science
> Northwestern University
> 2145 Sheridan Rd, Tech M384 
> Evanston, IL 60208-3118
> =================================================================
> Cel:   1-847-722-0876
> Tel:   1-847-491-8163
> Email: iraicu at eecs.northwestern.edu
> Web:   http://www.eecs.northwestern.edu/~iraicu/
>        https://wiki.cucis.eecs.northwestern.edu/
> =================================================================
> =================================================================
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>   

-- 
=================================================================
Ioan Raicu, Ph.D.
NSF/CRA Computing Innovation Fellow
=================================================================
Center for Ultra-scale Computing and Information Security (CUCIS)
Department of Electrical Engineering and Computer Science
Northwestern University
2145 Sheridan Rd, Tech M384 
Evanston, IL 60208-3118
=================================================================
Cel:   1-847-722-0876
Tel:   1-847-491-8163
Email: iraicu at eecs.northwestern.edu
Web:   http://www.eecs.northwestern.edu/~iraicu/
       https://wiki.cucis.eecs.northwestern.edu/
=================================================================
=================================================================


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20091027/2a1c0ab6/attachment.html>