[Swift-devel] Including app resource utilization in the info file

[Michael Wilde]

Hi Luiz, All,

I'd like to incorporate the mod Luiz made in the provenance prototype to capture app() resource utilization. This is in svn at:

https://svn.ci.uchicago.edu/svn/vdl2/provenancedb/swift_mod/_swiftwrap_runtime_aggregate

Im testing a version of this on midway that unconditionally runs every app invocation under /usr/bin/time (if it exists), and captures all of the resource data in an info log line in a single string of the form:

APP_RESOURCES=maxrss:1837872,walltime:1027.64,systime:62.77,usertime:3217.07,cpu:319%,fsin:0,fsout:116232,timesswapped:0,socketrecv:0,socketsent:0,majorpagefaults:21,minorpagefaults:4232202,contextswitchesinv:4624,contextswitchesvol:78762

Luiz also has code which periodically samples the app invocation's resource utilization to build a profile of its behavior over time. I'd like to integrate that as an option at a later date (and make the style of resource capture selectable: aggregate on/off; polled on/off).

Luiz: note that I changed the name; I also changed the implementation so that the above line is captured in a file in the job dir called swift.resources instead of appended to the stderr file. (This way its less intrusive to the app).

I'd like to make a few changes to the format to closely track the names and order of the resource counters that are listed in the Linux man page.

Can we do this in a way that keeps the prototype provenance code in sync and functional with this change?

Also, on Midway, at the moment /usr/bin/time doesnt exist. If thats the case then _swiftwrap tries to see if there is a $HOME/swift.time command and runs that instead. I was able to just copy a /usr/bin/time executable from another similar Linux system, and it worked.

- Mike

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory