<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
Hi Srinivasan,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
Re: 3., there is a PR here that Mathieu and I have been iterating on:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<a href="https://xgitlab.cels.anl.gov/sds/margo/merge_requests/16"> https://xgitlab.cels.anl.gov/sds/margo/merge_requests/16</a></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
... and you can also install it as part of a spack install by using the<span style="color: rgb(255, 255, 255)"><span style="color: black"> @develop-rpc-breadcrumb version of margo.<br>
</span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span style="color: rgb(255, 255, 255)"><span style="color: black"><br>
</span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span style="color: rgb(255, 255, 255)"><span style="color: black"> spack install mobject ^margo@develop-rpc-breadcrumb</span>iobject ^margo@develop-rpc-breadcrumb</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
I don't have documentation yet (this week got away from me), but the short version is that you can activate it by calling margo_diag_start() and then display simple cumulative statistics about all RPCs issued by that margo instance by running margo_diag_dump().
The programs in the examples/ subdirectory do this so you could see a quick example. Each RPC type gets its own min/max/avg/count counters, and RPC identifiers are recorded as a stack so that you get separate counters for RPC x() if it was triggered by RPC
bar() or RPC foo(). This is important for our composition because there maybe be multiple ways to trigger an RPC on an internal component.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
The output is not user friendly; this PR is just experimenting with an instrumentation mechanism rather than any storage or analysis of that instrumentation. The internal margo_breadcrumb_measure() function is where the cumulative counters are updated right
now.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
The thing that's dapper-like about it is that it is timing RPCs and chains of RPCs. The thing that's un-dapper like about it is that it is accumulating aggregate statistics rather than storing individual measurements, and there is no sampling mechanism.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
-Phil<br>
</div>
<div></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
</div>
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> mochi-devel <mochi-devel-bounces@lists.mcs.anl.gov> on behalf of Dorier, Matthieu via mochi-devel <mochi-devel@lists.mcs.anl.gov><br>
<b>Sent:</b> Friday, June 28, 2019 7:44 AM<br>
<b>To:</b> Srinivasan Ramesh; mochi-devel@lists.mcs.anl.gov<br>
<b>Subject:</b> Re: [Mochi-devel] An overview of application usage scenarios</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi Srinivasan,<br>
<br>
Glad that the tutorials were useful! Regarding your questions:<br>
<br>
1.a: HEPnOS is intended to be deployed separately from the application. The physicists we are working with deploy it in a set of containers. We envision leaving it running for the duration of an experimental campaign (which may be weeks to months). FlameStore
is supposed to act like a cache for a single workflow. It lives for the duration of the workflow. As for SDSDKV, I'm not too familiar with that one but I think it's supposed to be deployed for the duration of an application as well.<br>
<br>
1.b: HEPnOS is not part of the application (though it could be deployed as part of it). FlameStore is part of the application (same MPI_COMM_WORLD). For SDSDKV I don't know.<br>
<br>
1.c: Yes, HEPnOS is intended to be long-running.<br>
<br>
2: I wish we had such a code; our physicists colleagues are working on it right now.<br>
<br>
Thanks,<br>
<br>
Matthieu<br>
<br>
On 27/06/2019, 17:31, "mochi-devel on behalf of Srinivasan Ramesh via mochi-devel" <mochi-devel-bounces@lists.mcs.anl.gov on behalf of mochi-devel@lists.mcs.anl.gov> wrote:<br>
<br>
Hi team,<br>
<br>
@Mattheiu: Thanks for the wiki tutorial for Mochi. I found it extremely <br>
useful for my understanding and tried out the hands-on tutorials.<br>
<br>
I re-read the PDSW "Methodology for rapid development..." paper and <br>
installed HePNOS locally on my laptop. A few questions come to mind:<br>
<br>
1. For each of the popular data-services mentioned in the paper <br>
(Framestore, HePNOS, SDSKV), what is the model of usage/topology? <br>
Specifically:<br>
a. Are these services part of a workflow? Meaning, a node allocation <br>
is managed, and the services are long-running for the duration of the <br>
workflow. Jobs within the workflow come and go, and use the service <br>
during their execution.<br>
b. Are these services part of the application itself? Meaning a <br>
"regular" MPI job where the service is built into each MPI process and <br>
loaded as a library local to the process.<br>
c. Is it possible that certain services are long-running on the <br>
system "forever" (reduces to (a) I guess?)<br>
<br>
The methodology paper hints at the topology but doesn't really provide a <br>
concrete description. With regard to performance measurement,<br>
I am fully aware that data-services can span the entire range of <br>
possibilities. However, I think it may not be a bad idea to start with <br>
specific scenarios in mind and then go from there onto more general <br>
cases once we have a grasp on the problem.<br>
<br>
2. Can I get access to a high-energy physics code that actually uses the <br>
HepNOS service? Can I run this setup on my laptop?<br>
<br>
3. @Phil: I remember you mentioning that you had a branch where you had <br>
developed a dapper-like request tracing infrastructure? Could you kindly<br>
point me to this?<br>
<br>
Regards,<br>
-- <br>
Srinivasan Ramesh<br>
_______________________________________________<br>
mochi-devel mailing list<br>
mochi-devel@lists.mcs.anl.gov<br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel">https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel</a><br>
<a href="https://www.mcs.anl.gov/research/projects/mochi">https://www.mcs.anl.gov/research/projects/mochi</a><br>
<br>
<br>
_______________________________________________<br>
mochi-devel mailing list<br>
mochi-devel@lists.mcs.anl.gov<br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel">https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel</a><br>
<a href="https://www.mcs.anl.gov/research/projects/mochi">https://www.mcs.anl.gov/research/projects/mochi</a><br>
</div>
</span></font></div>
</body>
</html>