<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We do have a work-in-progress PR on our repo that is supposed to help workaround this limit:
<a href="https://github.com/darshan-hpc/darshan/pull/405" id="LPNoLPOWALinkPreview">
https://github.com/darshan-hpc/darshan/pull/405</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1"></div>
<br>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
That PR is generally about modifying Darshan to accept different configuration options from users at runtime (currently only using a config file, but ultimately there will be corresponding env variables to control things), including an ability to control how
many records are pre-allocated for each module (via MAX_RECORDS configuration setting). I think things work well enough on that branch that you could try requesting more records for the POSIX module, or whatever modules you're interested in, and see how things
work. The PR has some details on how to do this.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
As Phil mentions, we hope to have this functionality cleaned up for our next release. Hopefully that's something we can have ready by the end of the year or soon thereafter. In the meantime, please let us know if you have any questions about using that branch
or if you have feedback on anything.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
--Shane<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Darshan-users <darshan-users-bounces@lists.mcs.anl.gov> on behalf of Phil Carns <carns@mcs.anl.gov><br>
<b>Sent:</b> Monday, November 1, 2021 10:47 AM<br>
<b>To:</b> darshan-users@lists.mcs.anl.gov <darshan-users@lists.mcs.anl.gov><br>
<b>Subject:</b> Re: [Darshan-users] Recording more than 1024 files</font>
<div> </div>
</div>
<div>
<p>Hi George,</p>
<p><br>
</p>
<p>Unfortunately it is hardcoded for the moment (see DARSHAN_DEF_MOD_REC_COUNT) regardless of how much total memory in bytes you allow each module to use. We are working to have that tunable in an upcoming release.</p>
<p><br>
</p>
<p>Another option, depending on your use case, may be to configure Darshan to exclude some directories so that it doesn't use up entries on files that you aren't interested in. There is some info about that option in this mailing list post:</p>
<p><br>
</p>
<p><a class="x_moz-txt-link-freetext" href="https://lists.mcs.anl.gov/pipermail/darshan-users/2020-July/000636.html">https://lists.mcs.anl.gov/pipermail/darshan-users/2020-July/000636.html</a></p>
<p><br>
</p>
<p>thanks,</p>
<p>-Phil<br>
</p>
<p><br>
</p>
<div class="x_moz-cite-prefix">On 10/29/21 7:09 PM, George Papadimitriou wrote:<br>
</div>
<blockquote type="cite"><style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3">Dear darshan devs,</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3"><br>
</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3">I ran into a limit when using Darshan 3.3.1, where darshan-runtime records only the first 1024 files.</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3">Is there a way to build Darshan and increase this limit?</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3">I couldn't find something in the documentation. I increased the module memory, but it didn't affect this.</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3"><br>
</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3">Thanks,</font></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; color:rgb(0,0,0)"><font size="3">George</font></div>
<br>
<fieldset class="x_mimeAttachmentHeader"></fieldset>
<pre class="x_moz-quote-pre">_______________________________________________
Darshan-users mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:Darshan-users@lists.mcs.anl.gov">Darshan-users@lists.mcs.anl.gov</a>
<a class="x_moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/darshan-users">https://lists.mcs.anl.gov/mailman/listinfo/darshan-users</a>
</pre>
</blockquote>
</div>
</body>
</html>