<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Jeff,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Disclaimer: I'm not a heavy Python developer and have not experienced this particular issue yet...</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
It does look like there's just an incompatibility with the numpy from your conda install with the one shipped in the PyDarshan wheel (i.e., via pip install), which doesn't seem unreasonable, I guess.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I did find this quickly googling around about conda and wheels: <a href="https://docs.conda.io/projects/conda-build/en/latest/user-guide/wheel-files.html" id="LPlnk">
https://docs.conda.io/projects/conda-build/en/latest/user-guide/wheel-files.html</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1">
<div id="LPBorder_GTaHR0cHM6Ly9kb2NzLmNvbmRhLmlvL3Byb2plY3RzL2NvbmRhLWJ1aWxkL2VuL2xhdGVzdC91c2VyLWd1aWRlL3doZWVsLWZpbGVzLmh0bWw." class="LPBorder627480" style="width: 100%; margin-top: 16px; margin-bottom: 16px; position: relative; max-width: 800px; min-width: 424px;">
<table id="LPContainer627480" role="presentation" style="padding: 12px 36px 12px 12px; width: 100%; border-width: 1px; border-style: solid; border-color: rgb(200, 200, 200); border-radius: 2px;">
<tbody>
<tr style="border-spacing: 0px;" valign="top">
<td style="width: 100%;">
<div id="LPTitle627480" style="font-size: 21px; font-weight: 300; margin-right: 8px; font-family: "wf_segoe-ui_light", "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px;">
<a target="_blank" id="LPUrlAnchor627480" href="https://docs.conda.io/projects/conda-build/en/latest/user-guide/wheel-files.html" style="text-decoration: none; color:var(--themePrimary);">Using wheel files with conda — conda-build 3.21.4+11.gb01fc2b2.dirty
documentation</a></div>
<div id="LPDescription627480" style="font-size: 14px; max-height: 100px; color: rgb(102, 102, 102); font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px; margin-right: 8px; overflow: hidden;">
Using wheel files with conda¶. If you have software in a Python wheel file and want to use it with conda or install it in a conda environment, there are 3 ways.. The best way is to obtain the source code for the software and build a conda package from the source
and not from a wheel.</div>
<div id="LPMetadata627480" style="font-size: 14px; font-weight: 400; color: rgb(166, 166, 166); font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif;">
docs.conda.io</div>
</td>
</tr>
</tbody>
</table>
<div id="LPCloseButtonContainer627480" class="_2G1t7TsffMpTdsxcutbAaz" tabindex="0" title="Remove link preview" role="button">
<i data-icon-name="Cancel" aria-hidden="true" id="LPCloseButton627480" class="_17cZVF34hdKub7ce2cOnAW root-87"></i></div>
</div>
</div>
<br>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
It sounds like there may be a few options to try to ensure compatibility, including building PyDarshan from source in your conda environment or building a conda package based on a Darshan wheel. Any chance you'd be willing to try those and report back on any
issues or successes? Probably good for us to better understand interactions with conda as I know it's popular among the Python community, but not something that's been really tested yet.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
--Shane<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Jeffrey Layton <laytonjb@gmail.com><br>
<b>Sent:</b> Wednesday, July 14, 2021 9:12 AM<br>
<b>To:</b> Snyder, Shane <ssnyder@mcs.anl.gov><br>
<b>Cc:</b> darshan-users@lists.mcs.anl.gov <darshan-users@lists.mcs.anl.gov><br>
<b>Subject:</b> Re: [Darshan-users] PyDarshan issues</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Great! Thanks!</div>
<div><br>
</div>
<div>I thought I would work through the notebooks for PyDarshan. I use Anaconda for my Python which is the source of the issue I think.</div>
<div><br>
</div>
<div>I used pip to install PyDarshan on my system (most of the time I can get pip installed Python modules to work with conda). I'm trying the 01_darsha_introduction notebook. I tried running the first cell and got the following error:</div>
<div><br>
</div>
<div><br>
</div>
<div>
<pre><span class="x_gmail-ansi-red-fg">---------------------------------------------------------------------------</span>
<span class="x_gmail-ansi-red-fg">ValueError</span> Traceback (most recent call last)
<span class="x_gmail-ansi-green-fg"><ipython-input-1-923d294d02c5></span> in <span class="x_gmail-ansi-cyan-fg"><module></span>
<span class="x_gmail-ansi-green-fg">----> 1</span><span class="x_gmail-ansi-red-fg"> </span><span class="x_gmail-ansi-green-fg">import</span> darshan
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 2</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 3</span> report <span class="x_gmail-ansi-blue-fg">=</span> darshan<span class="x_gmail-ansi-blue-fg">.</span>DarshanReport<span class="x_gmail-ansi-blue-fg">(</span><span class="x_gmail-ansi-blue-fg">"example-logs/example.darshan"</span><span class="x_gmail-ansi-blue-fg">,</span> read_all<span class="x_gmail-ansi-blue-fg">=</span><span class="x_gmail-ansi-green-fg">True</span><span class="x_gmail-ansi-blue-fg">)</span> <span class="x_gmail-ansi-red-fg"># Default behavior</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 4</span> report<span class="x_gmail-ansi-blue-fg">.</span>info<span class="x_gmail-ansi-blue-fg">(</span><span class="x_gmail-ansi-blue-fg">)</span>
<span class="x_gmail-ansi-green-fg">~/.local/lib/python3.8/site-packages/darshan/__init__.py</span> in <span class="x_gmail-ansi-cyan-fg"><module></span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 17</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 18</span> <span class="x_gmail-ansi-red-fg">#from darshan.backend.cffi_backend import *</span>
<span class="x_gmail-ansi-green-fg">---> 19</span><span class="x_gmail-ansi-red-fg"> </span><span class="x_gmail-ansi-green-fg">from</span> darshan<span class="x_gmail-ansi-blue-fg">.</span>report <span class="x_gmail-ansi-green-fg">import</span> DarshanReport
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 20</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 21</span>
<span class="x_gmail-ansi-green-fg">~/.local/lib/python3.8/site-packages/darshan/report.py</span> in <span class="x_gmail-ansi-cyan-fg"><module></span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 8</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 9</span>
<span class="x_gmail-ansi-green-fg">---> 10</span><span class="x_gmail-ansi-red-fg"> </span><span class="x_gmail-ansi-green-fg">import</span> darshan<span class="x_gmail-ansi-blue-fg">.</span>backend<span class="x_gmail-ansi-blue-fg">.</span>cffi_backend <span class="x_gmail-ansi-green-fg">as</span> backend
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 11</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 12</span> <span class="x_gmail-ansi-green-fg">import</span> json
<span class="x_gmail-ansi-green-fg">~/.local/lib/python3.8/site-packages/darshan/backend/cffi_backend.py</span> in <span class="x_gmail-ansi-cyan-fg"><module></span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 9</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 10</span> <span class="x_gmail-ansi-green-fg">import</span> numpy <span class="x_gmail-ansi-green-fg">as</span> np
<span class="x_gmail-ansi-green-fg">---> 11</span><span class="x_gmail-ansi-red-fg"> </span><span class="x_gmail-ansi-green-fg">import</span> pandas <span class="x_gmail-ansi-green-fg">as</span> pd
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 12</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 13</span> <span class="x_gmail-ansi-green-fg">import</span> logging
<span class="x_gmail-ansi-green-fg">~/anaconda3/lib/python3.8/site-packages/pandas/__init__.py</span> in <span class="x_gmail-ansi-cyan-fg"><module></span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 27</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 28</span> <span class="x_gmail-ansi-green-fg">try</span><span class="x_gmail-ansi-blue-fg">:</span>
<span class="x_gmail-ansi-green-fg">---> 29</span><span class="x_gmail-ansi-red-fg"> </span><span class="x_gmail-ansi-green-fg">from</span> pandas<span class="x_gmail-ansi-blue-fg">.</span>_libs <span class="x_gmail-ansi-green-fg">import</span> hashtable <span class="x_gmail-ansi-green-fg">as</span> _hashtable<span class="x_gmail-ansi-blue-fg">,</span> lib <span class="x_gmail-ansi-green-fg">as</span> _lib<span class="x_gmail-ansi-blue-fg">,</span> tslib <span class="x_gmail-ansi-green-fg">as</span> _tslib
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 30</span> <span class="x_gmail-ansi-green-fg">except</span> ImportError <span class="x_gmail-ansi-green-fg">as</span> e<span class="x_gmail-ansi-blue-fg">:</span> <span class="x_gmail-ansi-red-fg"># pragma: no cover</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 31</span> <span class="x_gmail-ansi-red-fg"># hack but overkill to use re</span>
<span class="x_gmail-ansi-green-fg">~/anaconda3/lib/python3.8/site-packages/pandas/_libs/__init__.py</span> in <span class="x_gmail-ansi-cyan-fg"><module></span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 11</span>
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 12</span>
<span class="x_gmail-ansi-green-fg">---> 13</span><span class="x_gmail-ansi-red-fg"> </span><span class="x_gmail-ansi-green-fg">from</span> pandas<span class="x_gmail-ansi-blue-fg">.</span>_libs<span class="x_gmail-ansi-blue-fg">.</span>interval <span class="x_gmail-ansi-green-fg">import</span> Interval
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 14</span> from pandas._libs.tslibs import (
<span class="x_gmail-ansi-green-intense-fg x_gmail-ansi-bold"> 15</span> NaT<span class="x_gmail-ansi-blue-fg">,</span>
<span class="x_gmail-ansi-green-fg">pandas/_libs/interval.pyx</span> in <span class="x_gmail-ansi-cyan-fg">init pandas._libs.interval</span><span class="x_gmail-ansi-blue-fg">()</span>
<span class="x_gmail-ansi-red-fg">ValueError</span>: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
</pre>
</div>
<div><br>
</div>
<div>I'm guessing this is an interaction issue between conda and pip due to the last line. Any thoughts on this? I could ask for a PyDarshan package for conda and I wish I could help but I would be less than worthless in that regard.</div>
<div><br>
</div>
<div>Thanks for any advice!</div>
<div><br>
</div>
<div>Jeff</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Wed, Jul 14, 2021 at 10:02 AM Snyder, Shane <<a href="mailto:ssnyder@mcs.anl.gov">ssnyder@mcs.anl.gov</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Jeff,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Fire away! This mailing list and issues on our GitHub page (<a href="https://github.com/darshan-hpc/darshan" id="x_gmail-m_-8781932659329392958LPlnk" target="_blank">https://github.com/darshan-hpc/darshan</a>) are the best place for both Darshan and PyDarshan
issues.<br>
</div>
<div>
<div id="x_gmail-m_-8781932659329392958LPBorder_GTaHR0cHM6Ly9naXRodWIuY29tL2RhcnNoYW4taHBjL2RhcnNoYW4." style="width:100%; margin-top:16px; margin-bottom:16px; max-width:800px; min-width:424px">
<table id="x_gmail-m_-8781932659329392958LPContainer597864" role="presentation" style="padding:12px 36px 12px 12px; width:100%; border-width:1px; border-style:solid; border-color:rgb(200,200,200); border-radius:2px">
<tbody>
<tr valign="top" style="border-spacing:0px">
<td>
<div id="x_gmail-m_-8781932659329392958LPImageContainer597864" style="margin-right:12px; height:120px; overflow:hidden; width:240px">
<a id="x_gmail-m_-8781932659329392958LPImageAnchor597864" href="https://github.com/darshan-hpc/darshan" target="_blank"><img id="x_gmail-m_-8781932659329392958LPThumbnailImageId597864" alt="" width="240" height="120" style="display:block" src="https://opengraph.githubassets.com/9ddeb56c895b7ef4b0be59ea140c1eaeeda7a8341b43cf692edc557e7e6367a4/darshan-hpc/darshan"></a></div>
</td>
<td style="width:100%">
<div id="x_gmail-m_-8781932659329392958LPTitle597864" style="font-size:21px; font-weight:300; margin-right:8px; font-family:"wf_segoe-ui_light","Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; margin-bottom:12px">
<a id="x_gmail-m_-8781932659329392958LPUrlAnchor597864" href="https://github.com/darshan-hpc/darshan" target="_blank" style="text-decoration:none">darshan-hpc/darshan</a></div>
<div id="x_gmail-m_-8781932659329392958LPDescription597864" style="font-size:14px; max-height:100px; color:rgb(102,102,102); font-family:"wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif; margin-bottom:12px; margin-right:8px; overflow:hidden">
Darshan I/O characterization tool. Contribute to darshan-hpc/darshan development by creating an account on GitHub.</div>
<div id="x_gmail-m_-8781932659329392958LPMetadata597864" style="font-size:14px; font-weight:400; color:rgb(166,166,166); font-family:"wf_segoe-ui_normal","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif">
<a href="http://github.com" target="_blank">github.com</a></div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thanks,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
--Shane<br>
</div>
<div id="x_gmail-m_-8781932659329392958appendonsend"></div>
<hr style="display:inline-block; width:98%">
<div id="x_gmail-m_-8781932659329392958divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Darshan-users <<a href="mailto:darshan-users-bounces@lists.mcs.anl.gov" target="_blank">darshan-users-bounces@lists.mcs.anl.gov</a>>
on behalf of Jeffrey Layton <<a href="mailto:laytonjb@gmail.com" target="_blank">laytonjb@gmail.com</a>><br>
<b>Sent:</b> Wednesday, July 14, 2021 8:11 AM<br>
<b>To:</b> <a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a> <<a href="mailto:darshan-users@lists.mcs.anl.gov" target="_blank">darshan-users@lists.mcs.anl.gov</a>><br>
<b>Subject:</b> [Darshan-users] PyDarshan issues</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>Good morning,</div>
<div><br>
</div>
<div>Is this is a good place to bring up PyDarshan issues or is there a better one?</div>
<div><br>
</div>
<div>Thanks!</div>
<div><br>
</div>
<div>Jeff</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>