[Darshan-users] PyDarshan issues

[Snyder, Shane]

Hi Jeff,

Disclaimer: I'm not a heavy Python developer and have not experienced this particular issue yet...

It does look like there's just an incompatibility with the numpy from your conda install with the one shipped in the PyDarshan wheel (i.e., via pip install), which doesn't seem unreasonable, I guess.

I did find this quickly googling around about conda and wheels: https://docs.conda.io/projects/conda-build/en/latest/user-guide/wheel-files.html
Using wheel files with conda — conda-build 3.21.4+11.gb01fc2b2.dirty documentation<https://docs.conda.io/projects/conda-build/en/latest/user-guide/wheel-files.html>
Using wheel files with conda¶. If you have software in a Python wheel file and want to use it with conda or install it in a conda environment, there are 3 ways.. The best way is to obtain the source code for the software and build a conda package from the source and not from a wheel.
docs.conda.io


It sounds like there may be a few options to try to ensure compatibility, including building PyDarshan from source in your conda environment or building a conda package based on a Darshan wheel. Any chance you'd be willing to try those and report back on any issues or successes? Probably good for us to better understand interactions with conda as I know it's popular among the Python community, but not something that's been really tested yet.

Thanks,
--Shane
________________________________
From: Jeffrey Layton <laytonjb at gmail.com>
Sent: Wednesday, July 14, 2021 9:12 AM
To: Snyder, Shane <ssnyder at mcs.anl.gov>
Cc: darshan-users at lists.mcs.anl.gov <darshan-users at lists.mcs.anl.gov>
Subject: Re: [Darshan-users] PyDarshan issues

Great! Thanks!

I thought I would work through the notebooks for PyDarshan. I use Anaconda for my Python which is the source of the issue I think.

I used pip to install PyDarshan on my system (most of the time I can get pip installed Python modules to work with conda). I'm trying the 01_darsha_introduction notebook. I tried running the first cell and got the following error:



---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-923d294d02c5> in <module>
----> 1 import darshan
      2
      3 report = darshan.DarshanReport("example-logs/example.darshan", read_all=True)  # Default behavior
      4 report.info()

~/.local/lib/python3.8/site-packages/darshan/__init__.py in <module>
     17
     18 #from darshan.backend.cffi_backend import *
---> 19 from darshan.report import DarshanReport
     20
     21

~/.local/lib/python3.8/site-packages/darshan/report.py in <module>
      8
      9
---> 10 import darshan.backend.cffi_backend as backend
     11
     12 import json

~/.local/lib/python3.8/site-packages/darshan/backend/cffi_backend.py in <module>
      9
     10 import numpy as np
---> 11 import pandas as pd
     12
     13 import logging

~/anaconda3/lib/python3.8/site-packages/pandas/__init__.py in <module>
     27
     28 try:
---> 29     from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
     30 except ImportError as e:  # pragma: no cover
     31     # hack but overkill to use re

~/anaconda3/lib/python3.8/site-packages/pandas/_libs/__init__.py in <module>
     11
     12
---> 13 from pandas._libs.interval import Interval
     14 from pandas._libs.tslibs import (
     15     NaT,

pandas/_libs/interval.pyx in init pandas._libs.interval()

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject


I'm guessing this is an interaction issue between conda and pip due to the last line. Any thoughts on this? I could ask for a PyDarshan package for conda and I wish I could help but I would be less than worthless in that regard.

Thanks for any advice!

Jeff



On Wed, Jul 14, 2021 at 10:02 AM Snyder, Shane <ssnyder at mcs.anl.gov<mailto:ssnyder at mcs.anl.gov>> wrote:
Hi Jeff,

Fire away! This mailing list and issues on our GitHub page (https://github.com/darshan-hpc/darshan) are the best place for both Darshan and PyDarshan issues.
[https://opengraph.githubassets.com/9ddeb56c895b7ef4b0be59ea140c1eaeeda7a8341b43cf692edc557e7e6367a4/darshan-hpc/darshan]<https://github.com/darshan-hpc/darshan>
darshan-hpc/darshan<https://github.com/darshan-hpc/darshan>
Darshan I/O characterization tool. Contribute to darshan-hpc/darshan development by creating an account on GitHub.
github.com<http://github.com>

Thanks,
--Shane
________________________________
From: Darshan-users <darshan-users-bounces at lists.mcs.anl.gov<mailto:darshan-users-bounces at lists.mcs.anl.gov>> on behalf of Jeffrey Layton <laytonjb at gmail.com<mailto:laytonjb at gmail.com>>
Sent: Wednesday, July 14, 2021 8:11 AM
To: darshan-users at lists.mcs.anl.gov<mailto:darshan-users at lists.mcs.anl.gov> <darshan-users at lists.mcs.anl.gov<mailto:darshan-users at lists.mcs.anl.gov>>
Subject: [Darshan-users] PyDarshan issues

Good morning,

Is this is a good place to bring up PyDarshan issues or is there a better one?

Thanks!

Jeff

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20210714/633287f9/attachment-0001.html>