[ExM Users] Swift/T using wrong Python

Tue Apr 21 09:54:03 CDT 2015

It looks like the environment variable PYTHONHOME=/path/to/anaconda needs
to be set for it to look in the right places for the packages.  I'm looking
to see if we can automatically set this up so that it does the right thing
in future.

- Tim

On 20 April 2015 at 18:24, Ozik, Jonathan <jozik at anl.gov> wrote:

>  Tim,
>
>  Just for kicks I tried to create a standalone conda environment for
> Python 2.7 and used that as my PYTHON_INSTALL location but I still am
> seeing incorrect sys.version, platform.python_version(), etc. Not sure
> what’s going on, but like I said earlier, it doesn’t seem to affect the
> running of the code.
>
>  Jonathan
>
>  On Apr 20, 2015, at 4:56 PM, Jonathan Ozik <jozik at anl.gov> wrote:
>
>  Tim,
>
>  I had:
> PYTHON_INSTALL=/Users/jozik/anaconda
>
>  I had to go in and undo some of the attempts I had made to fix the issue
> (but probably just exacerbated it), and now with everything back to its
> initial settings I get the intended:
> Using Python lib directory: /Users/jozik/anaconda/lib
> Using Python lib name:      python2.7
>
>  If I ignore what I’m seeing returned by sys.version, it does look like
> the correct Python version is being picked up, at least based on the
> modules that are available. Maybe the sys.version issue is associated with
> the subtle issue you mentioned?
>
>  In any case, thank you for the help.
>
>  Jonathan
>
>   On Apr 20, 2015, at 9:46 AM, Tim Armstrong <tim.g.armstrong at gmail.com>
> wrote:
>
>   What did you have PYTHON_INSTALL set to in exm-settings?  It should be
> printing the directory/name there.
>
>  I was actually able to replicate a similar problem, which may not be what
> you were seeing.  There's a subtle issue with ordering of library
> dependencies where it was picking up the system Python library rather than
> the other one in some cases.
>
>  - Tim
>
> On 19 April 2015 at 23:28, Ozik, Jonathan <jozik at anl.gov> wrote:
>
>> I’m not an Autoconf expert, so take this with a grain of salt, but the
>> two lines in the turbine configure script:
>>     echo "Using Python lib directory: ${USE_PYTHON_LIBDIR}"
>>     echo "Using Python lib name:      ${USE_PYTHON_NAME}”
>>
>>  were showing:
>> Using Python lib directory: .
>> Using Python lib name:      1
>>
>>  so I looked at how those variables were being derived, and it looks
>> like there might be some issues there.
>> Also, exm-settings.sh doesn’t mention the --with-python-lib option either.
>>
>>  As for the sys.version issue that I’m seeing, it’s really strange,
>> because it is possible that sys.version is not correct, as odd as that may
>> sound. I know that sys.executable is not always correct, but I thought I
>> could trust sys.version. The python that is being used definitely has
>> modules installed that are not in the version of python that’s being
>> reported. Odd. But the good news is that I’m now able to move on with what
>> we’re actually trying to do.
>>
>>  Thanks again for helping me think through this,
>>
>>  Jonathan
>>
>>  On Apr 19, 2015, at 9:21 PM, Tim Armstrong <tim.g.armstrong at gmail.com>
>> wrote:
>>
>>  Hmm, so it's probably picking up the system Python by default.   This
>> may be an oversight on our part - I'm creating a bug to look into it.  Our
>> build should be linking it to your other python library but isn't.
>>
>>  As a workaround, you can override this with the DYLD_LIBRARY_PATH
>> environment variable on Mac OS X.
>>
>> If you add the directory with your preferred python .dylib file to that
>> environment variable, it should link against that version of python instead
>> of the system one.
>>
>>  E.g.
>>
>> export DYLD_LIBRARY_PATH=/path/to/anaconda/lib
>>
>>  I *think* this should give you a way to work around it in the meantime.
>>
>>  - Tim
>>
>> On 19 April 2015 at 20:19, Ozik, Jonathan <jozik at anl.gov> wrote:
>>
>>> Thanks Tim. Here’s what I get on OS X:
>>>
>>>  XioMBP:swift_simphony_example_DEAP jozik$ otool
>>> -L /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib
>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib:
>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib
>>> (compatibility version 0.0.0, current version 0.0.0)
>>> /Users/jozik/work/swift_t/install/lb/lib/libadlb.dylib (compatibility
>>> version 0.0.0, current version 0.0.0)
>>> /Users/jozik/work/swift_t/install/c-utils/lib/libexmcutils.dylib
>>> (compatibility version 0.0.0, current version 0.0.0)
>>> /usr/local/opt/tcl-tk/lib/libtcl8.6.dylib (compatibility version 8.6.0,
>>> current version 8.6.3)
>>> /usr/local/lib/libmpi.12.dylib (compatibility version 13.0.0, current
>>> version 13.4.0)
>>> libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0)
>>> libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
>>> /usr/local/lib/libsz.2.dylib (compatibility version 3.0.0, current
>>> version 3.0.0)
>>> /usr/local/lib/libhdf5.8.dylib (compatibility version 9.0.0, current
>>> version 9.2.0)
>>> /usr/local/lib/libhdf5_hl.8.dylib (compatibility version 9.0.0, current
>>> version 9.2.0)
>>> /usr/local/lib/libpmpi.12.dylib (compatibility version 13.0.0, current
>>> version 13.4.0)
>>> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
>>> 1213.0.0)
>>> /usr/local/Cellar/gcc/4.9.2/lib/gcc/x86_64-apple-darwin14.0.0/4.9.2/libgcc_s.1.dylib (compatibility
>>> version 1.0.0, current version 1.0.0)
>>>
>>>  Which doesn’t answer our question of which libpython2.7.dylib is being
>>> linked to… If there’s a build log file that you want to point me to, I can
>>> do some more digging.
>>>
>>>  Jonathan
>>>
>>>  On Apr 19, 2015, at 7:36 PM, Tim Armstrong <tim.g.armstrong at gmail.com>
>>> wrote:
>>>
>>>  It might be helpful to get some linker information from your
>>> installation to see which Python version it's linking against.
>>>
>>> In linux:
>>> ldd  /path/to/turbine/lib/libtclturbine.so
>>>
>>>  In OS X I think it's something like:
>>> otool -L /path/to/turbine/lib/libtclturbine.dylib
>>>
>>>  - Tim
>>>
>>> On 19 April 2015 at 15:55, Ozik, Jonathan <jozik at anl.gov> wrote:
>>>
>>>> An update. I looked to see whether this was a PATH issue, and saw that
>>>> there was a difference between how my bash and zsh shells populate the PATH
>>>> variable. I have made them compatible now, but am still seeing the same
>>>> issue with the wrong python being picked up.
>>>>
>>>> Jonathan
>>>>
>>>> > On Apr 18, 2015, at 11:21 PM, Ozik, Jonathan <jozik at anl.gov> wrote:
>>>> >
>>>> > Hi all,
>>>> >
>>>> > I’ve run up against an issue where Swift/T’s python and
>>>> python_persist are using the wrong Python version. I’ve checked and rebuilt
>>>> and it looks like the Python install location specified in exm-settings.sh
>>>> is being used in the build log but when I run a simple test:
>>>> > string ss = python_persist("import sys\nsys.version”);
>>>> > I’m getting the Python installation at /usr/bin/python rather than
>>>> the Python version installed as part of the Anaconda distribution (which is
>>>> what the exm-settings.sh is configured to use).
>>>> > Any advice on how I might better diagnose this would be appreciated.
>>>> I’m on Mac OS X (10.10.3).
>>>> >
>>>> > Jonathan
>>>> >
>>>> > _______________________________________________
>>>> > ExM-user mailing list
>>>> > ExM-user at lists.mcs.anl.gov
>>>> > https://lists.mcs.anl.gov/mailman/listinfo/exm-user
>>>>
>>>> _______________________________________________
>>>> ExM-user mailing list
>>>> ExM-user at lists.mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/exm-user
>>>>
>>>
>>>
>>>
>>
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/exm-user/attachments/20150421/530ebf8a/attachment.html>