<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
So there’s another python executable in /usr/bin?
<div class="">Do you have access to a machine with Mac OS X? It could be a platform idiosyncrasy.</div>
<div class="">
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Apr 21, 2015, at 4:57 PM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com" class="">tim.g.armstrong@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="">Hmm, I got it working with the following settings:<br class="">
<span style="font-family:monospace,monospace" class=""><br class="">
ENABLE_PYTHON=1<br class="">
PYTHON_INSTALL=/home/tim/anaconda</span><br class="">
<br class="">
</div>
<div class="">I rebuilt everything from scratch with the rebuild-all.sh script, then was able to get it to report the correct version:<br class="">
</div>
<div class=""><span style="font-family:monospace,monospace" class=""><br class="">
$ PYTHONHOME=$HOME/anaconda TURBINE_LOG=0 swift-t py-version.swift <br class="">
<br class="">
2.7.9 |Anaconda 2.2.0 (64-bit)| (default, Mar 9 2015, 16:20:48) <br class="">
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]</span><br class="">
<br class="">
</div>
py-version.swift is just:<span style="font-family:monospace,monospace" class=""><br class="">
<br class="">
import io;<br class="">
import python;<br class="">
<br class="">
printf(python("import sys\nsys.version"));</span><br class="">
<br class="">
<div class=""><br class="">
<br class="">
</div>
</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">On 21 April 2015 at 11:23, Matthew Turk <span dir="ltr" class="">
<<a href="mailto:matthewturk@gmail.com" target="_blank" class="">matthewturk@gmail.com</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi all,<br class="">
<br class="">
I set up Swift/T for the first time the other day and had similar<br class="">
problems getting it to use my custom-built Python. I was able to get<br class="">
exm-settings.sh to be recognized, but then during execution it failed.<br class="">
I ended up having to set both PYTHONHOME *and* LD_PRELOAD, where<br class="">
LD_PRELOAD had to have both the MPI .so file and the <a href="http://libpython2.7.so/" target="_blank" class="">
libpython2.7.so</a><br class="">
included. After that it worked just fine.<br class="">
<br class="">
-Matt<br class="">
<div class="HOEnZb">
<div class="h5"><br class="">
On Tue, Apr 21, 2015 at 11:19 AM, Ozik, Jonathan <<a href="mailto:jozik@anl.gov" class="">jozik@anl.gov</a>> wrote:<br class="">
> Tim,<br class="">
><br class="">
> I’ve tried to set the PYTHONHOME variable a few times to see if I can force<br class="">
> the right python version to be found but I have yet to be successful.<br class="">
><br class="">
> Jonathan<br class="">
><br class="">
> On Apr 21, 2015, at 9:54 AM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com" class="">tim.g.armstrong@gmail.com</a>><br class="">
> wrote:<br class="">
><br class="">
> It looks like the environment variable PYTHONHOME=/path/to/anaconda needs to<br class="">
> be set for it to look in the right places for the packages. I'm looking to<br class="">
> see if we can automatically set this up so that it does the right thing in<br class="">
> future.<br class="">
><br class="">
> - Tim<br class="">
><br class="">
> On 20 April 2015 at 18:24, Ozik, Jonathan <<a href="mailto:jozik@anl.gov" class="">jozik@anl.gov</a>> wrote:<br class="">
>><br class="">
>> Tim,<br class="">
>><br class="">
>> Just for kicks I tried to create a standalone conda environment for Python<br class="">
>> 2.7 and used that as my PYTHON_INSTALL location but I still am seeing<br class="">
>> incorrect sys.version, platform.python_version(), etc. Not sure what’s going<br class="">
>> on, but like I said earlier, it doesn’t seem to affect the running of the<br class="">
>> code.<br class="">
>><br class="">
>> Jonathan<br class="">
>><br class="">
>> On Apr 20, 2015, at 4:56 PM, Jonathan Ozik <<a href="mailto:jozik@anl.gov" class="">jozik@anl.gov</a>> wrote:<br class="">
>><br class="">
>> Tim,<br class="">
>><br class="">
>> I had:<br class="">
>> PYTHON_INSTALL=/Users/jozik/anaconda<br class="">
>><br class="">
>> I had to go in and undo some of the attempts I had made to fix the issue<br class="">
>> (but probably just exacerbated it), and now with everything back to its<br class="">
>> initial settings I get the intended:<br class="">
>> Using Python lib directory: /Users/jozik/anaconda/lib<br class="">
>> Using Python lib name: python2.7<br class="">
>><br class="">
>> If I ignore what I’m seeing returned by sys.version, it does look like the<br class="">
>> correct Python version is being picked up, at least based on the modules<br class="">
>> that are available. Maybe the sys.version issue is associated with the<br class="">
>> subtle issue you mentioned?<br class="">
>><br class="">
>> In any case, thank you for the help.<br class="">
>><br class="">
>> Jonathan<br class="">
>><br class="">
>> On Apr 20, 2015, at 9:46 AM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com" class="">tim.g.armstrong@gmail.com</a>><br class="">
>> wrote:<br class="">
>><br class="">
>> What did you have PYTHON_INSTALL set to in exm-settings? It should be<br class="">
>> printing the directory/name there.<br class="">
>><br class="">
>> I was actually able to replicate a similar problem, which may not be what<br class="">
>> you were seeing. There's a subtle issue with ordering of library<br class="">
>> dependencies where it was picking up the system Python library rather than<br class="">
>> the other one in some cases.<br class="">
>><br class="">
>> - Tim<br class="">
>><br class="">
>> On 19 April 2015 at 23:28, Ozik, Jonathan <<a href="mailto:jozik@anl.gov" class="">jozik@anl.gov</a>> wrote:<br class="">
>>><br class="">
>>> I’m not an Autoconf expert, so take this with a grain of salt, but the<br class="">
>>> two lines in the turbine configure script:<br class="">
>>> echo "Using Python lib directory: ${USE_PYTHON_LIBDIR}"<br class="">
>>> echo "Using Python lib name: ${USE_PYTHON_NAME}”<br class="">
>>><br class="">
>>> were showing:<br class="">
>>> Using Python lib directory: .<br class="">
>>> Using Python lib name: 1<br class="">
>>><br class="">
>>> so I looked at how those variables were being derived, and it looks like<br class="">
>>> there might be some issues there.<br class="">
>>> Also, exm-settings.sh doesn’t mention the --with-python-lib option<br class="">
>>> either.<br class="">
>>><br class="">
>>> As for the sys.version issue that I’m seeing, it’s really strange,<br class="">
>>> because it is possible that sys.version is not correct, as odd as that may<br class="">
>>> sound. I know that sys.executable is not always correct, but I thought I<br class="">
>>> could trust sys.version. The python that is being used definitely has<br class="">
>>> modules installed that are not in the version of python that’s being<br class="">
>>> reported. Odd. But the good news is that I’m now able to move on with what<br class="">
>>> we’re actually trying to do.<br class="">
>>><br class="">
>>> Thanks again for helping me think through this,<br class="">
>>><br class="">
>>> Jonathan<br class="">
>>><br class="">
>>> On Apr 19, 2015, at 9:21 PM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com" class="">tim.g.armstrong@gmail.com</a>><br class="">
>>> wrote:<br class="">
>>><br class="">
>>> Hmm, so it's probably picking up the system Python by default. This may<br class="">
>>> be an oversight on our part - I'm creating a bug to look into it. Our build<br class="">
>>> should be linking it to your other python library but isn't.<br class="">
>>><br class="">
>>> As a workaround, you can override this with the DYLD_LIBRARY_PATH<br class="">
>>> environment variable on Mac OS X.<br class="">
>>><br class="">
>>> If you add the directory with your preferred python .dylib file to that<br class="">
>>> environment variable, it should link against that version of python instead<br class="">
>>> of the system one.<br class="">
>>><br class="">
>>> E.g.<br class="">
>>><br class="">
>>> export DYLD_LIBRARY_PATH=/path/to/anaconda/lib<br class="">
>>><br class="">
>>> I *think* this should give you a way to work around it in the meantime.<br class="">
>>><br class="">
>>> - Tim<br class="">
>>><br class="">
>>> On 19 April 2015 at 20:19, Ozik, Jonathan <<a href="mailto:jozik@anl.gov" class="">jozik@anl.gov</a>> wrote:<br class="">
>>>><br class="">
>>>> Thanks Tim. Here’s what I get on OS X:<br class="">
>>>><br class="">
>>>> XioMBP:swift_simphony_example_DEAP jozik$ otool -L<br class="">
>>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib<br class="">
>>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib:<br class="">
>>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib<br class="">
>>>> (compatibility version 0.0.0, current version 0.0.0)<br class="">
>>>> /Users/jozik/work/swift_t/install/lb/lib/libadlb.dylib (compatibility<br class="">
>>>> version 0.0.0, current version 0.0.0)<br class="">
>>>> /Users/jozik/work/swift_t/install/c-utils/lib/libexmcutils.dylib<br class="">
>>>> (compatibility version 0.0.0, current version 0.0.0)<br class="">
>>>> /usr/local/opt/tcl-tk/lib/libtcl8.6.dylib (compatibility version 8.6.0,<br class="">
>>>> current version 8.6.3)<br class="">
>>>> /usr/local/lib/libmpi.12.dylib (compatibility version 13.0.0, current<br class="">
>>>> version 13.4.0)<br class="">
>>>> libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0)<br class="">
>>>> libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)<br class="">
>>>> /usr/local/lib/libsz.2.dylib (compatibility version 3.0.0, current<br class="">
>>>> version 3.0.0)<br class="">
>>>> /usr/local/lib/libhdf5.8.dylib (compatibility version 9.0.0, current<br class="">
>>>> version 9.2.0)<br class="">
>>>> /usr/local/lib/libhdf5_hl.8.dylib (compatibility version 9.0.0, current<br class="">
>>>> version 9.2.0)<br class="">
>>>> /usr/local/lib/libpmpi.12.dylib (compatibility version 13.0.0, current<br class="">
>>>> version 13.4.0)<br class="">
>>>> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version<br class="">
>>>> 1213.0.0)<br class="">
>>>><br class="">
>>>> /usr/local/Cellar/gcc/4.9.2/lib/gcc/x86_64-apple-darwin14.0.0/4.9.2/libgcc_s.1.dylib<br class="">
>>>> (compatibility version 1.0.0, current version 1.0.0)<br class="">
>>>><br class="">
>>>> Which doesn’t answer our question of which libpython2.7.dylib is being<br class="">
>>>> linked to… If there’s a build log file that you want to point me to, I can<br class="">
>>>> do some more digging.<br class="">
>>>><br class="">
>>>> Jonathan<br class="">
>>>><br class="">
>>>> On Apr 19, 2015, at 7:36 PM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com" class="">tim.g.armstrong@gmail.com</a>><br class="">
>>>> wrote:<br class="">
>>>><br class="">
>>>> It might be helpful to get some linker information from your<br class="">
>>>> installation to see which Python version it's linking against.<br class="">
>>>><br class="">
>>>> In linux:<br class="">
>>>> ldd /path/to/turbine/lib/libtclturbine.so<br class="">
>>>><br class="">
>>>> In OS X I think it's something like:<br class="">
>>>> otool -L /path/to/turbine/lib/libtclturbine.dylib<br class="">
>>>><br class="">
>>>> - Tim<br class="">
>>>><br class="">
>>>> On 19 April 2015 at 15:55, Ozik, Jonathan <<a href="mailto:jozik@anl.gov" class="">jozik@anl.gov</a>> wrote:<br class="">
>>>>><br class="">
>>>>> An update. I looked to see whether this was a PATH issue, and saw that<br class="">
>>>>> there was a difference between how my bash and zsh shells populate the PATH<br class="">
>>>>> variable. I have made them compatible now, but am still seeing the same<br class="">
>>>>> issue with the wrong python being picked up.<br class="">
>>>>><br class="">
>>>>> Jonathan<br class="">
>>>>><br class="">
>>>>> > On Apr 18, 2015, at 11:21 PM, Ozik, Jonathan <<a href="mailto:jozik@anl.gov" class="">jozik@anl.gov</a>> wrote:<br class="">
>>>>> ><br class="">
>>>>> > Hi all,<br class="">
>>>>> ><br class="">
>>>>> > I’ve run up against an issue where Swift/T’s python and<br class="">
>>>>> > python_persist are using the wrong Python version. I’ve checked and rebuilt<br class="">
>>>>> > and it looks like the Python install location specified in exm-settings.sh<br class="">
>>>>> > is being used in the build log but when I run a simple test:<br class="">
>>>>> > string ss = python_persist("import sys\nsys.version”);<br class="">
>>>>> > I’m getting the Python installation at /usr/bin/python rather than<br class="">
>>>>> > the Python version installed as part of the Anaconda distribution (which is<br class="">
>>>>> > what the exm-settings.sh is configured to use).<br class="">
>>>>> > Any advice on how I might better diagnose this would be appreciated.<br class="">
>>>>> > I’m on Mac OS X (10.10.3).<br class="">
>>>>> ><br class="">
>>>>> > Jonathan<br class="">
>>>>> ><br class="">
>>>>> > _______________________________________________<br class="">
>>>>> > ExM-user mailing list<br class="">
>>>>> > <a href="mailto:ExM-user@lists.mcs.anl.gov" class="">ExM-user@lists.mcs.anl.gov</a><br class="">
>>>>> > <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank" class="">
https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br class="">
>>>>><br class="">
>>>>> _______________________________________________<br class="">
>>>>> ExM-user mailing list<br class="">
>>>>> <a href="mailto:ExM-user@lists.mcs.anl.gov" class="">ExM-user@lists.mcs.anl.gov</a><br class="">
>>>>> <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank" class="">
https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br class="">
>>>><br class="">
>>>><br class="">
>>>><br class="">
>>><br class="">
>>><br class="">
>><br class="">
>><br class="">
>><br class="">
><br class="">
> _______________________________________________<br class="">
> ExM-user mailing list<br class="">
> <a href="mailto:ExM-user@lists.mcs.anl.gov" class="">ExM-user@lists.mcs.anl.gov</a><br class="">
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank" class="">
https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br class="">
><br class="">
><br class="">
><br class="">
> _______________________________________________<br class="">
> ExM-user mailing list<br class="">
> <a href="mailto:ExM-user@lists.mcs.anl.gov" class="">ExM-user@lists.mcs.anl.gov</a><br class="">
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank" class="">
https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br class="">
><br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</body>
</html>