<div dir="ltr"><div>Hmm, I got it working with the following settings:<br><span style="font-family:monospace,monospace"><br>ENABLE_PYTHON=1<br>PYTHON_INSTALL=/home/tim/anaconda</span><br><br></div><div>I rebuilt everything from scratch with the rebuild-all.sh script, then was able to get it to report the correct version:<br></div><div><span style="font-family:monospace,monospace"><br>$ PYTHONHOME=$HOME/anaconda TURBINE_LOG=0 swift-t py-version.swift <br><br>2.7.9 |Anaconda 2.2.0 (64-bit)| (default, Mar 9 2015, 16:20:48) <br>[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]</span><br><br></div>py-version.swift is just:<span style="font-family:monospace,monospace"><br><br>import io;<br>import python;<br><br>printf(python("import sys\nsys.version"));</span><br><br><div><br><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 21 April 2015 at 11:23, Matthew Turk <span dir="ltr"><<a href="mailto:matthewturk@gmail.com" target="_blank">matthewturk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
I set up Swift/T for the first time the other day and had similar<br>
problems getting it to use my custom-built Python. I was able to get<br>
exm-settings.sh to be recognized, but then during execution it failed.<br>
I ended up having to set both PYTHONHOME *and* LD_PRELOAD, where<br>
LD_PRELOAD had to have both the MPI .so file and the <a href="http://libpython2.7.so" target="_blank">libpython2.7.so</a><br>
included. After that it worked just fine.<br>
<br>
-Matt<br>
<div class="HOEnZb"><div class="h5"><br>
On Tue, Apr 21, 2015 at 11:19 AM, Ozik, Jonathan <<a href="mailto:jozik@anl.gov">jozik@anl.gov</a>> wrote:<br>
> Tim,<br>
><br>
> I’ve tried to set the PYTHONHOME variable a few times to see if I can force<br>
> the right python version to be found but I have yet to be successful.<br>
><br>
> Jonathan<br>
><br>
> On Apr 21, 2015, at 9:54 AM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com">tim.g.armstrong@gmail.com</a>><br>
> wrote:<br>
><br>
> It looks like the environment variable PYTHONHOME=/path/to/anaconda needs to<br>
> be set for it to look in the right places for the packages. I'm looking to<br>
> see if we can automatically set this up so that it does the right thing in<br>
> future.<br>
><br>
> - Tim<br>
><br>
> On 20 April 2015 at 18:24, Ozik, Jonathan <<a href="mailto:jozik@anl.gov">jozik@anl.gov</a>> wrote:<br>
>><br>
>> Tim,<br>
>><br>
>> Just for kicks I tried to create a standalone conda environment for Python<br>
>> 2.7 and used that as my PYTHON_INSTALL location but I still am seeing<br>
>> incorrect sys.version, platform.python_version(), etc. Not sure what’s going<br>
>> on, but like I said earlier, it doesn’t seem to affect the running of the<br>
>> code.<br>
>><br>
>> Jonathan<br>
>><br>
>> On Apr 20, 2015, at 4:56 PM, Jonathan Ozik <<a href="mailto:jozik@anl.gov">jozik@anl.gov</a>> wrote:<br>
>><br>
>> Tim,<br>
>><br>
>> I had:<br>
>> PYTHON_INSTALL=/Users/jozik/anaconda<br>
>><br>
>> I had to go in and undo some of the attempts I had made to fix the issue<br>
>> (but probably just exacerbated it), and now with everything back to its<br>
>> initial settings I get the intended:<br>
>> Using Python lib directory: /Users/jozik/anaconda/lib<br>
>> Using Python lib name: python2.7<br>
>><br>
>> If I ignore what I’m seeing returned by sys.version, it does look like the<br>
>> correct Python version is being picked up, at least based on the modules<br>
>> that are available. Maybe the sys.version issue is associated with the<br>
>> subtle issue you mentioned?<br>
>><br>
>> In any case, thank you for the help.<br>
>><br>
>> Jonathan<br>
>><br>
>> On Apr 20, 2015, at 9:46 AM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com">tim.g.armstrong@gmail.com</a>><br>
>> wrote:<br>
>><br>
>> What did you have PYTHON_INSTALL set to in exm-settings? It should be<br>
>> printing the directory/name there.<br>
>><br>
>> I was actually able to replicate a similar problem, which may not be what<br>
>> you were seeing. There's a subtle issue with ordering of library<br>
>> dependencies where it was picking up the system Python library rather than<br>
>> the other one in some cases.<br>
>><br>
>> - Tim<br>
>><br>
>> On 19 April 2015 at 23:28, Ozik, Jonathan <<a href="mailto:jozik@anl.gov">jozik@anl.gov</a>> wrote:<br>
>>><br>
>>> I’m not an Autoconf expert, so take this with a grain of salt, but the<br>
>>> two lines in the turbine configure script:<br>
>>> echo "Using Python lib directory: ${USE_PYTHON_LIBDIR}"<br>
>>> echo "Using Python lib name: ${USE_PYTHON_NAME}”<br>
>>><br>
>>> were showing:<br>
>>> Using Python lib directory: .<br>
>>> Using Python lib name: 1<br>
>>><br>
>>> so I looked at how those variables were being derived, and it looks like<br>
>>> there might be some issues there.<br>
>>> Also, exm-settings.sh doesn’t mention the --with-python-lib option<br>
>>> either.<br>
>>><br>
>>> As for the sys.version issue that I’m seeing, it’s really strange,<br>
>>> because it is possible that sys.version is not correct, as odd as that may<br>
>>> sound. I know that sys.executable is not always correct, but I thought I<br>
>>> could trust sys.version. The python that is being used definitely has<br>
>>> modules installed that are not in the version of python that’s being<br>
>>> reported. Odd. But the good news is that I’m now able to move on with what<br>
>>> we’re actually trying to do.<br>
>>><br>
>>> Thanks again for helping me think through this,<br>
>>><br>
>>> Jonathan<br>
>>><br>
>>> On Apr 19, 2015, at 9:21 PM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com">tim.g.armstrong@gmail.com</a>><br>
>>> wrote:<br>
>>><br>
>>> Hmm, so it's probably picking up the system Python by default. This may<br>
>>> be an oversight on our part - I'm creating a bug to look into it. Our build<br>
>>> should be linking it to your other python library but isn't.<br>
>>><br>
>>> As a workaround, you can override this with the DYLD_LIBRARY_PATH<br>
>>> environment variable on Mac OS X.<br>
>>><br>
>>> If you add the directory with your preferred python .dylib file to that<br>
>>> environment variable, it should link against that version of python instead<br>
>>> of the system one.<br>
>>><br>
>>> E.g.<br>
>>><br>
>>> export DYLD_LIBRARY_PATH=/path/to/anaconda/lib<br>
>>><br>
>>> I *think* this should give you a way to work around it in the meantime.<br>
>>><br>
>>> - Tim<br>
>>><br>
>>> On 19 April 2015 at 20:19, Ozik, Jonathan <<a href="mailto:jozik@anl.gov">jozik@anl.gov</a>> wrote:<br>
>>>><br>
>>>> Thanks Tim. Here’s what I get on OS X:<br>
>>>><br>
>>>> XioMBP:swift_simphony_example_DEAP jozik$ otool -L<br>
>>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib<br>
>>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib:<br>
>>>> /Users/jozik/work/swift_t/install/turbine/lib/libtclturbine.dylib<br>
>>>> (compatibility version 0.0.0, current version 0.0.0)<br>
>>>> /Users/jozik/work/swift_t/install/lb/lib/libadlb.dylib (compatibility<br>
>>>> version 0.0.0, current version 0.0.0)<br>
>>>> /Users/jozik/work/swift_t/install/c-utils/lib/libexmcutils.dylib<br>
>>>> (compatibility version 0.0.0, current version 0.0.0)<br>
>>>> /usr/local/opt/tcl-tk/lib/libtcl8.6.dylib (compatibility version 8.6.0,<br>
>>>> current version 8.6.3)<br>
>>>> /usr/local/lib/libmpi.12.dylib (compatibility version 13.0.0, current<br>
>>>> version 13.4.0)<br>
>>>> libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0)<br>
>>>> libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)<br>
>>>> /usr/local/lib/libsz.2.dylib (compatibility version 3.0.0, current<br>
>>>> version 3.0.0)<br>
>>>> /usr/local/lib/libhdf5.8.dylib (compatibility version 9.0.0, current<br>
>>>> version 9.2.0)<br>
>>>> /usr/local/lib/libhdf5_hl.8.dylib (compatibility version 9.0.0, current<br>
>>>> version 9.2.0)<br>
>>>> /usr/local/lib/libpmpi.12.dylib (compatibility version 13.0.0, current<br>
>>>> version 13.4.0)<br>
>>>> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version<br>
>>>> 1213.0.0)<br>
>>>><br>
>>>> /usr/local/Cellar/gcc/4.9.2/lib/gcc/x86_64-apple-darwin14.0.0/4.9.2/libgcc_s.1.dylib<br>
>>>> (compatibility version 1.0.0, current version 1.0.0)<br>
>>>><br>
>>>> Which doesn’t answer our question of which libpython2.7.dylib is being<br>
>>>> linked to… If there’s a build log file that you want to point me to, I can<br>
>>>> do some more digging.<br>
>>>><br>
>>>> Jonathan<br>
>>>><br>
>>>> On Apr 19, 2015, at 7:36 PM, Tim Armstrong <<a href="mailto:tim.g.armstrong@gmail.com">tim.g.armstrong@gmail.com</a>><br>
>>>> wrote:<br>
>>>><br>
>>>> It might be helpful to get some linker information from your<br>
>>>> installation to see which Python version it's linking against.<br>
>>>><br>
>>>> In linux:<br>
>>>> ldd /path/to/turbine/lib/libtclturbine.so<br>
>>>><br>
>>>> In OS X I think it's something like:<br>
>>>> otool -L /path/to/turbine/lib/libtclturbine.dylib<br>
>>>><br>
>>>> - Tim<br>
>>>><br>
>>>> On 19 April 2015 at 15:55, Ozik, Jonathan <<a href="mailto:jozik@anl.gov">jozik@anl.gov</a>> wrote:<br>
>>>>><br>
>>>>> An update. I looked to see whether this was a PATH issue, and saw that<br>
>>>>> there was a difference between how my bash and zsh shells populate the PATH<br>
>>>>> variable. I have made them compatible now, but am still seeing the same<br>
>>>>> issue with the wrong python being picked up.<br>
>>>>><br>
>>>>> Jonathan<br>
>>>>><br>
>>>>> > On Apr 18, 2015, at 11:21 PM, Ozik, Jonathan <<a href="mailto:jozik@anl.gov">jozik@anl.gov</a>> wrote:<br>
>>>>> ><br>
>>>>> > Hi all,<br>
>>>>> ><br>
>>>>> > I’ve run up against an issue where Swift/T’s python and<br>
>>>>> > python_persist are using the wrong Python version. I’ve checked and rebuilt<br>
>>>>> > and it looks like the Python install location specified in exm-settings.sh<br>
>>>>> > is being used in the build log but when I run a simple test:<br>
>>>>> > string ss = python_persist("import sys\nsys.version”);<br>
>>>>> > I’m getting the Python installation at /usr/bin/python rather than<br>
>>>>> > the Python version installed as part of the Anaconda distribution (which is<br>
>>>>> > what the exm-settings.sh is configured to use).<br>
>>>>> > Any advice on how I might better diagnose this would be appreciated.<br>
>>>>> > I’m on Mac OS X (10.10.3).<br>
>>>>> ><br>
>>>>> > Jonathan<br>
>>>>> ><br>
>>>>> > _______________________________________________<br>
>>>>> > ExM-user mailing list<br>
>>>>> > <a href="mailto:ExM-user@lists.mcs.anl.gov">ExM-user@lists.mcs.anl.gov</a><br>
>>>>> > <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> ExM-user mailing list<br>
>>>>> <a href="mailto:ExM-user@lists.mcs.anl.gov">ExM-user@lists.mcs.anl.gov</a><br>
>>>>> <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
>>>><br>
>>>><br>
>>>><br>
>>><br>
>>><br>
>><br>
>><br>
>><br>
><br>
> _______________________________________________<br>
> ExM-user mailing list<br>
> <a href="mailto:ExM-user@lists.mcs.anl.gov">ExM-user@lists.mcs.anl.gov</a><br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> ExM-user mailing list<br>
> <a href="mailto:ExM-user@lists.mcs.anl.gov">ExM-user@lists.mcs.anl.gov</a><br>
> <a href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
><br>
</div></div></blockquote></div><br></div>