<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Wed, Mar 29, 2017 at 5:12 PM, Austin Herrema <span dir="ltr"><<a href="mailto:aherrema@iastate.edu" target="_blank">aherrema@iastate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Quick update on this issue in case it brings any other thoughts/ideas to light. For a very simple, small problem, I am successfully able to use MatSeqAIJSetPreallocation in a fortran-based code compiled for python via f2py. I am still unsure why, in a larger code, this particular function call fails when the code is executed in python (on a setup that runs fine under pure Fortran). Does the error "<span style="font-size:12.8px"> nnz cannot be greater than row length: local row 2 value 1330400321 rowlength 37065" imply that the program thinks I am trying to allocate </span><span style="font-size:12.8px">1330400321 nonzeros in a row of max length </span><span style="font-size:12.8px">37065?</span></div></blockquote><div><br></div><div>Yes.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><span style="font-size:12.8px"> That is obviously not my intent nor what I think I have coded. I am trying to skip preallocation and use merely MatSetUp but, as we would expect, the dynamic allocation is ridiculously slow...</span></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 29, 2017 at 11:27 AM, Austin Herrema <span dir="ltr"><<a href="mailto:aherrema@iastate.edu" target="_blank">aherrema@iastate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Got it--just had to link against other compiled source, as you said. I've attached my makefile for doing everything (including variable definitions, compiling source, and running requisite f2py commands) in case that's helpful for anyone else trying to do something similar. But obviously the meat of it is in what Gaetan provided.<br><br>I am now able to successfully run simple PETSc-based fortran codes in python. For a larger, more complex code, I am getting some PETSc errors when running in python that I don't normally get. In particular, preallocation is failing--the relevant fortran code block and PETSc error is below.<div><br></div><div><br><div> call MatCreate(PETSC_COMM_WORLD, LHS_pc, pc_ier)</div><div> call MatSetSizes(LHS_pc, PETSC_DECIDE, PETSC_DECIDE, NSD*FUN%NNODE, NSD*FUN%NNODE, pc_ier)</div><div> call MatSetFromOptions(LHS_pc, pc_ier)</div><div> call MatSeqAIJSetPreallocation(LHS_<wbr>pc, 500, PETSC_NULL_INTEGER, pc_ier)</div><div><br></div><div><br></div><div><div>[0]PETSC ERROR: --------------------- Error Message ------------------------------<wbr>------------------------------<wbr>--</div><div>[0]PETSC ERROR: Argument out of range</div><div>[0]PETSC ERROR: nnz cannot be greater than row length: local row 2 value 1330400321 rowlength 37065</div><div>[0]PETSC ERROR: See <a href="http://www.mcs.anl.gov/petsc/documentation/faq.html" target="_blank">http://www.mcs.anl.gov/petsc/d<wbr>ocumentation/faq.html</a> for trouble shooting.</div><div>[0]PETSC ERROR: Petsc Release Version 3.7.5, Jan, 01, 2017</div><div>[0]PETSC ERROR: Unknown Name on a real named <a href="http://austin-ethernet.student.iastate.edu" target="_blank">austin-ethernet.student.iastat<wbr>e.edu</a> by Austin Wed Mar 29 10:59:33 2017</div><div>[0]PETSC ERROR: Configure options CC=/usr/local/bin/mpicc CXX=/usr/local/bin/mpicxx F77=/usr/local/bin/mpif77 FC=/usr/local/bin/mpif90 --with-shared-libraries=1 --with-pthread=0 --with-openmp=0 --with-debugging=1 --with-ssl=0 --with-superlu_dist-include=/u<wbr>sr/local/opt/superlu_dist/incl<wbr>ude --with-superlu_dist-lib="-L/us<wbr>r/local/opt/superlu_dist/lib -lsuperlu_dist" --with-fftw-dir=/usr/local/opt<wbr>/fftw --with-netcdf-dir=/usr/local/o<wbr>pt/netcdf --with-suitesparse-dir=/usr/lo<wbr>cal/opt/suite-sparse --with-hdf5-dir=/usr/local/opt<wbr>/hdf5 --with-metis-dir=/usr/local/op<wbr>t/metis --with-parmetis-dir=/usr/local<wbr>/opt/parmetis --with-scalapack-dir=/usr/loca<wbr>l/opt/scalapack --with-mumps-dir=/usr/local/op<wbr>t/mumps/libexec --with-x=0 --prefix=/usr/local/Cellar/pet<wbr>sc/3.7.5/real --with-scalar-type=real --with-hypre-dir=/usr/local/op<wbr>t/hypre --with-sundials-dir=/usr/local<wbr>/opt/sundials --with-hwloc-dir=/usr/local/op<wbr>t/hwloc</div><div>[0]PETSC ERROR: #1 MatSeqAIJSetPreallocation_SeqA<wbr>IJ() line 3598 in /private/tmp/petsc-20170223-50<wbr>8-1xeniyc/petsc-3.7.5/src/mat/<wbr>impls/aij/seq/aij.c</div><div>[0]PETSC ERROR: #2 MatSeqAIJSetPreallocation() line 3570 in /private/tmp/petsc-20170223-50<wbr>8-1xeniyc/petsc-3.7.5/src/mat/<wbr>impls/aij/seq/aij.c</div></div><div><br></div><div><br></div>Is there anything about the MatSeqAIJSetPreallocation function that would make it not work correctly in Python even though everything else seems to work properly? If anyone has thoughts on this that would be great. But, again, I do realize I'm venturing into potentially unsupported territory.<br><br></div></div><div class="m_2533161059319932354HOEnZb"><div class="m_2533161059319932354h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 28, 2017 at 4:53 PM, Gaetan Kenway <span dir="ltr"><<a href="mailto:gaetank@gmail.com" target="_blank">gaetank@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Looks like it isn't finding your source from run_analysis.f90. You still need to compile that yourself and include in the final link. In my example, all the "original" source code was precompiled into a library from a different makefile and then this was run after-the-fact. <span class="m_2533161059319932354m_6184673939296804927HOEnZb"><font color="#888888"><div><br></div><div>Gaetan</div></font></span></div><div class="m_2533161059319932354m_6184673939296804927HOEnZb"><div class="m_2533161059319932354m_6184673939296804927h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 28, 2017 at 2:38 PM, Austin Herrema <span dir="ltr"><<a href="mailto:aherrema@iastate.edu" target="_blank">aherrema@iastate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Gotcha. In that case, it seems I should be good without that line. I've gotten the compile to succeed, but upon attempting to import the module I get the following:<br><br><div>>>> import run_analysis_final</div><span><div>Traceback (most recent call last):</div><div> File "<stdin>", line 1, in <module></div></span><div>ImportError: dlopen(./run_analysis_final.so<wbr>, 2): Symbol not found: _run_analysis_</div><div> Referenced from: ./run_analysis_final.so</div><div> Expected in: flat namespace</div><div> in ./run_analysis_final.so<br><br>Seems I may have gotten the linking wrong somehow. Will keep searching, but the simplified makefile that I used is attached in case anyone thinks they might be able to spot the issue in it. That said, I do realize that this may be starting to reach beyond the scope of this mailing list so feel free to ignore...</div></div><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395HOEnZb"><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 28, 2017 at 2:31 PM, Gaetan Kenway <span dir="ltr"><<a href="mailto:gaetank@gmail.com" target="_blank">gaetank@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">You only get that file if you have wrapped a module explicitly in the .pyf file. If you haven't wrapped a module, that doesn't get created. <span class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262HOEnZb"><font color="#888888"><div><br></div><div>Gaetan</div></font></span></div><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262HOEnZb"><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 28, 2017 at 12:28 PM, Austin Herrema <span dir="ltr"><<a href="mailto:aherrema@iastate.edu" target="_blank">aherrema@iastate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Gaetan,<br><br>Thank you for this. With your help, I think I am getting close to getting this to work for my case. At the moment, I am hung up on the line of your makefile which reads "$(FF90) $(FF90_ALL_FLAGS) -I$(MAIN_DIR)/mod -c warpustruct-f2pywrappers2.f90"<wbr>. Am I correct that warpustruct-f2pywrappers2.f90 should be generated by f2py? If so, do you (or does anyone else) know the command for telling f2py to do so? At the moment I am using:<br><br>f2py run_analysis.f90 -m run_analysis -h run_analysis.pyf<br><br>to get the requisite .pyf and .c files, but no .f90 file. If I am wrong about the origin of this file, please do tell me!<br><br>Thank you,<br>Austin</div><div class="gmail_extra"><div><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262m_8589576032988851607h5"><br><div class="gmail_quote">On Mon, Mar 27, 2017 at 5:13 PM, Gaetan Kenway <span dir="ltr"><<a href="mailto:gaetank@gmail.com" target="_blank">gaetank@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Austin<div><br></div><div>Here is the full makefile for a code we use. The variables defined externally in a separate config file are:</div><div>$(FF90)</div><div>$(FF90_FLAGS) </div><div>$(LIBDIR)</div><div>$(PETSC_LINKER_FLAGS)</div><div>$(LINKER_FLAGS)</div><div>$(CGNS_LINKER_FLAGS)</div><div><br></div><div>$(PYTHON) </div><div>$(PYTHON-CONIFG)</div><div>$(F2PY) </div><div>(These are usually use python, python-config and f2py. You can overwrite as necessary)</div><div><br></div><div>$(CC)</div><div>$(CC_ALL_FLAGS)</div><div><br></div><div>This essentially just mimics what f2py does automatically but we found it easier to control exactly what is going on. Essentially you are just compiling exactly as you normally an executable, but instead make a .so (with the -shared option) and including the additional .o files generated by compiling the f2py-generated wrappers.</div><div><br></div><div>Hope this helps,</div><div>Gaetan</div></div><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262m_8589576032988851607m_-4088625238782754276HOEnZb"><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262m_8589576032988851607m_-4088625238782754276h5"><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Mar 25, 2017 at 5:38 AM, Lisandro Dalcin <span dir="ltr"><<a href="mailto:dalcinl@gmail.com" target="_blank">dalcinl@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On 22 March 2017 at 20:29, Barry Smith <span dir="ltr"><<a href="mailto:bsmith@mcs.anl.gov" target="_blank">bsmith@mcs.anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Lisandro,<br>
<br>
We've had a couple questions similar to this with f2py; is there a way we could add to the PETSc/SLEPc makefile rules something to allow people to trivially use f2py without having to make their own (often incorrect) manual command lines?<br>
<br>
Thanks<br>
<span class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262m_8589576032988851607m_-4088625238782754276m_-1715386386467366153m_5568433048286632261HOEnZb"><font color="#888888"><br></font></span></blockquote><div><br></div></span><div>Barry, it is quite hard and hacky to get f2py working in the general case. I think the email from Gaetan in this thread proves my point.</div><div><br></div><div>IMHO, it is easier to write a small Fortran source exposing the API to call using ISO_C_BINDINGS, then wrap that code with the more traditional C-based "static" tools (SWIG, Cython) or even "dynamically" with ctypes or cffi (which use dlopen'ing).</div><div><br></div><div><br></div></div><div><br></div>-- <br><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262m_8589576032988851607m_-4088625238782754276m_-1715386386467366153m_5568433048286632261gmail_signature" data-smartmail="gmail_signature">Lisandro Dalcin<br>============<br>Research Scientist<br>Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)<br>Extreme Computing Research Center (ECRC)<br>King Abdullah University of Science and Technology (KAUST)<br><a href="http://ecrc.kaust.edu.sa/" target="_blank">http://ecrc.kaust.edu.sa/</a><br><br>4700 King Abdullah University of Science and Technology<br>al-Khawarizmi Bldg (Bldg 1), Office # 0109<br>Thuwal 23955-6900, Kingdom of Saudi Arabia<br><a href="http://www.kaust.edu.sa" target="_blank">http://www.kaust.edu.sa</a><br><br>Office Phone: <a href="tel:+966%2012%20808%200459" value="+966128080459" target="_blank">+966 12 808-0459</a></div>
</div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div></div></div><span>-- <br><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262m_8589576032988851607m_-4088625238782754276gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><b style="font-size:12.8px"><font color="#666666">Austin Herrema</font></b><br></div><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><div><font color="#999999">PhD Student | Graduate Research Assistant | Iowa State University</font></div><div><font color="#999999">Wind Energy Science, Engineering, and Policy | Mechanical Engineering</font></div></div></div></div>
</span></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="m_2533161059319932354m_6184673939296804927m_-3556955220701190395m_5895790734905528262gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><b style="font-size:12.8px"><font color="#666666">Austin Herrema</font></b><br></div><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><div><font color="#999999">PhD Student | Graduate Research Assistant | Iowa State University</font></div><div><font color="#999999">Wind Energy Science, Engineering, and Policy | Mechanical Engineering</font></div></div></div></div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><span class="HOEnZb"><font color="#888888"><div><br></div>-- <br><div class="m_2533161059319932354m_6184673939296804927gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><b style="font-size:12.8px"><font color="#666666">Austin Herrema</font></b><br></div><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><div><font color="#999999">PhD Student | Graduate Research Assistant | Iowa State University</font></div><div><font color="#999999">Wind Energy Science, Engineering, and Policy | Mechanical Engineering</font></div></div></div></div>
</font></span></div><span class="HOEnZb"><font color="#888888">
</font></span></div></div></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div class="m_2533161059319932354gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><b style="font-size:12.8px"><font color="#666666">Austin Herrema</font></b><br></div><div dir="ltr" style="color:rgb(136,136,136);font-size:12.8px"><div><font color="#999999">PhD Student | Graduate Research Assistant | Iowa State University</font></div><div><font color="#999999">Wind Energy Science, Engineering, and Policy | Mechanical Engineering</font></div></div></div></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div>
</div></div>