<div><br><div class="gmail_quote"><div dir="auto">On Sat, Feb 10, 2018 at 9:29 AM Éric Chamberland <<a href="mailto:Eric.Chamberland@giref.ulaval.ca">Eric.Chamberland@giref.ulaval.ca</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    <p>Hi Matthew,</p>
    <p>Yes, I heard that the MPI C++ API has been deprecated.</p></div></blockquote><div dir="auto"><br></div><div dir="auto">It was <i>deprecated</i> in 2.2 and <i>deleted</i> in 3.0. Implementations may not delete the code but there is absolutely no expectation that it is supported by any recent implementation. </div><div dir="auto"><br></div><div dir="auto">I fought the MPI Forum until the bitter end about this, by the way, because we broke existing code in order to save the authors of new MPI functionality a few minutes of LaTeX boilerplate generation.</div><div dir="auto"><br></div><div dir="auto">It would be almost trivial to extract C++ bindings from MPICH and/or Open-MPI as a standalone project to fix these issues, but nobody has made the effort.</div><div dir="auto"><br></div><div dir="auto">Jeff</div><div dir="auto"><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><p></p>
    <p>Yes, the mpi_cxx is now missing.  Our link line is formed mainly
      with PETSC_WITH_EXTERNAL_LIB variable that is now:</p>
    <p>PETSC_WITH_EXTERNAL_LIB = -L/opt/petsc-master_debug/lib
      -Wl,-rpath,/opt/petsc-master_debug/lib
      -L/opt/petsc-master_debug/lib
      -L/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64
      -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/mkl/lib/intel64
      -Wl,-rpath,/opt/openmpi-1.10.2/lib -L/opt/openmpi-1.10.2/lib
      -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/4.8.5
      -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5
      -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64
      -L/opt/intel/composer_xe_2015.2.164/compiler/lib/intel64
      -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64
      -L/opt/intel/composer_xe_2015.2.164/ipp/lib/intel64
      -Wl,-rpath,/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4
      -L/opt/intel/composer_xe_2015.2.164/tbb/lib/intel64/gcc4.4 -lpetsc
      -lsuperlu -lsuperlu_dist -lHYPRE -lcmumps -ldmumps -lsmumps
      -lzmumps -lmumps_common -lpord -lmkl_scalapack_lp64
      -lmkl_blacs_openmpi_lp64 -lml -lumfpack -lklu -lcholmod -lbtf
      -lccolamd -lcolamd -lcamd -lamd -lsuitesparseconfig
      -lmkl_intel_lp64 -lmkl_core -lmkl_intel_thread
      -lmkl_blacs_intelmpi_lp64 -liomp5 -ldl -lpthread -lparmetis
      -lmetis -lptesmumps -lptscotch -lptscotcherr -lesmumps -lscotch
      -lscotcherr -lm -lX11 -lstdc++ -ldl -lmpi_usempi -lmpi_mpifh -lmpi
      -lgfortran -lm -lgfortran -lm -lgcc_s -lquadmath -lpthread -lrt
      -lm -lpthread -lz -lstdc++ -ldl<br>
    </p>
    <p>We add some stuff to this (our own compiled libs), but nothing
      related to MPI or PETSc since we used to rely on
      PETSC_WITH_EXTERNAL_LIB for all our diffrement environments (think
      about different petsc version, MPI libs and compilers: clang, icc,
      g++) and it used to work until yesterday changes...</p>
    <p>As you can see in the diff, this "block" of libraries have been
      removed from PETSC_WITH_EXTERNAL_LIB: (the -- is part of the diff
      output):<br>
    </p></div><div text="#000000" bgcolor="#FFFFFF">
    <pre style="color:rgb(0,0,0);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-wrap:break-word;white-space:pre-wrap">--ldl
--lmpi_cxx
--lmpi
--lstdc++
--lm
--lgcc_s
--lpthread</pre></div><div text="#000000" bgcolor="#FFFFFF">
    and into this block there was the "-lmpi_cxx" that we need...<br>
    <br>
    I could send you our whole line of link, but the error is into this
    small change introduced yesterday into master...<br>
    <br>
    Thanks a lot!<br>
    <br>
    Eric<br>
    <br>
    <div class="m_6121047145293031975moz-cite-prefix">Le 18-02-10 à 10:34, Matthew Knepley a
      écrit :<br>
    </div></div><div text="#000000" bgcolor="#FFFFFF">
    <blockquote type="cite">
      <div>
        <div class="gmail_extra">
          <div class="gmail_quote">On Sat, Feb 10, 2018 at 9:42 AM, Éric
            Chamberland <span><<a href="mailto:Eric.Chamberland@giref.ulaval.ca" target="_blank">Eric.Chamberland@giref.ulaval.ca</a>></span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF">
                <p>Hi,</p>
                <p>we used to link our c++ code with PETSc using
                  PETSC_WITH_EXTERNAL_LIB variable defined in
                  $PETSC_DIR/lib/petsc/conf/petscvariables and
                  everything was fine until this night.</p>
                <p>It seems some libs have vanished from this variable,
                  see the diff here:</p>
                <pre style="color:rgb(0,0,0);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-wrap:break-word;white-space:pre-wrap"> -lptscotcherr
 -lesmumps
 -lscotch
 -lscotcherr
 -lm
 -lX11
+-lstdc++
 -ldl
 -lmpi_usempi
 -lmpi_mpifh
 -lmpi
 -lgfortran
 -lm
 -lgfortran
 -lm
 -lgcc_s
 -lquadmath
 -lpthread
--ldl
--lmpi_cxx
--lmpi
--lstdc++
--lm
--lgcc_s
--lpthread
 -lrt
 -lm
 -lpthread
 -lz
+-lstdc++
 -ldl


causing these errors at link phase for us:

<font color="blue"> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Op::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)': </font>
/opt/openmpi-1.10.2/include/openmpi/ompi/mpi/cxx/op_inln.h:122: undefined reference to `ompi_mpi_cxx_op_intercept'
<font color="blue"> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Create_graph(int, int const*, int const*, bool) const': </font>
/opt/openmpi-1.10.2/include/openmpi/ompi/mpi/cxx/intracomm.h:25: undefined reference to `MPI::Comm::Comm()'
<font color="blue"> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intercomm::Merge(bool) const': </font>
/opt/openmpi-1.10.2/include/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()'
<font color="blue"> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Split(int, int) const': </font>
/opt/openmpi-1.10.2/include/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()'
<font color="blue"> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Create(MPI::Group const&) const': </font>
/opt/openmpi-1.10.2/include/openmpi/ompi/mpi/cxx/intracomm_inln.h:23: undefined reference to `MPI::Comm::Comm()'
<font color="blue"> /pmi/cmpbib/compilation_BIB_gcc_redhat_petsc-master_debug/COMPILE_AUTO/GIREF/obj/dev/StatistiqueMemoire.o: In function `MPI::Intracomm::Clone() const': </font></pre>
              </div>
            </blockquote>
            <div>Hi Eric,</div>
            <div><br>
            </div>
            <div>These symbols are all coming from -lmpi_cxx. I would
              note that I believe the MPI Forum has deprecated the C++
              interface, so it will</div>
            <div>eventually go away. However, lets fix this. In the
              configure log, I see that mpicxx -show has that library in
              it, so it seems that the link</div>
            <div>is not being done with the C++ compiler. Can you send
              the whole link line?</div>
            <div><br>
            </div>
            <div>  Thanks,</div>
            <div><br>
            </div>
            <div>     Matt</div>
            <div> </div>
            <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF">
                <p>Is this a normal and definitive change or an
                  unwanted/unobserved bug?</p>
                <p>Thanks,</p>
                <p>Eric</p>
                <p>ps: here are the logs:</p>
                <p>this night:</p>
                <p>---------</p>
                <pre style="color:rgb(0,0,0);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial;word-wrap:break-word;white-space:pre-wrap"><a class="m_6121047145293031975m_1095262550823765165moz-txt-link-freetext" href="http://www.giref.ulaval.ca/%7Ecmpgiref/petsc-master-debug/2018.02.10.02h00m01s_configure.log" target="_blank">http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.10.02h00m01s_configure.log</a>
<a class="m_6121047145293031975m_1095262550823765165moz-txt-link-freetext" href="http://www.giref.ulaval.ca/%7Ecmpgiref/petsc-master-debug/2018.02.10.02h00m01s_make.log" target="_blank">http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.10.02h00m01s_make.log</a>


a day before:
------------

<a class="m_6121047145293031975m_1095262550823765165moz-txt-link-freetext" href="http://www.giref.ulaval.ca/%7Ecmpgiref/petsc-master-debug/2018.02.09.02h00m02s_configure.log" target="_blank">http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.09.02h00m02s_configure.log</a>
<a class="m_6121047145293031975m_1095262550823765165moz-txt-link-freetext" href="http://www.giref.ulaval.ca/%7Ecmpgiref/petsc-master-debug/2018.02.09.02h00m02s_make.log" target="_blank">http://www.giref.ulaval.ca/~cmpgiref/petsc-master-debug/2018.02.09.02h00m02s_make.log</a>


</pre>
              </div>
            </blockquote>
          </div>
          <br>
          <br clear="all">
          <div><br>
          </div>
          -- <br>
          <div class="m_6121047145293031975gmail_signature" data-smartmail="gmail_signature">
            <div>
              <div>
                <div>
                  <div>What most experimenters take for granted before
                    they begin their experiments is infinitely more
                    interesting than any results to which their
                    experiments lead.<br>
                    -- Norbert Wiener</div>
                  <div><br>
                  </div>
                  <div><a href="http://www.caam.rice.edu/%7Emk51/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </div></blockquote></div></div><div dir="ltr">-- <br></div><div class="gmail_signature" data-smartmail="gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div>