<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
p.msipfooter6d2e06ff, li.msipfooter6d2e06ff, div.msipfooter6d2e06ff
{mso-style-name:msipfooter6d2e06ff;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">One strange thing I just found out.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Compile a *.c file make it work.<o:p></o:p></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">mpiicc -o ex5.o -c -fPIC -wd1572 -Ofast -xHost -I /NFS/home/home3/kjiao/software/petsc_3.10.4/include/ -I/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl/include
-I/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include lsqr.c<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal">Compile a *.cpp file does not work, even though .cpp file is exactly same as .c file.<o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">mpiicc -o ex5.o -c -Ofast -xHost -I /NFS/home/home3/kjiao/software/petsc_3.10.4/include/ -I/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl/include
-I/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include lsqr.cpp</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>From:</b> Mark Adams <mfadams@lbl.gov> <br>
<b>Sent:</b> Tuesday, March 26, 2019 3:38 PM<br>
<b>To:</b> Kun Jiao <KJiao@slb.com><br>
<b>Cc:</b> petsc-users <petsc-users@mcs.anl.gov><br>
<b>Subject:</b> Re: [petsc-users] [Ext] Re: error: identifier "MatCreateMPIAIJMKL" is undefined in 3.10.4<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Mar 26, 2019 at 3:00 PM Kun Jiao <<a href="mailto:KJiao@slb.com">KJiao@slb.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal">Strange things, when I compile my code in the test dir in PETSC, it works. After I "make install" PETSC, and try to compile my code against the installed PETSC, it doesn't work any more.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm not sure I follow what you are doing exactly but look at the compile lines (good and bad) and compare them. If one works and one does not then they must be different.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Anyway, as Satish said this interface was not enabled in any version that we see. (So we are puzzled that any version works.) You can wait for a fix to get pushed but using the method that I showed you should work now.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal"><br>
I guess this is what you means. <br>
<br>
Is there any way to reenable MatCreateMPIAIJMKL public interface?<br>
<br>
And, I am using intel MKL, here is my configure option:<br>
<br>
Configure Options: --configModules=PETSc.Configure --optionsModule=config.compilerOptions PETSC_ARCH=linux-gnu-intel --with-precision=single --with-cc=mpiicc --with-cxx=mpiicc --with-fc=mpiifort --with-mpi-include=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mpi/intel64/include
--with-mpi-lib="-L/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel//compilers_and_libraries_2019.2.187/linux/mpi/intel64/lib -lmpifort -lmpi_ilp64" --with-blaslapack-lib="-L/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl/lib/intel64
-Wl, --no-as-needed -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm -ldl" --with-scalapack-lib="-L/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64"
--with-scalapack-include=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl/include --with-mkl_pardiso-dir=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl
--with-mkl_sparse=1 --with-mkl_sparse-dir=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl --with-mkl_cpardiso=1 --with-mkl_cpardiso-dir=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl
--with-mkl_sparse_optimize=1 --with-mkl_sparse_optimize-dir=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl --with-mkl_sparse_sp2m=1 --with-mkl_sparse_sp2m-dir=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/intel/compilers_and_libraries_2019.2.187/linux/mkl
--with-cmake=1 --prefix=/wgdisk/hy3300/source_code_dev/imaging/kjiao/software/petsc_3.9.4 --known-endian=big --with-debugging=0 --COPTFLAGS=" -Ofast -xHost" --CXXOPTFLAGS=" -Ofast -xHost" --FOPTFLAGS=" -Ofast -xHost" --with-x=0<br>
Working directory: /wgdisk/hy3300/source_code_dev/imaging/kjiao/petsc-3.10.4<br>
<br>
<br>
<br>
Schlumberger-Private<br>
<br>
-----Original Message-----<br>
From: Balay, Satish <<a href="mailto:balay@mcs.anl.gov" target="_blank">balay@mcs.anl.gov</a>>
<br>
Sent: Tuesday, March 26, 2019 10:19 AM<br>
To: Kun Jiao <<a href="mailto:KJiao@slb.com" target="_blank">KJiao@slb.com</a>><br>
Cc: Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>;
<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
Subject: Re: [petsc-users] [Ext] Re: error: identifier "MatCreateMPIAIJMKL" is undefined in 3.10.4<br>
<br>
>>>>>>><br>
balay@sb /home/balay/petsc (maint=)<br>
$ git grep MatCreateMPIAIJMKL maint-3.8<br>
maint-3.8:src/mat/impls/aij/mpi/aijmkl/mpiaijmkl.c: MatCreateMPIAIJMKL - Creates a sparse parallel matrix whose local<br>
maint-3.8:src/mat/impls/aij/mpi/aijmkl/mpiaijmkl.c:PetscErrorCode MatCreateMPIAIJMKL(MPI_Comm comm,PetscInt m,PetscInt n,PetscInt M,PetscInt N,PetscInt d_nz,const PetscInt d_nnz[],PetscInt o_nz,const PetscInt o_nnz[],Mat *A)<br>
maint-3.8:src/mat/impls/aij/mpi/aijmkl/mpiaijmkl.c:.seealso: MatCreateMPIAIJMKL(), MATSEQAIJMKL, MATMPIAIJMKL<br>
maint-3.8:src/mat/impls/aij/seq/aijmkl/aijmkl.c:.seealso: MatCreate(), MatCreateMPIAIJMKL(), MatSetValues() balay@sb /home/balay/petsc (maint=) $ git grep MatCreateMPIAIJMKL maint<br>
maint:src/mat/impls/aij/mpi/aijmkl/mpiaijmkl.c: MatCreateMPIAIJMKL - Creates a sparse parallel matrix whose local<br>
maint:src/mat/impls/aij/mpi/aijmkl/mpiaijmkl.c:PetscErrorCode MatCreateMPIAIJMKL(MPI_Comm comm,PetscInt m,PetscInt n,PetscInt M,PetscInt N,PetscInt d_nz,const PetscInt d_nnz[],PetscInt o_nz,const PetscInt o_nnz[],Mat *A)<br>
maint:src/mat/impls/aij/mpi/aijmkl/mpiaijmkl.c:.seealso: MatCreateMPIAIJMKL(), MATSEQAIJMKL, MATMPIAIJMKL<br>
maint:src/mat/impls/aij/seq/aijmkl/aijmkl.c:.seealso: MatCreate(), MatCreateMPIAIJMKL(), MatSetValues() balay@sb /home/balay/petsc (maint=) $ <<<<<<<<<<<<br>
<br>
MatCreateMPIAIJMKL() exists in both petsc-3.8 and petsc-3.10. However the public interface is missing from both of these versions. So I'm surprised you don't get the same error with petsc-3.8<br>
<br>
Can you try the following change?<br>
<br>
diff --git a/include/petscmat.h b/include/petscmat.h index 1b8ac69377..c66f727994 100644<br>
--- a/include/petscmat.h<br>
+++ b/include/petscmat.h<br>
@@ -223,7 +223,8 @@ typedef enum {DIFFERENT_NONZERO_PATTERN,SUBSET_NONZERO_PATTERN,SAME_NONZERO_PATT<br>
<br>
#if defined PETSC_HAVE_MKL_SPARSE<br>
PETSC_EXTERN PetscErrorCode MatCreateBAIJMKL(MPI_Comm,PetscInt,PetscInt,PetscInt,PetscInt,PetscInt,PetscInt,const PetscInt[],PetscInt,const PetscInt[],Mat*); -PETSC_EXTERN PetscErrorCode MatCreateSeqBAIJMKL(MPI_Comm comm,PetscInt bs,PetscInt m,PetscInt n,PetscInt
nz,const PetscInt nnz[],Mat *A);<br>
+PETSC_EXTERN PetscErrorCode <br>
+MatCreateSeqBAIJMKL(MPI_Comm,PetscInt,PetscInt,PetscInt,PetscInt,const <br>
+PetscInt[],Mat*); PETSC_EXTERN PetscErrorCode <br>
+MatCreateMPIAIJMKL(MPI_Comm,PetscInt,PetscInt,PetscInt,PetscInt,PetscIn<br>
+t,const PetscInt[],PetscInt,const PetscInt[],Mat*);<br>
#endif<br>
<br>
PETSC_EXTERN PetscErrorCode MatCreateSeqSELL(MPI_Comm,PetscInt,PetscInt,PetscInt,const PetscInt[],Mat*);<br>
<br>
<br>
Also note: - this routine is available only when PETSc is built with Intel MKL<br>
<br>
Satish<br>
<br>
On Tue, 26 Mar 2019, Kun Jiao via petsc-users wrote:<br>
<br>
> [kjiao@hyi0016 src/lsqr]% make<br>
> [ 50%] Building CXX object lsqr/CMakeFiles/p_lsqr.dir/lsqr.cc.o<br>
> /wgdisk/hy3300/source_code_dev/imaging/kjiao/src/git/src/lsqr/lsqr.cc(318): error: identifier "MatCreateMPIAIJMKL" is undefined<br>
> ierr = MatCreateMPIAIJMKL(comm,m,n,M,N,maxnz,dialens,maxnz,offlens,&A);CHKERRQ(ierr);<br>
> ^<br>
> <br>
> /wgdisk/hy3300/source_code_dev/imaging/kjiao/src/git/src/lsqr/lsqr.cc(578): error: identifier "MatCreateMPIAIJMKL" is undefined<br>
> ierr = MatCreateMPIAIJMKL(comm,m,n,M,N,maxnz,dialens,maxnz,offlens,&A);CHKERRQ(ierr);<br>
> ^<br>
> <br>
> compilation aborted for <br>
> /wgdisk/hy3300/source_code_dev/imaging/kjiao/src/git/src/lsqr/lsqr.cc <br>
> (code 2)<br>
> <br>
> Thanks.<br>
> <br>
> <br>
> From: Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>><br>
> Sent: Tuesday, March 26, 2019 9:22 AM<br>
> To: Kun Jiao <<a href="mailto:KJiao@slb.com" target="_blank">KJiao@slb.com</a>><br>
> Cc: <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><br>
> Subject: Re: [Ext] Re: [petsc-users] error: identifier <br>
> "MatCreateMPIAIJMKL" is undefined in 3.10.4<br>
> <br>
> I assume the whole error message will have the line of code. Please send the whole error message and line of offending code if not included.<br>
> <br>
> On Tue, Mar 26, 2019 at 10:08 AM Kun Jiao <<a href="mailto:KJiao@slb.com" target="_blank">KJiao@slb.com</a><mailto:<a href="mailto:KJiao@slb.com" target="_blank">KJiao@slb.com</a>>> wrote:<br>
> It is compiling error, error message is:<br>
> <br>
> error: identifier "MatCreateMPIAIJMKL" is undefined.<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> From: Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a><mailto:<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>><br>
> Sent: Tuesday, March 26, 2019 6:48 AM<br>
> To: Kun Jiao <<a href="mailto:KJiao@slb.com" target="_blank">KJiao@slb.com</a><mailto:<a href="mailto:KJiao@slb.com" target="_blank">KJiao@slb.com</a>>><br>
> Cc: <a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><mailto:<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br>
> Subject: [Ext] Re: [petsc-users] error: identifier <br>
> "MatCreateMPIAIJMKL" is undefined in 3.10.4<br>
> <br>
> Please send the output of the error (runtime, compile time, link <br>
> time?)<br>
> <br>
> On Mon, Mar 25, 2019 at 10:50 PM Kun Jiao via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><mailto:<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>>> wrote:<br>
> Hi Petsc Experts,<br>
> <br>
> Is MatCreateMPIAIJMKL retired in 3.10.4?<br>
> <br>
> I got this error with my code which works fine in 3.8.3 version.<br>
> <br>
> Regards,<br>
> Kun<br>
> <br>
> <br>
> <br>
> Schlumberger-Private<br>
> <br>
> <br>
> Schlumberger-Private<br>
> <br>
> <br>
> Schlumberger-Private<br>
> <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="msipfooter6d2e06ff" align="center" style="margin:0in;margin-bottom:.0001pt;text-align:center">
<span style="font-size:10.0pt;color:black">Schlumberger-Private</span><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</body>
</html>