<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><span style="font-family: Calibri, sans-serif; font-size: 11pt;" class=""><br class=""></span></div><span style="font-family: Calibri, sans-serif; font-size: 11pt;" class="">PetscMatlabEngineEvaluate(mpc->mengine,"clc; clear all; close all; addpath([cd,'/MPC_matlab_code'])");</span><div class=""><span style="font-family: Calibri, sans-serif; font-size: 11pt;" class=""><br class=""></span></div><div class=""><span style="font-family: Calibri, sans-serif; font-size: 11pt;" class="">Undefined function or variable 'load_mpc_parameters'.</span></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class=""><br class=""></span></font></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class="">My guess is that the 'cd' may be different on the different machines, hence it does not do the addpath() you expect and hence cannot find the script </span></font><span style="font-family: Calibri, sans-serif; font-size: 14.666666984558105px;" class="">load_mpc_parameters ?</span></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class=""><br class=""></span></font></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class="">In the branch I provided I have just now added more output messages if you run with the PETSc option -info that might indicate what is happening. So you can git pull; make libs and run again.</span></font></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class=""><br class=""></span></font></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class=""><br class=""></span></font></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class="">Barry</span></font></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class=""><br class=""></span></font></div><div class=""><font face="Calibri, sans-serif" class=""><span style="font-size: 14.666666984558105px;" class=""><br class=""></span></font><div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class="">On Nov 12, 2020, at 1:48 AM, Kaustubh Khedkar <<a href="mailto:kkhedkar9879@sdsu.edu" class="">kkhedkar9879@sdsu.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252" class="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)" class="">
<style class=""><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Menlo;
panose-1:2 11 6 9 3 8 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word" class="">
<div class="WordSection1"><p class="MsoNormal">Hi Barry,<o:p class=""></o:p></p><p class="MsoNormal">I used the branch <b class=""><span style="font-size:12.0pt;font-family:Menlo;color:#C814C9" class="">barry/2020-11-10/add-matlab-engine-host.
</span></b>Now it appears that the matlab engine starts can start on any node but it is not able to execute any matlab script. If I revert back to PETSc 3.13.5 the head compute node is able to execute the MATLAB script. See the following error (load_mpc_parameters.m
is a MATLAB script that the code is trying to execute):<b class=""><span style="font-size:12.0pt;font-family:Menlo;color:#C814C9" class=""><o:p class=""></o:p></span></b></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal" style="margin-left:.5in">Starting Matlab engine...<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""> </o:p></p><p class="MsoNormal" style="margin-left:.5in">Matlab engine started<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Storing variables in Matlab workspace...<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""> </o:p></p><p class="MsoNormal" style="margin-left:.5in">Undefined function or variable 'load_mpc_parameters'.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Error using save<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Variable 'a' not found.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: Error in external library<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: Unable to get array a from matlab<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: See <a href="https://www.mcs.anl.gov/petsc/documentation/faq.html" class="">https://www.mcs.anl.gov/petsc/documentation/faq.html</a> for trouble shooting.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: unknown<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: ./main3d on a linux-opt named node35 by kkhedkar Thu Nov 12 02:39:47 2020<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: Configure options --CC=/home/kkhedkar/sfw/mpich3.3_mellanox/3.3/bin/mpicc --CXX=/home/kkhedkar/sfw/mpich3.3_mellanox/3.3/bin/mpicxx --FC=/home/kkhedkar/sfw/mpich3.3_mellanox/3.3/bin/mpif90 --COPTFLAGS=-O3
--CXXOPTFLAGS=-O3 --FOPTFLAGS=-O3 --PETSC_ARCH=linux-opt --with-debugging=0 --download-hypre=1 --with-x=0 --download-fblaslapack=1 --with-matlab-dir=/nas/app/MATLAB/matlab2019a/ --with-matlab-engine=1 --with-matlab-engine-dir=/nas/app/MATLAB/matlab2019a/extern/engines/<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #1 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Error using save<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Variable 'h' not found.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #2 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Error using save<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Variable 'mpc_start_time' not found.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #3 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Error using save<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Variable 'sample_size' not found.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""> </o:p></p><p class="MsoNormal" style="margin-left:.5in">sample_size = 0<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #4 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Error using save<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Variable 'dt_sampling' not found.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">Storing up past excitation force samples every 0 time-steps.<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in"><o:p class=""> </o:p></p><p class="MsoNormal" style="margin-left:.5in">[0]PETSC ERROR: #5 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:.5in">IBRedundantInitializer: Deallocating initialization data.<o:p class=""></o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal">My code is using Petsc matlab engine in this way (“master” is the hostname of the head node where MATLAB is licensed):<o:p class=""></o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal" style="margin-left:1.0in">if(SAMRAI_MPI::getRank() == 0)<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> {<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> std::cout << "\nStarting Matlab engine...\n" << std::endl;<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineCreate(PETSC_COMM_SELF, "master", &(mpc->mengine));<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> std::cout << "Matlab engine started" << std::endl; <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineEvaluate(mpc->mengine,"clc; clear all; close all; addpath([cd,'/MPC_matlab_code'])");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> std::cout << "Storing variables in Matlab workspace...\n" << std::endl;<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineEvaluate(mpc->mengine,"global hydro wave; wave.H_wave = %f ; wave.Tp = %f; m = %f;", input_db->getDouble("HEIGHT"), input_db->getDouble("TIME_PERIOD"), input_db->getDouble("MASS"));<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineEvaluate(mpc->mengine,"load_mpc_parameters; calculate_mpc_matrices");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &(mpc->m_plus_Ainf), "a");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &(mpc->dt_controller), "h");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &(mpc->mpc_start_time), "mpc_start_time");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscScalar sample_size;<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &sample_size, "sample_size");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> mpc->sample_size = int(sample_size);<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> std::cout << "\nsample_size = " << mpc->sample_size << std::endl;<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> mpc->F_past.resize(mpc->sample_size);<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> mpc->t_past.resize(mpc->sample_size);<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> double dt = input_db->getDouble("DT_MAX");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> mpc->initial_position = input_db->getDouble("ZCOM");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> mpc->current_position = mpc->initial_position;<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscScalar dt_sampling;<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &dt_sampling, "dt_sampling");<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> mpc->sampling_interval = int(dt_sampling / dt);<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> <o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> std::cout << "Storing up past excitation force samples every " << mpc->sampling_interval << " time-steps.\n" << std::endl;<o:p class=""></o:p></p><p class="MsoNormal" style="margin-left:1.0in"> }<o:p class=""></o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p><p class="MsoNormal"><o:p class=""> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in" class=""><p class="MsoNormal"><b class=""><span style="font-size: 12pt;" class="">From: </span></b><span style="font-size: 12pt;" class="">Barry Smith <<a href="mailto:bsmith@petsc.dev" class="">bsmith@petsc.dev</a>><br class="">
<b class="">Date: </b>Tuesday, November 10, 2020 at 5:22 PM<br class="">
<b class="">To: </b>Kaustubh Khedkar <<a href="mailto:kkhedkar9879@sdsu.edu" class="">kkhedkar9879@sdsu.edu</a>><br class="">
<b class="">Cc: </b>"<a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a>" <<a href="mailto:petsc-users@mcs.anl.gov" class="">petsc-users@mcs.anl.gov</a>><br class="">
<b class="">Subject: </b>Re: [petsc-users] Using Petsc-Matlab engine on a cluster<o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class=""><p class="MsoNormal"><o:p class=""> </o:p></p>
</div><p class="MsoNormal"> In the git branch <b class=""><span style="font-size:12.0pt;font-family:Menlo;color:#C814C9" class="">barry/2020-11-10/add-matlab-engine-host </span></b> I have added the option -matlab_engine_host hostname <o:p class=""></o:p></p>
<div class=""><p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class=""><p class="MsoNormal"> Note this does require you have ssh access to hostname from where your PETSc program is running which may require setting up some ssh files<o:p class=""></o:p></p>
</div>
<div class=""><p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class=""><p class="MsoNormal"> Good luck<o:p class=""></o:p></p>
</div>
<div class=""><p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class=""><p class="MsoNormal"> Barry<o:p class=""></o:p></p>
</div>
<div class=""><p class="MsoNormal"><o:p class=""> </o:p></p>
<div class=""><p class="MsoNormal"><br class="">
<br class="">
<o:p class=""></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt" class="">
<div class=""><p class="MsoNormal">On Nov 9, 2020, at 5:35 PM, Kaustubh Khedkar <<a href="mailto:kkhedkar9879@sdsu.edu" class="">kkhedkar9879@sdsu.edu</a>> wrote:<o:p class=""></o:p></p>
</div><p class="MsoNormal"><o:p class=""> </o:p></p>
<div class="">
<div class="">
<div class=""><p class="MsoNormal">Hello,<span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal">I am using PETSc with Matlab engine. I have built an application that uses Matlab engine through PETSc interface. I am trying to run this application on a cluster which has Matlab installed on a network drive (nas drive). I have made sure
that I am able to access Matlab using my university details on the cluster (by activating Matlab license through<span class="apple-converted-space"> </span><a href="http://mathworks.com/" class="">Mathworks.com</a><span class="apple-converted-space"> </span>account).
I am also able to run this application using mpirun on the head node (master) which starts the Matlab engine but if I try to run this application using slurm on compute node then the Matlab engine does not start. I have also confirmed that if I login into
one of the compute nodes and do a mpirun the Matlab engine does not start there. If I try to launch Matlab on a compute node it asks for Matlab license which the compute node is unable to activate possibly because it is not connected to the internet directly.<span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal"> <span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal">Is there a way start matlab engine on the head node through Petsc interface even though the application is running on compute nodes or is there any other work around this issue?<span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal"> <span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal"> <span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal">Thanks,<span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
<div class=""><p class="MsoNormal">Kaustubh Khedkar<span style="font-size:12.0pt" class=""><o:p class=""></o:p></span></p>
</div>
</div>
</div>
</blockquote>
</div><p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
</div>
</div>
</div></blockquote></div><br class=""></div></body></html>