<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 2.0cm 2.0cm;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="IT" link="blue" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Dear Satish,</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">let me first mention that using the OpenMPI runtime for running the executable built on top of the PETSc-mpich toolchain just came as an act of despair and was just a command away (despite knowing the ABI initiative is based on mpich).
I already had OpenMPI in Cygwin because I was planning to move there in case of PETSc failing to install MPICH. But as PETSc managed to handle that, that route is on hold for now (and I have not even tought about a plan for destributing it).</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Also, I didn’t mess up myself with any PATH variable nor I installed anything in system paths, and I Always checked in both systems (cygwin terminal and Windows prompt) that the mpiexec that was used was the correct one.</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For what I can tell, just copying all the mpi executables made by PETSc in the same folder of my executable (say, a folder on the desktop), going there with the cygwin terminal or the Windows prompt and running from there, actually worked
(yet, with the differences I specified in the first mail). So, in this very case, the idea would be that the user just installs cygwin and runs my executable from within the folder I send to him.</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If MS-MPI or Intel MPI had worked, it wouldn’t have been a problem (in my view) to let them install one of them, as long as a trivial install would have worked.</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Inviato da <a href="https://go.microsoft.com/fwlink/?LinkId=550986">
Posta</a> per Windows 10</p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="border:none;padding:0cm"><b>Da: </b><a href="mailto:balay@mcs.anl.gov">Satish Balay</a><br>
<b>Inviato: </b>domenica 28 giugno 2020 16:30<br>
<b>A: </b><a href="mailto:petsc-users@mcs.anl.gov">Satish Balay via petsc-users</a><br>
<b>Cc: </b><a href="mailto:paololampitella@hotmail.com">Paolo Lampitella</a>; <a href="mailto:pierre.jolivet@enseeiht.fr">
Pierre Jolivet</a><br>
<b>Oggetto: </b>Re: [petsc-users] PETSc and Windows 10</p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">BTW: How does redistributing MPI/runtime work with all the choices you have?<br>
<br>
For ex: with MS-MPI, Intel-MPI - wouldn't the user have to install these packages? [i.e you can't just copy them over to a folder and have mpiexec work - from what I can tell]<br>
<br>
And how did you plan on installing MPICH - but make mpiexec from OpenMPI redistributable? Did you use OpeMPI from cygwin - or install it manually?<br>
<br>
And presumably you don't want users installing cygwin.<br>
<br>
Satish<br>
<br>
On Sun, 28 Jun 2020, Satish Balay via petsc-users wrote:<br>
<br>
> On Sun, 28 Jun 2020, Paolo Lampitella wrote:<br>
> <br>
> > Dear PETSc users,<br>
> > <br>
> > I’ve been an happy PETSc user since version 3.3, using it both under Ubuntu (from 14.04 up to 20.04) and CentOS (from 5 to 8).<br>
> > <br>
> > I use it as an optional component for a parallel Fortran code (that, BTW, also uses metis) and, wherever allowed, I used to install myself MPI (both MPICH and OpenMPI) and PETSc on top of it without any trouble ever (besides being, myself, as dumb as one
can be in this).<br>
> > <br>
> > I did this on top of gnu compilers and, less extensively, intel compilers, both on a range of different systems (from virtual machines, to workstations to actual clusters).<br>
> > <br>
> > So far so good.<br>
> > <br>
> > Today I find myself in the need of deploying my application to Windows 10 users, which means giving them a folder with all the executables and libraries to make them run in it, including the mpi runtime. Unfortunately, I also have to rely on free tools
(can’t afford Intel for the moment).<br>
> > <br>
> > To the best of my knowledge, considering also far from optimal solutions, my options would then be: Virtual machines and WSL1, Cygwin, MSYS2-MinGW64, Cross compiling with MinGW64 from within Linux, PGI + Visual Studio + Cygwin (not sure about this one)<br>
> > <br>
> > I know this is largely unsupported, but I was wondering if there is, nonetheless, some general (and more official) knowledge available on the matter. What I tried so far:<br>
> > <br>
> > <br>
> > 1. Virtual machines and WSL1: both work like a charm, just like in the native OS, but very far from ideal for the distribution purpose<br>
> > <br>
> > <br>
> > 1. Cygwin with gnu compilers (as opposed to using Intel and Visual Studio): I was unable to compile myself MPI as I am used to on Linux, so I just tried going all in and let PETSc do everything for me (using static linking): download and install MPICH,
BLAS, LAPACK, METIS and HYPRE. Everything just worked (for now compiling and making trivial tests) and I am able to use everything from within a cygwin terminal (even with executables and dependencies outside cygwin). Still, even within cygwin, I can’t switch
to use, say, the cygwin ompi mpirun/mpiexec for an mpi program compiled with PETSc mpich (things run but not as expected). Some troubles start when I try to use cmd.exe (which I pictured as the more natural way to launch in Windows). In particular, using (note
that \ is in cmd.exe, / was used in cygwin terminal):<br>
> <br>
> I don't understand. Why build with MPICH - but use mpiexec from OpenMPI?<br>
> <br>
> If it is because you can easily redistribute OpenMPI - why not build PETSc with OpenMPI?<br>
> <br>
> You can't use Intel/MS-MPI from cygwin/gcc/gfortran<br>
> <br>
> Also - even-though --download-mpich works with cygwin/gcc - its no loner supported on windows [by MPICH group].<br>
> <br>
> > <br>
> > .\mpiexec.hydra.exe -np 8 .\my.exe<br>
> > <br>
> > Nothing happens unless I push Enter a second time. Things seem to work then, but if I try to run a serial executable with the command above I get the following errors (which, instead, doesn’t happen using the cygwin terminal):<br>
> > <br>
> > [proxy:0:0@Dell7540-Paolo] HYDU_sock_write (utils/sock/sock.c:286): write error (No such process)<br>
> > [proxy:0:0@Dell7540-Paolo] HYD_pmcd_pmip_control_cmd_cb (pm/pmiserv/pmip_cb.c:935): unable to write to downstream stdin<br>
> > [proxy:0:0@Dell7540-Paolo] HYDT_dmxu_poll_wait_for_event (tools/demux/demux_poll.c:76): callback returned error status<br>
> > [proxy:0:0@Dell7540-Paolo] main (pm/pmiserv/pmip.c:206): demux engine error waiting for event<br>
> > [mpiexec@Dell7540-Paolo] control_cb (pm/pmiserv/pmiserv_cb.c:200): assert (!closed) failed<br>
> > [mpiexec@Dell7540-Paolo] HYDT_dmxu_poll_wait_for_event (tools/demux/demux_poll.c:76): callback returned error status<br>
> > [mpiexec@Dell7540-Paolo] HYD_pmci_wait_for_completion (pm/pmiserv/pmiserv_pmci.c:198): error waiting for event<br>
> > [mpiexec@Dell7540-Paolo] main (ui/mpich/mpiexec.c:336): process manager error waiting for completion<br>
> > <br>
> > Just for the sake of completeness, I also tried using the Intel and Microsoft MPI redistributables, which might be more natural candidates, instead of the petsc compiled version of the MPI runtime (and they are MPICH derivatives, after all). But, running
with:<br>
> > <br>
> > mpiexec -np 1 my.exe<br>
> > <br>
> > I get the following error with Intel:<br>
> > <br>
> > [cli_0]: write_line error; fd=440 buf=:cmd=init pmi_version=1 pmi_subversion=1<br>
> > :<br>
> > system msg for write_line failure : Bad file descriptor<br>
> > [cli_0]: Unable to write to PMI_fd<br>
> > [cli_0]: write_line error; fd=440 buf=:cmd=get_appnum<br>
> > :<br>
> > system msg for write_line failure : Bad file descriptor<br>
> > Fatal error in MPI_Init: Other MPI error, error stack:<br>
> > MPIR_Init_thread(467):<br>
> > MPID_Init(140).......: channel initialization failed<br>
> > MPID_Init(421).......: PMI_Get_appnum returned -1<br>
> > [cli_0]: aborting job:<br>
> > Fatal error in MPI_Init: Other MPI error, error stack:<br>
> > MPIR_Init_thread(467):<br>
> > MPID_Init(140).......: channel initialization failed<br>
> > MPID_Init(421).......: PMI_Get_appnum returned -1<br>
> > <br>
> > And the following error with MS-MPI:<br>
> > <br>
> > [unset]: unable to decode hostport from 44e5747b-d19e-4ea8-ac7a-ec2102cabb21<br>
> > Fatal error in MPI_Init: Other MPI error, error stack:<br>
> > MPIR_Init_thread(467):<br>
> > MPID_Init(140).......: channel initialization failed<br>
> > MPID_Init(403).......: PMI_Init returned -1<br>
> > [unset]: aborting job:<br>
> > Fatal error in MPI_Init: Other MPI error, error stack:<br>
> > MPIR_Init_thread(467):<br>
> > MPID_Init(140).......: channel initialization failed<br>
> > MPID_Init(403).......: PMI_Init returned -1<br>
> > <br>
> > independently from the number of processes, but more processes produce more copies of this. However, both Intel and MS-MPI are able to run a serial fortran executable built with cygwin. I think I made everything correctly and adding -localhost didn’t help
(actually, it caused more problems to the interpretation of the cmd line arguments for mpiexec)<br>
> > <br>
> > <br>
> > 1. Cygwin with MinGW64 compilers. Never managed to compile MPI, not even trough PETSc.<br>
> > <br>
> > <br>
> > <br>
> > 1. MSYS2+MinGW64 compilers. I understood that MinGW is not well supported, probably because of how it handles paths, but I wanted to give it a try, because it should be more “native” and there seems to be relevant examples out there that managed to do
it. I first tried with the msys2 mpi distribution, produced the .mod file out of the mpi.f90 file in the distribution (I tried my best with different hacks from known limitations of this file as also present in the official MS-MPI distribution) and tried with
my code without petsc, but it failed in compiling the code with some strange MPI related error (argument mismatch between two unrelated MPI calls in the code, which is non sense to me). In contrast, simple mpi tests (hello world like) worked as expected. Then
I decided to follow this:<br>
> > <br>
> > <br>
> > <br>
> > <a href="https://doc.freefem.org/introduction/installation.html#compilation-on-windows">
https://doc.freefem.org/introduction/installation.html#compilation-on-windows</a><br>
> > <br>
> > <br>
> > <br>
> > but the exact same type of error came up (MPI calls in my code were different, but the error was the same). Trying again from scratch (i.e., without all the things I did in the beginning to compile my code) the same error came up in compiling some of the
freefem dependencies (this time not even mpi calls).<br>
> > <br>
> > <br>
> > <br>
> > As a side note, there seems to be an official effort in porting petsc to msys2 (<a href="https://github.com/okhlybov/MINGW-packages/tree/whpc/mingw-w64-petsc">https://github.com/okhlybov/MINGW-packages/tree/whpc/mingw-w64-petsc</a>), but it didn’t get into
the official packages yet, which I interpret as a warning<br>
> > <br>
> > <br>
> > <br>
> > 1. Didn’t give a try to cross compiling with MinGw from Linux, as I tought it couldn’t be any better than doing it from MSYS2<br>
> > 2. Didn’t try PGI as I actually didn’t know if I would then been able to make PETSc work.<br>
> > <br>
> > So, here there are some questions I have with respect to where I stand now and the points above:<br>
> > <br>
> > <br>
> > * I haven’t seen the MSYS2-MinGw64 toolchain mentioned at all in official documentation/discussions. Should I definitely abandon it (despite someone mentioning it as working) because of known issues?<br>
> <br>
> I don't have experience with MSYS2-MinGw64, However Pierre does - and perhaps can comment on this. I don't know how things work on the fortran side.<br>
> <br>
> > * What about the PGI route? I don’t see it mentioned as well. I guess it would require some work on win32fe<br>
> <br>
> Again - no experience here.<br>
> <br>
> > * For my Cygwin-GNU route (basically what is mentioned in PFLOTRAN documentation), am I expected to then run from the cygwin terminal or should the windows prompt work as well? Is the fact that I require a second Enter hit and the mismanagement of
serial executables the sign of something wrong with the Windows prompt?<br>
> <br>
> I would think Cygwin-GNU route should work. I'll have to see if I can reproduce the issues you have.<br>
> <br>
> Satish<br>
> <br>
> > * More generally, is there some known working, albeit non official, route given my constraints (free+fortran+windows+mpi+petsc)?<br>
> > <br>
> > Thanks for your attention and your great work on PETSc<br>
> > <br>
> > Best regards<br>
> > <br>
> > Paolo Lampitella<br>
> > <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>