<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas",serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Stefan, <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Is there a way to run this without the flock option? For a variety of reasons we don’t support it on our system and I have been trying to run nek but having the same issue. Are there any compile flags that I can use or a different version of mpi perhaps? Recommendations welcome.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Thanks, <o:p></o:p></p><p class=MsoNormal> Julie<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:12.0pt;color:black'>From: </span></b><span style='font-size:12.0pt;color:black'>Nek5000-users <nek5000-users-bounces@lists.mcs.anl.gov> on behalf of "nek5000-users@lists.mcs.anl.gov" <nek5000-users@lists.mcs.anl.gov><br><b>Reply-To: </b>"nek5000-users@lists.mcs.anl.gov" <nek5000-users@lists.mcs.anl.gov><br><b>Date: </b>Monday, April 16, 2018 at 7:49 PM<br><b>To: </b>"nek5000-users@lists.mcs.anl.gov" <nek5000-users@lists.mcs.anl.gov><br><b>Subject: </b>Re: [Nek5000-users] Problem on cluster<o:p></o:p></span></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><a name="_MailOriginalBody">Before updating, can you please check if the following advise helps:<o:p></o:p></a></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>If the file system is LUSTRE, ensure that the directory is mounted with the 'flock' option<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><br><br><o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>Stefan<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='mso-bookmark:_MailOriginalBody'><br>On 16 Apr 2018, at 19:39, "</span><a href="mailto:nek5000-users@lists.mcs.anl.gov"><span style='mso-bookmark:_MailOriginalBody'>nek5000-users@lists.mcs.anl.gov</span><span style='mso-bookmark:_MailOriginalBody'></span></a><span style='mso-bookmark:_MailOriginalBody'>" <</span><a href="mailto:nek5000-users@lists.mcs.anl.gov"><span style='mso-bookmark:_MailOriginalBody'>nek5000-users@lists.mcs.anl.gov</span><span style='mso-bookmark:_MailOriginalBody'></span></a><span style='mso-bookmark:_MailOriginalBody'>> wrote:<o:p></o:p></span></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>Please use the release tarball instead of the GitHub master! The error message suggest that your MPI installation is outdated -> some necessary MPIIO features are missing. I think updating MPI will do the trick.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>Stefan.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='mso-bookmark:_MailOriginalBody'><br>On 16 Apr 2018, at 17:46, "</span><a href="mailto:nek5000-users@lists.mcs.anl.gov"><span style='mso-bookmark:_MailOriginalBody'>nek5000-users@lists.mcs.anl.gov</span><span style='mso-bookmark:_MailOriginalBody'></span></a><span style='mso-bookmark:_MailOriginalBody'>" <</span><a href="mailto:nek5000-users@lists.mcs.anl.gov"><span style='mso-bookmark:_MailOriginalBody'>nek5000-users@lists.mcs.anl.gov</span><span style='mso-bookmark:_MailOriginalBody'></span></a><span style='mso-bookmark:_MailOriginalBody'>> wrote:<o:p></o:p></span></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>Dear Nek users, I'm having a problem when I use the latest version of Nek on a HPC cluster. I can compile, but when I run my simulations they finish. The logfile of a generic case is like the following: <o:p></o:p></span></p><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>===========================================================<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>/----------------------------------------------------------\\<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| _ __ ______ __ __ ______ ____ ____ ____ |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| / | / // ____// //_/ / ____/ / __ \\/ __ \\/ __ \\ |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| / |/ // __/ / ,< /___ \\ / / / // / / // / / / |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| / /| // /___ / /| | ____/ / / /_/ // /_/ // /_/ / |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| /_/ |_//_____//_/ |_|/_____/ \\___/ \\___/ \\___/ |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>|----------------------------------------------------------|<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| NEK5000: Open Source Spectral Element Solver |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| COPYRIGHT (c) 2008-2017 UCHICAGO ARGONNE, LLC |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| Version: 17.0-rc1 |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| Web: </span><a href="http://nek5000.mcs.anl.gov"><span style='mso-bookmark:_MailOriginalBody'>http://nek5000.mcs.anl.gov</span><span style='mso-bookmark:_MailOriginalBody'></span></a><span style='mso-bookmark:_MailOriginalBody'> |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>| |<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>\\----------------------------------------------------------/<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> Number of processors: 80<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> REAL wdsize : 8<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> INTEGER wdsize : 4<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> Timer accuracy : 0.00E+00<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> Reading /home/jrobinson/casos/Placa_6/Placa_6.rea <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> Reading /home/jrobinson/casos/Placa_6/Placa_6.re2 <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> mapping elements to processors<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> Reading /home/jrobinson/casos/Placa_6/Placa_6.ma2 <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> RANK 0 IEG 1754 1755 1756 1757 1758 1759 1760 1774<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1775 1776 1777 1778 1779 1780 1794 1795<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1796 1797 1798 1799 1800 1814 1815 1816<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1817 1818 1819 1820 1834 1835 1836 1837<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1838 1839 1840 1853 1854 1855 1856 1857<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1858 1859 1860 1873 1874 1875 1876 1877<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1878 1879 1880 1893 1894 1895 1896 1897<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1898 1899 1913 1914 1915 1916 1917 1918<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1919 1933 1934 1935 1936 1937 1938 1939<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1953 1954 1955 1956 1957 1958 1974 1975<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1976 1977 1978 1994 1995 1996 1997 1998<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 1999 2000 2014 2015 2016 2017 2018 2019<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 2020 9783 9784 9785 9786 9787 9788 9789<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 9790 9791 9792 9793 9794 9795 9796 9797<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 9798 9799 9800 9801 9802 9803 9804 9805<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 9806 9807 9808 9809 9810 9811 9812 9813<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 9814 9815 9816 9817 9818 9819 9820 9821<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 9822 9823 9824 9825 9826 9827 9828 9829<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> 9830 9849 9855 9856 9861 9862<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> element load imbalance: 1 150 151<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> done :: mapping 0.32155 sec<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'> preading mesh <o:p></o:p></span></p></div></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>=============================================================<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>So the last line is "preading mesh".<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>This doesn't give too much information, but the cluster generates a file with the following errors (at the end of this text).<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>When I use an old version of Nek on this same cluster, I have no problem running my cases. The problem is that I need to use the latest version because I'm using exo2nek routine for my meshes generated with Trelis (Cubit).<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>Any idea of what could I do?<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>Thank you all.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>Juan Pablo.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>=========================================================<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div><div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>This requires fcntl(2) to be implemented. As of 8/25/2011 it is not. Generic MPICH Message: File locking failed in ADIOI_Set_lock(fd 4A,cmd F_SETLKW/7,type F_RDLCK/0,whence 0) with return value FFFFFFFF and errno 26.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>- If the file system is NFS, you need to use NFS version 3, ensure that the lockd daemon is running on all the machines, and mount the directory with the 'noac' option (no attribute caching).<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>- If the file system is LUSTRE, ensure that the directory is mounted with the 'flock' option.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'>ADIOI_Set_lock:: Function not implemented<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='mso-bookmark:_MailOriginalBody'><o:p> </o:p></span></p></div></div></div></div></blockquote><pre><span style='mso-bookmark:_MailOriginalBody'>_______________________________________________<br><br><o:p></o:p></span></pre><pre><span style='mso-bookmark:_MailOriginalBody'>Nek5000-users mailing list<br><br><o:p></o:p></span></pre><pre><span style='mso-bookmark:_MailOriginalBody'></span><a href="mailto:Nek5000-users@lists.mcs.anl.gov"><span style='mso-bookmark:_MailOriginalBody'>Nek5000-users@lists.mcs.anl.gov</span><span style='mso-bookmark:_MailOriginalBody'></span></a><span style='mso-bookmark:_MailOriginalBody'><br><br><o:p></o:p></span></pre><pre><span style='mso-bookmark:_MailOriginalBody'></span><a href="https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users"><span style='mso-bookmark:_MailOriginalBody'>https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users</span><span style='mso-bookmark:_MailOriginalBody'></span></a><span style='mso-bookmark:_MailOriginalBody'><br><br><o:p></o:p></span></pre></div></blockquote></div></body></html>