<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">Hi Barry, all,</div>
<div class=""><br class="">
</div>
<div class="">I’m trying to understand some extremely old code (4 years now!) that people insist on using and therefore having me support, despite the fact that the DOE doesn’t think it is important enough to pay me to support.</div>
<div class=""><br class="">
</div>
<div class="">Barry — can you explain to me the history and logic in your change:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://bitbucket.org/petsc/petsc/commits/bd1fc5ae41626b6cf1674a6070035cfd93e0c1dd" class="">https://bitbucket.org/petsc/petsc/commits/bd1fc5ae41626b6cf1674a6070035cfd93e0c1dd</a></div>
<div class=""><br class="">
</div>
<div class="">that removed DMDA’s local-to-global map, in favor of doing the reverse scatter on global-to-local? When I wrote the aforementioned LBM code, the local-to-global scatter had different semantics than the reverse global-to-local scatter. The latter
merged owned and ghosted values from the local Vec into the owned global Vec, while the former ignored ghost values and did a direct copy from owned local values to owned global values. Under INSERT_VALUES, this was important as the latter was a race condition
while the former was a well-posed operation.</div>
<div class=""><br class="">
</div>
<div class="">Now I see that this L2G scatter has been removed (in 2014), which introduced a race condition. It was tweaked a bit for 1 process by this commit:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://bitbucket.org/petsc/petsc/commits/1eb28f2e8c580cb49316c983b5b6ec6c58d77ab8" class="">https://bitbucket.org/petsc/petsc/commits/1eb28f2e8c580cb49316c983b5b6ec6c58d77ab8</a></div>
<div class=""><br class="">
</div>
<div class="">which refers to a Lisandro email that I’m having trouble finding. Fortunately this caused some errors that I did see, as opposed to the previous race conditions which I didn’t see.</div>
<div class=""><br class="">
</div>
<div class="">Additionally there was documentation added to not do INSERT_VALUES with DMDAs at some point. (Maybe because it causes race conditions!) This documentation suggests to "<span style="font-family: Times; font-size: medium; background-color: rgb(255, 255, 255);" class="">simply
compute the values directly into a global vector instead of a local one."</span> This isn’t a good choice in my application, where I do many time steps in local vectors, using repeated calls to “DMLocalToLocalBegin()”, and only go back to the global vector
when I want to do i/o. Computing into global vectors requires managing two Vecs throughout the entire code, when otherwise I only manage one (except in the i/o portion of the code). I guess the answer is to create and use a L2G forward scatter myself? Is
there a better solution I’m not thinking of?</div>
<div class=""><br class="">
</div>
<div class="">Thanks,</div>
<div class=""><br class="">
</div>
<div class="">Ethan</div>
<div class=""><br class="">
</div>
<br class="">
<div apple-content-edited="true" class=""><br class="">
------------------------------------------------------------------------<br class="">
Ethan Coon<br class="">
Research Scientist<br class="">
Computational Earth Science -- EES-16<br class="">
Los Alamos National Laboratory<br class="">
505-665-8289<br class="">
<br class="">
<a href="http://www.lanl.gov/expertise/profiles/view/ethan-coon" class="">http://www.lanl.gov/expertise/profiles/view/ethan-coon</a><br class="">
------------------------------------------------------------------------ </div>
<br class="">
</body>
</html>