<div dir="ltr">OK, I tried again and deleted the arch directory every time:<br><div><br></div><div>(base) 09:56 (84933e400a...)|BISECTING ~/Codes/petsc2$ git bisect good<br>d7dd068b66a8daa3a37c9e1556b34bd8def54922 is the first bad commit<br>commit d7dd068b66a8daa3a37c9e1556b34bd8def54922<br>Author: Vaclav Hapla <<a href="mailto:vaclav.hapla@erdw.ethz.ch">vaclav.hapla@erdw.ethz.ch</a>><br>Date: Wed Apr 14 21:22:37 2021 +0200<br><br> HDF5: Improve timestepping.<br><br> * add PetscViewerHDF5{Push,Pop,Is}Timestepping to control timestepping mode<br> * write timestepping attribute for datasets and check it on reading<br> * fail gracefully if trying to read non-timestepped dataset in timestepping mode and vice-versa (fix #425)<br> * rewrite src/vec/vec/tutorials/ex19.c to improve coverage for timestepping testing<br><br> doc/documentation/changes/dev.rst | 19 ++-<br> include/petsc/private/viewerhdf5impl.h | 2 +<br> include/petscviewerhdf5.h | 4 +<br> src/dm/impls/da/gr2.c | 21 ++--<br> src/dm/impls/plex/plexhdf5.c | 25 +++-<br> src/sys/classes/viewer/impls/hdf5/hdf5v.c | 124 ++++++++++++++++++--<br> src/vec/is/is/impls/general/general.c | 10 +-<br> src/vec/is/utils/hdf5io.c | 45 +++++---<br> src/vec/vec/impls/mpi/pdvec.c | 14 ++-<br> src/vec/vec/tutorials/ex19.c | 186 +++++++++++++++++-------------<br> src/vec/vec/tutorials/output/ex19_2.out | 0<br> src/vec/vec/tutorials/output/ex19_3.out | 0<br> src/vec/vec/tutorials/output/ex19_4.out | 0<br> 13 files changed, 320 insertions(+), 130 deletions(-)<br> delete mode 100644 src/vec/vec/tutorials/output/ex19_2.out<br> delete mode 100644 src/vec/vec/tutorials/output/ex19_3.out<br> delete mode 100644 src/vec/vec/tutorials/output/ex19_4.out<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 24, 2021 at 7:04 AM Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Thu, Jun 24, 2021 at 7:00 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">OK, this is what I get with bisect:<div><br></div><div>(base) 06:54 (c3b2925bfb...)|BISECTING ~/Codes/petsc2$ git bisect good<br>The merge base c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47 is bad.<br>This means the bug has been fixed between c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47 and [09da24df01e50defd94bc4f7396f866a808ecea5 c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47].<br></div><div><br></div><div>(base) 06:56 (v3.15.1) ~/Codes/petsc2$ git checkout c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47<br>Previous HEAD position was 09da24df01 Increase patchlevel to 3.15.1<br>HEAD is now at c3b2925bfb Merge branch 'knepley-main-patch-78504' into 'release'<br>(base) 06:56 (c3b2925bfb...) ~/Codes/petsc2$</div></div></blockquote><div><br></div><div>I do not understand. Neither of those commits do anything.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 11:14 PM Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Mark,<div> I am not sure what your problem is. If it is a regression, can you bisect it?</div><div><div><div dir="ltr"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 4:04 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I also tried commenting out the second VecView, so there is just one step in the file, and the .h5 file is only 8 bytes smaller and the .xmf file goes from 5373 bytes to 3090 bytes.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 4:01 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">It is not a device issue but it is a regression.<br><div><br></div><div>Landau ex1 is tiny and just calls VecView before and after the TSsolve, which is one time step. If you add "<b><font color="#ff0000">-dm_view hdf5:f.h5 -vec_view hdf5:f.h5::append -dm_landau_Ez 10.</font></b>" to landau/ex1 (see below), you get an h5 file with two time steps, as it should be. </div><div>This is a huge electric field, Ez=10, which makes the electron distribution (u_e) get visibly pulled off center. </div><div>In Visit, both time steps have identical data that is clearly after the solve and not the initial condition (see attached). </div><div><br></div><div>I ran this again with <span style="color:rgb(153,0,255)">-ex1_ts_max_steps 0 </span>and get the expected result of two steps/frames with the symmetric initial condition in both. THis is correct behavior.</div><div><br></div><div>Any ideas?</div><div>Thanks</div><font color="#00ffff"></font><div><br></div><div>diff --git a/src/ts/utils/dmplexlandau/tutorials/ex1.c b/src/ts/utils/dmplexlandau/tutorials/ex1.c<br>index 9e4c8f1b61..31dfda2fad 100644<br>--- a/src/ts/utils/dmplexlandau/tutorials/ex1.c<br>+++ b/src/ts/utils/dmplexlandau/tutorials/ex1.c<br>@@ -66,6 +66,6 @@ int main(int argc, char **argv)<br> test:<br> suffix: 0<br> requires: p4est !complex<br>- args: -petscspace_degree 3 -petscspace_poly_tensor 1 -dm_landau_type p4est -dm_landau_ion_masses 2,4 -dm_landau_ion_charges 1,18 -dm_landau_thermal_temps 5,5,.5 -dm_landau_n 1.00018,1,1e-5 -dm_landau_n_0 1e20 -ex1_ts_monitor -ex1_snes_rtol 1.e-14 -ex1_snes_stol 1.e-14 -ex1_snes_monitor -ex1_snes_converged_reason -ex1_ts_type arkimex -ex1_ts_arkimex_type 1bee -ex1_ts_max_snes_failures -1 -ex1_ts_rtol 1e-1 -ex1_ts_dt 1.e-1 -ex1_ts_max_time 1 -ex1_ts_adapt_clip .5,1.25 -ex1_ts_adapt_scale_solve_failed 0.75 -ex1_ts_adapt_time_step_increase_delay 5 -ex1_ts_max_steps 1 -ex1_pc_type lu -ex1_ksp_type preonly -dm_landau_amr_levels_max 7 -dm_landau_domain_radius 5 -dm_landau_amr_re_levels 0 -dm_landau_re_radius 1 -dm_landau_amr_z_refine1 1 -dm_landau_amr_z_refine2 0 -dm_landau_amr_post_refine 0 -dm_landau_z_radius1 .1 -dm_landau_z_radius2 .1 -dm_refine 1 -dm_landau_gpu_assembly false<br>+ args: -petscspace_degree 3 -petscspace_poly_tensor 1 -dm_landau_type p4est -dm_landau_ion_masses 2,4 -dm_landau_ion_charges 1,18 -dm_landau_thermal_temps 5,5,.5 -dm_landau_n 1.00018,1,1e-5 -dm_landau_n_0 1e20 -ex1_ts_monitor -ex1_snes_rtol 1.e-14 -ex1_snes_stol 1.e-14 -ex1_snes_monitor -ex1_snes_converged_reason -ex1_ts_type arkimex -ex1_ts_arkimex_type 1bee -ex1_ts_max_snes_failures -1 -ex1_ts_rtol 1e-1 -ex1_ts_dt 1.e-1 -ex1_ts_max_time 1 -ex1_ts_adapt_clip .5,1.25 -ex1_ts_adapt_scale_solve_failed 0.75 -ex1_ts_adapt_time_step_increase_delay 5<font color="#00ffff"> </font><font color="#9900ff">-ex1_ts_max_steps 1</font><font color="#00ffff"> </font>-ex1_pc_type lu -ex1_ksp_type preonly -dm_landau_amr_levels_max 7 -dm_landau_domain_radius 5 -dm_landau_amr_re_levels 0 -dm_landau_re_radius 1 -dm_landau_amr_z_refine1 1 -dm_landau_amr_z_refine2 0 -dm_landau_amr_post_refine 0 -dm_landau_z_radius1 .1 -dm_landau_z_radius2 .1 -dm_refine 1 -dm_landau_gpu_assembly false <b><font color="#ff0000">-dm_view hdf5:f.h5 -vec_view hdf5:f.h5::append -dm_landau_Ez 10.</font></b><br><br> TEST*/<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 1:38 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Landau ex1 should work. I will test.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 10:47 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, Jun 23, 2021 at 10:44 AM Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Use VecGetArrayRead/Write() to get up-to-date host pointers to the vector array.</div></blockquote><div><br></div><div>I think Mark is saying that those are not working. We do call VecGetArrayRead() in the HDF5 code.</div><div><br></div><div>Mark, it seem like a small broken code is necessary.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div dir="ltr"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 9:15 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">First, there seem to be two pages for VecGetArrayAndMemType (one has a pointer to the other).<div><br></div><div>So I need to get a CPU array for HDF5 viewing. Totally broken for devices.</div><div><br></div><div>I don't find a VecGetArrayCpu[HOST] that does the right thing. </div><div><br></div><div>Perhaps have VecGetArrayAndMemType return a valid CPU pointer when "mtype==NULL"?</div><div><br></div><div>Mark</div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>