<div dir="ltr">I am not sure how to proceed.<div><br></div><div>What I think is the current recommended practice of setting the sequence number that I usei n Landau/ex1 (src/ts/utils/dmplexlandau/tutorials):</div><div><br></div><div>ierr = DMSetOutputSequenceNumber(dm, 0, time);CHKERRQ(ierr);<br>ierr = VecViewFromOptions(X,NULL,"-vec_view");CHKERRQ(ierr);<br></div><div>....</div><div><div>ierr = DMSetOutputSequenceNumber(dm, 1, time);CHKERRQ(ierr);<br>ierr = VecViewFromOptions(X,NULL,"-vec_view");CHKERRQ(ierr);<br></div><div><br></div><div></div></div>Does not work after d7dd068b. Only the last View's data is in the .h5 file. The example in this commit (ex19) is hardwired for HDF5 and ex1 is not.<div>Ex1 is tiny (40 lines)<br><div><br></div><div>I assume we don't want to abandon the abstract interface and this idea of letting the Viewer keep track of the sequence number is a great idea.</div><div><br></div><div>So I'm not sure how to proceed. We could a) abandon the abstract interface and I will move to HDF5, b) someone fix this, or c) extend/abstract this idea to not hardwire for HDF5.</div><div>(c) would be ideal and I could see making it work for HDF5 (only) for now and fix other viewers later. I could then move my ex1 and ex2 to this and just use HDF5.<br><div><br></div></div><div>Thanks,</div><div>Mark</div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 24, 2021 at 5:30 PM Mark Adams <<a href="mailto:mfadams@lbl.gov">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">OK, I see that this change is aimed at _not_ having the application keep track of the time step and allow for incrementing the step number.<div>That is definitely a better design.</div><div>I will change these two tests to use this new system.</div><div>ex19 uses a Viewer. I would prefer the ViewFromOptions method. Not sure this works.</div><div><br></div><div>Thanks,</div><div>Mark</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 24, 2021 at 5:19 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 24, 2021 at 3:05 PM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Thu, Jun 24, 2021 at 9:58 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">OK, I tried again and deleted the arch directory every time:</div></blockquote><div><br></div><div>Okay, this changes the interface for OutputSequenceNumber. It _should_ be backwards compatible. The likely error I would think</div><div>is that it thinks you have a timestep at first but not later, however that should be an HDF5 error. Maybe it think the timestep is always</div><div>the same and overwrites?</div></div></div></blockquote><div><br></div><div>* The symptom seems to be that there is only one time step in the file and it is the last one. So yes, it seems to overwrite one file location over and over.</div><div><br></div><div>* I commented out the 2nd View call in ex1 and the file was only 8 bytes smaller (eg, the time step). The .xmf file was a bit under 2x bigger.</div><div><br></div><div>* Visit "sees" all the time steps. That is, I can step through or run a movie of all the time steps, but the data is exactly the same at all steps. I've run longer problems with ex2, for each time step.</div><div><br></div><div>Mark</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>(base) 09:56 (84933e400a...)|BISECTING ~/Codes/petsc2$ git bisect good<br>d7dd068b66a8daa3a37c9e1556b34bd8def54922 is the first bad commit<br>commit d7dd068b66a8daa3a37c9e1556b34bd8def54922<br>Author: Vaclav Hapla <<a href="mailto:vaclav.hapla@erdw.ethz.ch" target="_blank">vaclav.hapla@erdw.ethz.ch</a>><br>Date: Wed Apr 14 21:22:37 2021 +0200<br><br> HDF5: Improve timestepping.<br><br> * add PetscViewerHDF5{Push,Pop,Is}Timestepping to control timestepping mode<br> * write timestepping attribute for datasets and check it on reading<br> * fail gracefully if trying to read non-timestepped dataset in timestepping mode and vice-versa (fix #425)<br> * rewrite src/vec/vec/tutorials/ex19.c to improve coverage for timestepping testing<br><br> doc/documentation/changes/dev.rst | 19 ++-<br> include/petsc/private/viewerhdf5impl.h | 2 +<br> include/petscviewerhdf5.h | 4 +<br> src/dm/impls/da/gr2.c | 21 ++--<br> src/dm/impls/plex/plexhdf5.c | 25 +++-<br> src/sys/classes/viewer/impls/hdf5/hdf5v.c | 124 ++++++++++++++++++--<br> src/vec/is/is/impls/general/general.c | 10 +-<br> src/vec/is/utils/hdf5io.c | 45 +++++---<br> src/vec/vec/impls/mpi/pdvec.c | 14 ++-<br> src/vec/vec/tutorials/ex19.c | 186 +++++++++++++++++-------------<br> src/vec/vec/tutorials/output/ex19_2.out | 0<br> src/vec/vec/tutorials/output/ex19_3.out | 0<br> src/vec/vec/tutorials/output/ex19_4.out | 0<br> 13 files changed, 320 insertions(+), 130 deletions(-)<br> delete mode 100644 src/vec/vec/tutorials/output/ex19_2.out<br> delete mode 100644 src/vec/vec/tutorials/output/ex19_3.out<br> delete mode 100644 src/vec/vec/tutorials/output/ex19_4.out<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 24, 2021 at 7:04 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Thu, Jun 24, 2021 at 7:00 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">OK, this is what I get with bisect:<div><br></div><div>(base) 06:54 (c3b2925bfb...)|BISECTING ~/Codes/petsc2$ git bisect good<br>The merge base c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47 is bad.<br>This means the bug has been fixed between c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47 and [09da24df01e50defd94bc4f7396f866a808ecea5 c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47].<br></div><div><br></div><div>(base) 06:56 (v3.15.1) ~/Codes/petsc2$ git checkout c3b2925bfbe1c0a0b3a69c9a76295d0747e33d47<br>Previous HEAD position was 09da24df01 Increase patchlevel to 3.15.1<br>HEAD is now at c3b2925bfb Merge branch 'knepley-main-patch-78504' into 'release'<br>(base) 06:56 (c3b2925bfb...) ~/Codes/petsc2$</div></div></blockquote><div><br></div><div>I do not understand. Neither of those commits do anything.</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 11:14 PM Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Mark,<div> I am not sure what your problem is. If it is a regression, can you bisect it?</div><div><div><div dir="ltr"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 4:04 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I also tried commenting out the second VecView, so there is just one step in the file, and the .h5 file is only 8 bytes smaller and the .xmf file goes from 5373 bytes to 3090 bytes.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 4:01 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">It is not a device issue but it is a regression.<br><div><br></div><div>Landau ex1 is tiny and just calls VecView before and after the TSsolve, which is one time step. If you add "<b><font color="#ff0000">-dm_view hdf5:f.h5 -vec_view hdf5:f.h5::append -dm_landau_Ez 10.</font></b>" to landau/ex1 (see below), you get an h5 file with two time steps, as it should be. </div><div>This is a huge electric field, Ez=10, which makes the electron distribution (u_e) get visibly pulled off center. </div><div>In Visit, both time steps have identical data that is clearly after the solve and not the initial condition (see attached). </div><div><br></div><div>I ran this again with <span style="color:rgb(153,0,255)">-ex1_ts_max_steps 0 </span>and get the expected result of two steps/frames with the symmetric initial condition in both. THis is correct behavior.</div><div><br></div><div>Any ideas?</div><div>Thanks</div><font color="#00ffff"></font><div><br></div><div>diff --git a/src/ts/utils/dmplexlandau/tutorials/ex1.c b/src/ts/utils/dmplexlandau/tutorials/ex1.c<br>index 9e4c8f1b61..31dfda2fad 100644<br>--- a/src/ts/utils/dmplexlandau/tutorials/ex1.c<br>+++ b/src/ts/utils/dmplexlandau/tutorials/ex1.c<br>@@ -66,6 +66,6 @@ int main(int argc, char **argv)<br> test:<br> suffix: 0<br> requires: p4est !complex<br>- args: -petscspace_degree 3 -petscspace_poly_tensor 1 -dm_landau_type p4est -dm_landau_ion_masses 2,4 -dm_landau_ion_charges 1,18 -dm_landau_thermal_temps 5,5,.5 -dm_landau_n 1.00018,1,1e-5 -dm_landau_n_0 1e20 -ex1_ts_monitor -ex1_snes_rtol 1.e-14 -ex1_snes_stol 1.e-14 -ex1_snes_monitor -ex1_snes_converged_reason -ex1_ts_type arkimex -ex1_ts_arkimex_type 1bee -ex1_ts_max_snes_failures -1 -ex1_ts_rtol 1e-1 -ex1_ts_dt 1.e-1 -ex1_ts_max_time 1 -ex1_ts_adapt_clip .5,1.25 -ex1_ts_adapt_scale_solve_failed 0.75 -ex1_ts_adapt_time_step_increase_delay 5 -ex1_ts_max_steps 1 -ex1_pc_type lu -ex1_ksp_type preonly -dm_landau_amr_levels_max 7 -dm_landau_domain_radius 5 -dm_landau_amr_re_levels 0 -dm_landau_re_radius 1 -dm_landau_amr_z_refine1 1 -dm_landau_amr_z_refine2 0 -dm_landau_amr_post_refine 0 -dm_landau_z_radius1 .1 -dm_landau_z_radius2 .1 -dm_refine 1 -dm_landau_gpu_assembly false<br>+ args: -petscspace_degree 3 -petscspace_poly_tensor 1 -dm_landau_type p4est -dm_landau_ion_masses 2,4 -dm_landau_ion_charges 1,18 -dm_landau_thermal_temps 5,5,.5 -dm_landau_n 1.00018,1,1e-5 -dm_landau_n_0 1e20 -ex1_ts_monitor -ex1_snes_rtol 1.e-14 -ex1_snes_stol 1.e-14 -ex1_snes_monitor -ex1_snes_converged_reason -ex1_ts_type arkimex -ex1_ts_arkimex_type 1bee -ex1_ts_max_snes_failures -1 -ex1_ts_rtol 1e-1 -ex1_ts_dt 1.e-1 -ex1_ts_max_time 1 -ex1_ts_adapt_clip .5,1.25 -ex1_ts_adapt_scale_solve_failed 0.75 -ex1_ts_adapt_time_step_increase_delay 5<font color="#00ffff"> </font><font color="#9900ff">-ex1_ts_max_steps 1</font><font color="#00ffff"> </font>-ex1_pc_type lu -ex1_ksp_type preonly -dm_landau_amr_levels_max 7 -dm_landau_domain_radius 5 -dm_landau_amr_re_levels 0 -dm_landau_re_radius 1 -dm_landau_amr_z_refine1 1 -dm_landau_amr_z_refine2 0 -dm_landau_amr_post_refine 0 -dm_landau_z_radius1 .1 -dm_landau_z_radius2 .1 -dm_refine 1 -dm_landau_gpu_assembly false <b><font color="#ff0000">-dm_view hdf5:f.h5 -vec_view hdf5:f.h5::append -dm_landau_Ez 10.</font></b><br><br> TEST*/<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 1:38 PM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Landau ex1 should work. I will test.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 10:47 AM Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Wed, Jun 23, 2021 at 10:44 AM Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Use VecGetArrayRead/Write() to get up-to-date host pointers to the vector array.</div></blockquote><div><br></div><div>I think Mark is saying that those are not working. We do call VecGetArrayRead() in the HDF5 code.</div><div><br></div><div>Mark, it seem like a small broken code is necessary.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div dir="ltr"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 23, 2021 at 9:15 AM Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">First, there seem to be two pages for VecGetArrayAndMemType (one has a pointer to the other).<div><br></div><div>So I need to get a CPU array for HDF5 viewing. Totally broken for devices.</div><div><br></div><div>I don't find a VecGetArrayCpu[HOST] that does the right thing. </div><div><br></div><div>Perhaps have VecGetArrayAndMemType return a valid CPU pointer when "mtype==NULL"?</div><div><br></div><div>Mark</div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div>
</blockquote></div></div>
</blockquote></div>
</blockquote></div>