<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
On May 9, 2024, at 2: 52 PM, Shatanawi, Sawsan Muhammad via petsc-users <petsc-users@ mcs. anl. gov> wrote:
</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBanner8txhfwp" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBanner8txhfwp" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBanner8txhfwp" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBanner8txhfwp" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html><head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBanner8txhfwp { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButton8txhfwp:hover, .pfptPrimaryButton8txhfwp:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButton8txhfwp:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On May 9, 2024, at 2:52 PM, Shatanawi, Sawsan Muhammad via petsc-users <petsc-users@mcs.anl.gov> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div dir="ltr" id="pfptBanneru80ly4f" style="color-scheme: revert !important; writing-mode: revert !important; -webkit-ruby-position: revert !important; color: revert !important; font-family: revert !important; font-feature-settings: revert !important; font-kerning: revert !important; font-optical-sizing: revert !important; font-palette: revert !important; font-size: revert !important; font-size-adjust: revert !important; font-stretch: revert !important; font-style: revert !important; font-synthesis-small-caps: revert !important; font-synthesis-style: revert !important; font-synthesis-weight: revert !important; font-variant-alternates: revert !important; font-variant-caps: revert !important; font-variant-east-asian: revert !important; font-variant-emoji: revert !important; font-variant-ligatures: revert !important; font-variant-numeric: revert !important; font-variant-position: revert !important; font-variation-settings: revert !important; font-weight: revert !important; text-orientation: revert !important; text-rendering: revert !important; zoom: revert !important; -webkit-font-smoothing: revert !important; -webkit-locale: revert !important; -webkit-text-size-adjust: revert !important; -webkit-text-zoom: revert !important; letter-spacing: revert !important; text-autospace: revert !important; text-spacing-trim: revert !important; accent-color: revert !important; align-content: revert !important; align-items: revert !important; align-self: revert !important; alignment-baseline: revert !important; alt: revert !important; animation-composition: revert !important; animation: revert !important; animation-timeline: revert !important; appearance: revert !important; aspect-ratio: revert !important; backface-visibility: revert !important; background-attachment: revert !important; background-blend-mode: revert !important; background-image: revert !important; background-position: revert !important; background-repeat: revert !important; background-size: revert !important; baseline-shift: revert !important; block-step-insert: revert !important; block-step-size: revert !important; border-collapse: revert !important; border-image: revert !important; box-sizing: revert !important; break-after: revert !important; break-before: revert !important; break-inside: revert !important; buffered-rendering: revert !important; caption-side: revert !important; caret-color: revert !important; clear: revert !important; clip: revert !important; clip-path: revert !important; clip-rule: revert !important; color-interpolation: revert !important; color-interpolation-filters: revert !important; column-count: revert !important; column-fill: revert !important; column-gap: revert !important; column-rule-color: revert !important; column-rule-style: revert !important; column-rule-width: revert !important; column-span: revert !important; column-width: revert !important; contain: revert !important; container-name: revert !important; container-type: revert !important; content: revert !important; content-visibility: revert !important; counter-increment: revert !important; counter-reset: revert !important; counter-set: revert !important; cursor: revert !important; cx: revert !important; cy: revert !important; dominant-baseline: revert !important; empty-cells: revert !important; fill: revert !important; fill-opacity: revert !important; fill-rule: revert !important; filter: revert !important; flex: revert !important; flex-flow: revert !important; float: revert !important; flood-color: revert !important; flood-opacity: revert !important; glyph-orientation-horizontal: revert !important; glyph-orientation-vertical: revert !important; grid: revert !important; grid-column-end: revert !important; grid-column-start: revert !important; grid-row-end: revert !important; grid-row-start: revert !important; hanging-punctuation: revert !important; hyphenate-character: revert !important; hyphens: revert !important; image-orientation: revert !important; image-rendering: revert !important; input-security: revert !important; isolation: revert !important; justify-content: revert !important; justify-items: revert !important; justify-self: revert !important; kerning: revert !important; lighting-color: revert !important; line-break: revert !important; line-height: revert !important; list-style: revert !important; margin-trim: revert !important; marker: revert !important; mask: revert !important; mask-size: revert !important; mask-type: revert !important; masonry-auto-flow: revert !important; math-style: revert !important; mix-blend-mode: revert !important; object-fit: revert !important; object-position: revert !important; offset-anchor: revert !important; offset-distance: revert !important; offset-path: revert !important; offset-position: revert !important; offset-rotate: revert !important; order: revert !important; orphans: revert !important; outline: revert !important; outline-offset: revert !important; overflow-anchor: revert !important; overflow-wrap: revert !important; overflow: revert !important; page: revert !important; paint-order: revert !important; perspective: revert !important; perspective-origin: revert !important; pointer-events: revert !important; position: revert !important; print-color-adjust: revert !important; quotes: revert !important; r: revert !important; resize: revert !important; rotate: revert !important; row-gap: revert !important; rx: revert !important; ry: revert !important; scale: revert !important; scroll-behavior: revert !important; scroll-snap-align: revert !important; scroll-snap-stop: revert !important; scroll-snap-type: revert !important; scroll-timeline: revert !important; scrollbar-color: revert !important; scrollbar-gutter: revert !important; scrollbar-width: revert !important; shape-image-threshold: revert !important; shape-margin: revert !important; shape-outside: revert !important; shape-rendering: revert !important; size: revert !important; speak-as: revert !important; stop-color: revert !important; stop-opacity: revert !important; stroke: revert !important; stroke-color: revert !important; stroke-dasharray: revert !important; stroke-dashoffset: revert !important; stroke-linecap: revert !important; stroke-linejoin: revert !important; stroke-miterlimit: revert !important; stroke-opacity: revert !important; stroke-width: revert !important; tab-size: revert !important; table-layout: revert !important; text-align: left !important; text-align-last: revert !important; text-anchor: revert !important; text-box-edge: revert !important; text-box-trim: revert !important; text-decoration-color: revert !important; text-decoration: revert !important; text-decoration-skip-ink: revert !important; text-decoration-style: revert !important; text-decoration-thickness: revert !important; text-emphasis-color: revert !important; text-emphasis-position: revert !important; text-emphasis-style: revert !important; text-group-align: revert !important; text-indent: revert !important; text-justify: revert !important; text-overflow: revert !important; text-shadow: revert !important; text-transform: revert !important; text-underline-offset: revert !important; text-underline-position: revert !important; text-wrap: revert !important; touch-action: revert !important; transform: revert !important; transform-box: revert !important; transform-origin: revert !important; transform-style: revert !important; transition: revert !important; translate: revert !important; vector-effect: revert !important; vertical-align: revert !important; view-timeline: revert !important; view-transition-name: revert !important; white-space: revert !important; widows: revert !important; will-change: revert !important; word-break: revert !important; x: revert !important; y: revert !important; z-index: revert !important; -apple-color-filter: revert !important; -apple-pay-button-style: revert !important; -apple-pay-button-type: revert !important; -internal-text-autosizing-status: revert !important; border-spacing: revert !important; -webkit-box-align: revert !important; -webkit-box-decoration-break: revert !important; -webkit-box-direction: revert !important; -webkit-box-flex: revert !important; -webkit-box-flex-group: revert !important; -webkit-box-lines: revert !important; -webkit-box-ordinal-group: revert !important; -webkit-box-orient: revert !important; -webkit-box-pack: revert !important; -webkit-box-reflect: revert !important; -webkit-column-axis: revert !important; -webkit-column-progression: revert !important; -webkit-cursor-visibility: revert !important; -webkit-font-size-delta: revert !important; -webkit-hyphenate-limit-after: revert !important; -webkit-hyphenate-limit-before: revert !important; -webkit-hyphenate-limit-lines: revert !important; -webkit-initial-letter: revert !important; -webkit-line-align: revert !important; -webkit-line-box-contain: revert !important; -webkit-line-clamp: revert !important; -webkit-line-grid: revert !important; -webkit-line-snap: revert !important; -webkit-marquee-direction: revert !important; -webkit-marquee-increment: revert !important; -webkit-marquee-repetition: revert !important; -webkit-marquee-speed: revert !important; -webkit-marquee-style: revert !important; -webkit-nbsp-mode: revert !important; -webkit-rtl-ordering: revert !important; -webkit-text-decorations-in-effect: revert !important; -webkit-text-fill-color: revert !important; -webkit-text-security: revert !important; -webkit-text-stroke-color: revert !important; -webkit-text-stroke-width: revert !important; -webkit-user-drag: revert !important; -webkit-user-modify: revert !important; -webkit-user-select: revert !important; word-spacing: revert !important; backdrop-filter: revert !important; background-clip: revert !important; background-origin: revert !important; box-shadow: revert !important; mask-composite: revert !important; mask-mode: revert !important; text-combine-upright: revert !important; -webkit-backdrop-filter: revert !important; -webkit-background-clip: revert !important; -webkit-background-origin: revert !important; -webkit-box-shadow: revert !important; -webkit-mask: revert !important; -webkit-mask-composite: revert !important; -webkit-mask-source-type: revert !important; -webkit-text-combine: revert !important; display: block !important; visibility: revert !important; opacity: revert !important; background-color: rgb(208, 216, 220) !important; block-size: revert !important; border-block: revert !important; border-bottom-color: revert !important; border-bottom-style: revert !important; border-bottom-width: revert !important; border-end-end-radius: revert !important; border-end-start-radius: revert !important; border-inline: revert !important; border-left-color: revert !important; border-left-style: revert !important; border-left-width: revert !important; border-right-color: revert !important; border-right-style: revert !important; border-right-width: revert !important; border-start-end-radius: revert !important; border-start-start-radius: revert !important; inset: revert !important; contain-intrinsic-block-size: revert !important; contain-intrinsic-size: revert !important; contain-intrinsic-inline-size: revert !important; height: revert !important; inline-size: revert !important; inset-block: revert !important; inset-inline: revert !important; margin-block: revert !important; margin-inline: revert !important; max-block-size: revert !important; max-height: revert !important; max-inline-size: revert !important; max-width: revert !important; min-block-size: revert !important; min-height: revert !important; min-inline-size: revert !important; overscroll-behavior-block: revert !important; overscroll-behavior-inline: revert !important; overscroll-behavior: revert !important; padding-block: revert !important; padding-inline: revert !important; scroll-margin-block: revert !important; scroll-margin: revert !important; scroll-margin-inline: revert !important; scroll-padding-block: revert !important; scroll-padding: revert !important; scroll-padding-inline: revert !important; width: revert !important; margin: 16px 0px !important; padding: 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; border-top-width: 4px !important; border-top-style: solid !important; border-top-color: rgb(144, 164, 174) !important;"><div id="pfptBanneru80ly4f" style="color-scheme: unset !important; writing-mode: unset !important; -webkit-ruby-position: unset !important; color: unset !important; font-family: unset !important; font-feature-settings: unset !important; font-kerning: unset !important; font-optical-sizing: unset !important; font-palette: unset !important; font-size: unset !important; font-size-adjust: unset !important; font-stretch: unset !important; font-style: unset !important; font-synthesis-small-caps: unset !important; font-synthesis-style: unset !important; font-synthesis-weight: unset !important; font-variant-alternates: unset !important; font-variant-caps: unset !important; font-variant-east-asian: unset !important; font-variant-emoji: unset !important; font-variant-ligatures: unset !important; font-variant-numeric: unset !important; font-variant-position: unset !important; font-variation-settings: unset !important; font-weight: unset !important; text-orientation: unset !important; text-rendering: unset !important; zoom: unset !important; -webkit-font-smoothing: unset !important; -webkit-locale: unset !important; -webkit-text-size-adjust: unset !important; -webkit-text-zoom: unset !important; letter-spacing: unset !important; text-autospace: unset !important; text-spacing-trim: unset !important; accent-color: unset !important; align-content: unset !important; align-items: unset !important; align-self: unset !important; alignment-baseline: unset !important; alt: unset !important; animation-composition: unset !important; animation: unset !important; animation-timeline: unset !important; appearance: unset !important; aspect-ratio: unset !important; backface-visibility: unset !important; background: unset !important; background-blend-mode: unset !important; baseline-shift: unset !important; block-step-insert: unset !important; block-step-size: unset !important; border-collapse: unset !important; border: unset !important; box-sizing: unset !important; break-after: unset !important; break-before: unset !important; break-inside: unset !important; buffered-rendering: unset !important; caption-side: unset !important; caret-color: unset !important; clear: unset !important; clip: unset !important; clip-path: unset !important; clip-rule: unset !important; color-interpolation: unset !important; color-interpolation-filters: unset !important; column-count: unset !important; column-fill: unset !important; column-gap: unset !important; column-rule-color: unset !important; column-rule-style: unset !important; column-rule-width: unset !important; column-span: unset !important; column-width: unset !important; contain: unset !important; container-name: unset !important; container-type: unset !important; content: unset !important; content-visibility: unset !important; counter-increment: unset !important; counter-reset: unset !important; counter-set: unset !important; cursor: unset !important; cx: unset !important; cy: unset !important; dominant-baseline: unset !important; empty-cells: unset !important; fill: unset !important; fill-opacity: unset !important; fill-rule: unset !important; filter: unset !important; flex: unset !important; flex-flow: unset !important; float: left !important; flood-color: unset !important; flood-opacity: unset !important; glyph-orientation-horizontal: unset !important; glyph-orientation-vertical: unset !important; grid: unset !important; grid-column-end: unset !important; grid-column-start: unset !important; grid-row-end: unset !important; grid-row-start: unset !important; hanging-punctuation: unset !important; hyphenate-character: unset !important; hyphens: unset !important; image-orientation: unset !important; image-rendering: unset !important; input-security: unset !important; isolation: unset !important; justify-content: unset !important; justify-items: unset !important; justify-self: unset !important; kerning: unset !important; lighting-color: unset !important; line-break: unset !important; line-height: unset !important; list-style: unset !important; margin-trim: unset !important; marker: unset !important; mask: unset !important; mask-size: unset !important; mask-type: unset !important; masonry-auto-flow: unset !important; math-style: unset !important; mix-blend-mode: unset !important; object-fit: unset !important; object-position: unset !important; offset-anchor: unset !important; offset-distance: unset !important; offset-path: unset !important; offset-position: unset !important; offset-rotate: unset !important; order: unset !important; orphans: unset !important; outline: unset !important; outline-offset: unset !important; overflow-anchor: unset !important; overflow-wrap: unset !important; overflow: unset !important; page: unset !important; paint-order: unset !important; perspective: unset !important; perspective-origin: unset !important; pointer-events: unset !important; position: unset !important; print-color-adjust: unset !important; quotes: unset !important; r: unset !important; resize: unset !important; rotate: unset !important; row-gap: unset !important; rx: unset !important; ry: unset !important; scale: unset !important; scroll-behavior: unset !important; scroll-snap-align: unset !important; scroll-snap-stop: unset !important; scroll-snap-type: unset !important; scroll-timeline: unset !important; scrollbar-color: unset !important; scrollbar-gutter: unset !important; scrollbar-width: unset !important; shape-image-threshold: unset !important; shape-margin: unset !important; shape-outside: unset !important; shape-rendering: unset !important; size: unset !important; speak-as: unset !important; stop-color: unset !important; stop-opacity: unset !important; stroke: unset !important; stroke-color: unset !important; stroke-dasharray: unset !important; stroke-dashoffset: unset !important; stroke-linecap: unset !important; stroke-linejoin: unset !important; stroke-miterlimit: unset !important; stroke-opacity: unset !important; stroke-width: unset !important; tab-size: unset !important; table-layout: unset !important; text-align: unset !important; text-align-last: unset !important; text-anchor: unset !important; text-box-edge: unset !important; text-box-trim: unset !important; text-decoration-color: unset !important; text-decoration: unset !important; text-decoration-skip-ink: unset !important; text-decoration-style: unset !important; text-decoration-thickness: unset !important; text-emphasis-color: unset !important; text-emphasis-position: unset !important; text-emphasis-style: unset !important; text-group-align: unset !important; text-indent: unset !important; text-justify: unset !important; text-overflow: unset !important; text-shadow: unset !important; text-transform: unset !important; text-underline-offset: unset !important; text-underline-position: unset !important; text-wrap: unset !important; touch-action: unset !important; transform: unset !important; transform-box: unset !important; transform-origin: unset !important; transform-style: unset !important; transition: unset !important; translate: unset !important; vector-effect: unset !important; vertical-align: unset !important; view-timeline: unset !important; view-transition-name: unset !important; white-space: unset !important; widows: unset !important; will-change: unset !important; word-break: unset !important; x: unset !important; y: unset !important; z-index: unset !important; -apple-color-filter: unset !important; -apple-pay-button-style: unset !important; -apple-pay-button-type: unset !important; -internal-text-autosizing-status: unset !important; border-spacing: unset !important; -webkit-box-align: unset !important; -webkit-box-decoration-break: unset !important; -webkit-box-direction: unset !important; -webkit-box-flex: unset !important; -webkit-box-flex-group: unset !important; -webkit-box-lines: unset !important; -webkit-box-ordinal-group: unset !important; -webkit-box-orient: unset !important; -webkit-box-pack: unset !important; -webkit-box-reflect: unset !important; -webkit-column-axis: unset !important; -webkit-column-progression: unset !important; -webkit-cursor-visibility: unset !important; -webkit-font-size-delta: unset !important; -webkit-hyphenate-limit-after: unset !important; -webkit-hyphenate-limit-before: unset !important; -webkit-hyphenate-limit-lines: unset !important; -webkit-initial-letter: unset !important; -webkit-line-align: unset !important; -webkit-line-box-contain: unset !important; -webkit-line-clamp: unset !important; -webkit-line-grid: unset !important; -webkit-line-snap: unset !important; -webkit-marquee-direction: unset !important; -webkit-marquee-increment: unset !important; -webkit-marquee-repetition: unset !important; -webkit-marquee-speed: unset !important; -webkit-marquee-style: unset !important; -webkit-nbsp-mode: unset !important; -webkit-rtl-ordering: unset !important; -webkit-text-decorations-in-effect: unset !important; -webkit-text-fill-color: unset !important; -webkit-text-security: unset !important; -webkit-text-stroke-color: unset !important; -webkit-text-stroke-width: unset !important; -webkit-user-drag: unset !important; -webkit-user-modify: unset !important; -webkit-user-select: unset !important; word-spacing: unset !important; backdrop-filter: unset !important; box-shadow: unset !important; mask-composite: unset !important; mask-mode: unset !important; text-combine-upright: unset !important; -webkit-backdrop-filter: unset !important; -webkit-background-clip: unset !important; -webkit-background-origin: unset !important; -webkit-box-shadow: unset !important; -webkit-mask: unset !important; -webkit-mask-composite: unset !important; -webkit-mask-source-type: unset !important; -webkit-text-combine: unset !important; display: block !important; visibility: unset !important; opacity: unset !important; block-size: unset !important; border-block: unset !important; border-radius: unset !important; border-end-end-radius: unset !important; border-end-start-radius: unset !important; border-inline: unset !important; border-start-end-radius: unset !important; border-start-start-radius: unset !important; inset: unset !important; contain-intrinsic-block-size: unset !important; contain-intrinsic-size: unset !important; contain-intrinsic-inline-size: unset !important; height: unset !important; inline-size: unset !important; inset-block: unset !important; inset-inline: unset !important; margin-block: unset !important; margin-inline: unset !important; max-block-size: unset !important; max-height: unset !important; max-inline-size: unset !important; min-block-size: unset !important; min-height: unset !important; min-inline-size: unset !important; min-width: unset !important; overscroll-behavior-block: unset !important; overscroll-behavior-inline: unset !important; overscroll-behavior: unset !important; padding-block: unset !important; padding: unset !important; padding-inline: unset !important; scroll-margin-block: unset !important; scroll-margin: unset !important; scroll-margin-inline: unset !important; scroll-padding-block: unset !important; scroll-padding: unset !important; scroll-padding-inline: unset !important; width: unset !important; margin: 0px 0px 1px !important; max-width: 600px !important;"><div id="pfptBanneru80ly4f" style="color-scheme: unset !important; writing-mode: unset !important; -webkit-ruby-position: unset !important; font-family: Arial, sans-serif !important; font-feature-settings: unset !important; font-kerning: unset !important; font-optical-sizing: unset !important; font-palette: unset !important; font-size: 14px !important; font-size-adjust: unset !important; font-stretch: unset !important; font-style: unset !important; font-synthesis-small-caps: unset !important; font-synthesis-style: unset !important; font-synthesis-weight: unset !important; font-variant-alternates: unset !important; font-variant-caps: unset !important; font-variant-east-asian: unset !important; font-variant-ligatures: unset !important; font-variant-numeric: unset !important; font-variant-position: unset !important; font-variation-settings: unset !important; font-weight: bold !important; text-orientation: unset !important; text-rendering: unset !important; zoom: unset !important; -webkit-font-smoothing: unset !important; -webkit-locale: unset !important; -webkit-text-zoom: unset !important; letter-spacing: unset !important; accent-color: unset !important; align-content: unset !important; align-items: unset !important; align-self: unset !important; alignment-baseline: unset !important; alt: unset !important; animation-composition: unset !important; animation: unset !important; appearance: unset !important; aspect-ratio: unset !important; backface-visibility: unset !important; background-attachment: unset !important; background-blend-mode: unset !important; background-image: unset !important; background-position: unset !important; background-repeat: unset !important; background-size: unset !important; baseline-shift: unset !important; border-collapse: unset !important; border: unset !important; box-sizing: unset !important; break-after: unset !important; break-before: unset !important; break-inside: unset !important; buffered-rendering: unset !important; caption-side: unset !important; caret-color: unset !important; clear: unset !important; clip: unset !important; clip-path: unset !important; clip-rule: unset !important; color-interpolation: unset !important; color-interpolation-filters: unset !important; column-count: unset !important; column-fill: unset !important; column-gap: unset !important; column-rule-color: unset !important; column-rule-style: unset !important; column-rule-width: unset !important; column-span: unset !important; column-width: unset !important; contain: unset !important; container-name: unset !important; container-type: unset !important; content: unset !important; counter-increment: unset !important; counter-reset: unset !important; counter-set: unset !important; cursor: unset !important; cx: unset !important; cy: unset !important; dominant-baseline: unset !important; empty-cells: unset !important; fill: unset !important; fill-opacity: unset !important; fill-rule: unset !important; filter: unset !important; flex: unset !important; flex-flow: unset !important; float: unset !important; flood-color: unset !important; flood-opacity: unset !important; glyph-orientation-horizontal: unset !important; glyph-orientation-vertical: unset !important; grid: unset !important; grid-column-end: unset !important; grid-column-start: unset !important; grid-row-end: unset !important; grid-row-start: unset !important; hanging-punctuation: unset !important; hyphenate-character: unset !important; hyphens: unset !important; image-orientation: unset !important; image-rendering: unset !important; isolation: unset !important; justify-content: unset !important; justify-items: unset !important; justify-self: unset !important; kerning: unset !important; lighting-color: unset !important; line-break: unset !important; line-height: 18px !important; list-style: unset !important; margin-trim: unset !important; marker: unset !important; mask: unset !important; mask-type: unset !important; math-style: unset !important; mix-blend-mode: unset !important; object-fit: unset !important; object-position: unset !important; offset-anchor: unset !important; offset-distance: unset !important; offset-path: unset !important; offset-position: unset !important; offset-rotate: unset !important; order: unset !important; orphans: unset !important; outline: unset !important; outline-offset: unset !important; overflow-wrap: unset !important; overflow: unset !important; page: unset !important; paint-order: unset !important; perspective: unset !important; perspective-origin: unset !important; pointer-events: unset !important; position: unset !important; print-color-adjust: unset !important; quotes: unset !important; r: unset !important; resize: unset !important; rotate: unset !important; row-gap: unset !important; rx: unset !important; ry: unset !important; scale: unset !important; scroll-behavior: unset !important; scroll-snap-align: unset !important; scroll-snap-stop: unset !important; scroll-snap-type: unset !important; shape-image-threshold: unset !important; shape-margin: unset !important; shape-outside: unset !important; shape-rendering: unset !important; size: unset !important; speak-as: unset !important; stop-color: unset !important; stop-opacity: unset !important; stroke: unset !important; stroke-color: unset !important; stroke-dasharray: unset !important; stroke-dashoffset: unset !important; stroke-linecap: unset !important; stroke-linejoin: unset !important; stroke-miterlimit: unset !important; stroke-opacity: unset !important; stroke-width: unset !important; tab-size: unset !important; table-layout: unset !important; text-align: unset !important; text-align-last: unset !important; text-anchor: unset !important; text-decoration-color: unset !important; text-decoration: unset !important; text-decoration-skip-ink: unset !important; text-decoration-style: unset !important; text-decoration-thickness: unset !important; text-emphasis-color: unset !important; text-emphasis-position: unset !important; text-emphasis-style: unset !important; text-indent: unset !important; text-overflow: unset !important; text-shadow: unset !important; text-transform: unset !important; text-underline-offset: unset !important; text-underline-position: unset !important; text-wrap: unset !important; touch-action: unset !important; transform: unset !important; transform-box: unset !important; transform-origin: unset !important; transform-style: unset !important; transition: unset !important; translate: unset !important; vector-effect: unset !important; vertical-align: unset !important; white-space: unset !important; widows: unset !important; will-change: unset !important; word-break: unset !important; x: unset !important; y: unset !important; z-index: unset !important; -apple-color-filter: unset !important; -apple-pay-button-style: unset !important; -apple-pay-button-type: unset !important; border-spacing: unset !important; -webkit-box-align: unset !important; -webkit-box-decoration-break: unset !important; -webkit-box-direction: unset !important; -webkit-box-flex: unset !important; -webkit-box-flex-group: unset !important; -webkit-box-lines: unset !important; -webkit-box-ordinal-group: unset !important; -webkit-box-orient: unset !important; -webkit-box-pack: unset !important; -webkit-box-reflect: unset !important; -webkit-column-axis: unset !important; -webkit-column-progression: unset !important; -webkit-cursor-visibility: unset !important; -webkit-hyphenate-limit-after: unset !important; -webkit-hyphenate-limit-before: unset !important; -webkit-hyphenate-limit-lines: unset !important; -webkit-initial-letter: unset !important; -webkit-line-align: unset !important; -webkit-line-box-contain: unset !important; -webkit-line-clamp: unset !important; -webkit-line-grid: unset !important; -webkit-line-snap: unset !important; -webkit-nbsp-mode: unset !important; -webkit-rtl-ordering: unset !important; -webkit-text-decorations-in-effect: unset !important; -webkit-text-fill-color: unset !important; -webkit-text-security: unset !important; -webkit-text-stroke-color: unset !important; -webkit-text-stroke-width: unset !important; -webkit-user-drag: unset !important; -webkit-user-modify: unset !important; -webkit-user-select: unset !important; word-spacing: unset !important; background-clip: unset !important; background-origin: unset !important; box-shadow: unset !important; mask-composite: unset !important; mask-mode: unset !important; text-combine-upright: unset !important; -webkit-backdrop-filter: unset !important; -webkit-background-clip: unset !important; -webkit-background-origin: unset !important; -webkit-box-shadow: unset !important; mask-size: unset !important; -webkit-mask: unset !important; -webkit-mask-composite: unset !important; -webkit-mask-source-type: unset !important; -webkit-text-combine: unset !important; display: block !important; visibility: visible !important; opacity: unset !important; background-color: rgb(208, 216, 220) !important; block-size: unset !important; border-block: unset !important; border-radius: unset !important; border-end-end-radius: unset !important; border-end-start-radius: unset !important; border-inline: unset !important; border-start-end-radius: unset !important; border-start-start-radius: unset !important; inset: unset !important; contain-intrinsic-block-size: unset !important; contain-intrinsic-size: unset !important; contain-intrinsic-inline-size: unset !important; height: unset !important; inline-size: unset !important; inset-block: unset !important; inset-inline: unset !important; margin-block: unset !important; margin: unset !important; margin-inline: unset !important; max-block-size: unset !important; max-height: unset !important; max-inline-size: unset !important; max-width: unset !important; min-block-size: unset !important; min-height: unset !important; min-inline-size: unset !important; min-width: unset !important; overscroll-behavior-block: unset !important; overscroll-behavior-inline: unset !important; overscroll-behavior: unset !important; padding-block: unset !important; padding: unset !important; padding-inline: unset !important; scroll-margin-block: unset !important; scroll-margin: unset !important; scroll-margin-inline: unset !important; scroll-padding-block: unset !important; scroll-padding: unset !important; scroll-padding-inline: unset !important; width: unset !important;">This Message Is From an External Sender</div><div id="pfptBanneru80ly4f" style="color-scheme: unset !important; writing-mode: unset !important; -webkit-ruby-position: unset !important; font-family: Arial, sans-serif !important; font-feature-settings: unset !important; font-kerning: unset !important; font-optical-sizing: unset !important; font-palette: unset !important; font-size: 12px !important; font-size-adjust: unset !important; font-stretch: unset !important; font-style: unset !important; font-synthesis-small-caps: unset !important; font-synthesis-style: unset !important; font-synthesis-weight: unset !important; font-variant-alternates: unset !important; font-variant-caps: unset !important; font-variant-east-asian: unset !important; font-variant-ligatures: unset !important; font-variant-numeric: unset !important; font-variant-position: unset !important; font-variation-settings: unset !important; font-weight: unset !important; text-orientation: unset !important; text-rendering: unset !important; zoom: unset !important; -webkit-font-smoothing: unset !important; -webkit-locale: unset !important; -webkit-text-zoom: unset !important; letter-spacing: unset !important; accent-color: unset !important; align-content: unset !important; align-items: unset !important; align-self: unset !important; alignment-baseline: unset !important; alt: unset !important; animation-composition: unset !important; animation: unset !important; appearance: unset !important; aspect-ratio: unset !important; backface-visibility: unset !important; background-attachment: unset !important; background-blend-mode: unset !important; background-image: unset !important; background-position: unset !important; background-repeat: unset !important; background-size: unset !important; baseline-shift: unset !important; border-collapse: unset !important; border: unset !important; box-sizing: unset !important; break-after: unset !important; break-before: unset !important; break-inside: unset !important; buffered-rendering: unset !important; caption-side: unset !important; caret-color: unset !important; clear: unset !important; clip: unset !important; clip-path: unset !important; clip-rule: unset !important; color-interpolation: unset !important; color-interpolation-filters: unset !important; column-count: unset !important; column-fill: unset !important; column-gap: unset !important; column-rule-color: unset !important; column-rule-style: unset !important; column-rule-width: unset !important; column-span: unset !important; column-width: unset !important; contain: unset !important; container-name: unset !important; container-type: unset !important; content: unset !important; counter-increment: unset !important; counter-reset: unset !important; counter-set: unset !important; cursor: unset !important; cx: unset !important; cy: unset !important; dominant-baseline: unset !important; empty-cells: unset !important; fill: unset !important; fill-opacity: unset !important; fill-rule: unset !important; filter: unset !important; flex: unset !important; flex-flow: unset !important; float: unset !important; flood-color: unset !important; flood-opacity: unset !important; glyph-orientation-horizontal: unset !important; glyph-orientation-vertical: unset !important; grid: unset !important; grid-column-end: unset !important; grid-column-start: unset !important; grid-row-end: unset !important; grid-row-start: unset !important; hanging-punctuation: unset !important; hyphenate-character: unset !important; hyphens: unset !important; image-orientation: unset !important; image-rendering: unset !important; isolation: unset !important; justify-content: unset !important; justify-items: unset !important; justify-self: unset !important; kerning: unset !important; lighting-color: unset !important; line-break: unset !important; line-height: 18px !important; list-style: unset !important; margin-trim: unset !important; marker: unset !important; mask: unset !important; mask-type: unset !important; math-style: unset !important; mix-blend-mode: unset !important; object-fit: unset !important; object-position: unset !important; offset-anchor: unset !important; offset-distance: unset !important; offset-path: unset !important; offset-position: unset !important; offset-rotate: unset !important; order: unset !important; orphans: unset !important; outline: unset !important; outline-offset: unset !important; overflow-wrap: unset !important; overflow: unset !important; page: unset !important; paint-order: unset !important; perspective: unset !important; perspective-origin: unset !important; pointer-events: unset !important; position: unset !important; print-color-adjust: unset !important; quotes: unset !important; r: unset !important; resize: unset !important; rotate: unset !important; row-gap: unset !important; rx: unset !important; ry: unset !important; scale: unset !important; scroll-behavior: unset !important; scroll-snap-align: unset !important; scroll-snap-stop: unset !important; scroll-snap-type: unset !important; shape-image-threshold: unset !important; shape-margin: unset !important; shape-outside: unset !important; shape-rendering: unset !important; size: unset !important; speak-as: unset !important; stop-color: unset !important; stop-opacity: unset !important; stroke: unset !important; stroke-color: unset !important; stroke-dasharray: unset !important; stroke-dashoffset: unset !important; stroke-linecap: unset !important; stroke-linejoin: unset !important; stroke-miterlimit: unset !important; stroke-opacity: unset !important; stroke-width: unset !important; tab-size: unset !important; table-layout: unset !important; text-align: unset !important; text-align-last: unset !important; text-anchor: unset !important; text-decoration-color: unset !important; text-decoration: unset !important; text-decoration-skip-ink: unset !important; text-decoration-style: unset !important; text-decoration-thickness: unset !important; text-emphasis-color: unset !important; text-emphasis-position: unset !important; text-emphasis-style: unset !important; text-indent: unset !important; text-overflow: unset !important; text-shadow: unset !important; text-transform: unset !important; text-underline-offset: unset !important; text-underline-position: unset !important; text-wrap: unset !important; touch-action: unset !important; transform: unset !important; transform-box: unset !important; transform-origin: unset !important; transform-style: unset !important; transition: unset !important; translate: unset !important; vector-effect: unset !important; vertical-align: unset !important; white-space: unset !important; widows: unset !important; will-change: unset !important; word-break: unset !important; x: unset !important; y: unset !important; z-index: unset !important; -apple-color-filter: unset !important; -apple-pay-button-style: unset !important; -apple-pay-button-type: unset !important; border-spacing: unset !important; -webkit-box-align: unset !important; -webkit-box-decoration-break: unset !important; -webkit-box-direction: unset !important; -webkit-box-flex: unset !important; -webkit-box-flex-group: unset !important; -webkit-box-lines: unset !important; -webkit-box-ordinal-group: unset !important; -webkit-box-orient: unset !important; -webkit-box-pack: unset !important; -webkit-box-reflect: unset !important; -webkit-column-axis: unset !important; -webkit-column-progression: unset !important; -webkit-cursor-visibility: unset !important; -webkit-hyphenate-limit-after: unset !important; -webkit-hyphenate-limit-before: unset !important; -webkit-hyphenate-limit-lines: unset !important; -webkit-initial-letter: unset !important; -webkit-line-align: unset !important; -webkit-line-box-contain: unset !important; -webkit-line-clamp: unset !important; -webkit-line-grid: unset !important; -webkit-line-snap: unset !important; -webkit-nbsp-mode: unset !important; -webkit-rtl-ordering: unset !important; -webkit-text-decorations-in-effect: unset !important; -webkit-text-fill-color: unset !important; -webkit-text-security: unset !important; -webkit-text-stroke-color: unset !important; -webkit-text-stroke-width: unset !important; -webkit-user-drag: unset !important; -webkit-user-modify: unset !important; -webkit-user-select: unset !important; word-spacing: unset !important; background-clip: unset !important; background-origin: unset !important; box-shadow: unset !important; mask-composite: unset !important; mask-mode: unset !important; text-combine-upright: unset !important; -webkit-backdrop-filter: unset !important; -webkit-background-clip: unset !important; -webkit-background-origin: unset !important; -webkit-box-shadow: unset !important; mask-size: unset !important; -webkit-mask: unset !important; -webkit-mask-composite: unset !important; -webkit-mask-source-type: unset !important; -webkit-text-combine: unset !important; display: block !important; visibility: visible !important; opacity: unset !important; background-color: rgb(208, 216, 220) !important; block-size: unset !important; border-block: unset !important; border-radius: unset !important; border-end-end-radius: unset !important; border-end-start-radius: unset !important; border-inline: unset !important; border-start-end-radius: unset !important; border-start-start-radius: unset !important; inset: unset !important; contain-intrinsic-block-size: unset !important; contain-intrinsic-size: unset !important; contain-intrinsic-inline-size: unset !important; height: unset !important; inline-size: unset !important; inset-block: unset !important; inset-inline: unset !important; margin-block: unset !important; margin-bottom: unset !important; margin-inline: unset !important; margin-left: unset !important; margin-right: unset !important; max-block-size: unset !important; max-height: unset !important; max-inline-size: unset !important; max-width: unset !important; min-block-size: unset !important; min-height: unset !important; min-inline-size: unset !important; min-width: unset !important; overscroll-behavior-block: unset !important; overscroll-behavior-inline: unset !important; overscroll-behavior: unset !important; padding-block: unset !important; padding: unset !important; padding-inline: unset !important; scroll-margin-block: unset !important; scroll-margin: unset !important; scroll-margin-inline: unset !important; scroll-padding-block: unset !important; scroll-padding: unset !important; scroll-padding-inline: unset !important; width: unset !important; margin-top: 2px !important;">This message came from outside your organization.</div></div><div style="height: 0px; clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important;"></div></div><div style="margin: 0px 0in; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 15.693334px; font-family: Aptos; font-size: 11pt;">Hello everyone,<br><br>Thank you for your responses and feedback,<br><br>I checked PFLOTRAN and found that it is a model to simulate groundwater flow, contaminant transport, and other subsurface processes.<br>my goal is not to simulate the groundwater flow, my goal is to develop a code from scratch to simulate the groundwater flow with specific conditions, and then integrate this code with land surface models.<br>Later, the simulation of this code will be on a large scale.<br><br>I want PETSc to calculate the Jacobian because the system is large and has complex nonlinear behavior, and I don’t risk calculating the derivative by myself.</div><div style="margin: 0px 0in; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 15.693334px; font-family: Aptos; font-size: 11pt;">My A-Matrix has parts of source terms that depend on the flow fields, and independent parts will be in the RHS vector.<br></div></div></blockquote><div><br></div> With coloring SNESComputeJacobianDefaultColor() PETSc can compute Jacobian's pretty efficiently. You do not to provide the residual function and you need to provide the nonzero pattern of the sparse Jacobian; that is what residual components f_i are coupled to what input variables in the array x_i. This information comes from your PDE and discretization and appears implicitly in your residual function.</div><div><br></div><div> Barry</div><div><br><blockquote type="cite"><div><div style="margin: 0px 0in; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 15.693334px; font-family: Aptos; font-size: 11pt;"><br>I hope I have answered your questions, and I apologize that I wasn’t clear from the beginning, I was trying to keep my descriptions brief. <br><br></div><div style="margin: 0px 0in; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; line-height: 15.693334px; font-family: Aptos; font-size: 11pt;">Bests,<br>Sawsan</div><div id="appendonsend" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"></div><hr tabindex="-1" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline-block; width: 1120.125px;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"></span><div id="divRplyFwdMsg" dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>><br><b>Sent:</b><span class="Apple-converted-space"> </span>Tuesday, May 7, 2024 5:17 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu">sawsan.shatanawi@wsu.edu</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); background-color: rgb(255, 235, 156); padding: 1px;"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><div dir="ltr"><div dir="ltr">On Tue, May 7, 2024 at 2:23 PM Shatanawi, Sawsan Muhammad via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov">petsc-users@mcs.anl.gov</a>> wrote:<br></div><div class="x_gmail_quote"><blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="x_msg-4071540292083130403"><u></u><div dir="ltr" id="x_m_-4071540292083130403pfptBannery32hwk8" style="display: block; text-align: left; margin: 16px 0px; padding: 8px 16px; border-radius: 4px; min-width: 200px; background-color: rgb(208, 216, 220); border-top-width: 4px; border-top-style: solid; border-top-color: rgb(144, 164, 174);"><div id="x_m_-4071540292083130403pfptBannery32hwk8" style="float: left; display: block; margin: 0px 0px 1px; max-width: 600px;"><div id="x_m_-4071540292083130403pfptBannery32hwk8" style="display: block; background-color: rgb(208, 216, 220); font-family: Arial, sans-serif; font-weight: bold; font-size: 14px; line-height: 18px;">This Message Is From an External Sender<span class="Apple-converted-space"> </span></div><div id="x_m_-4071540292083130403pfptBannery32hwk8" style="font-weight: normal; display: block; background-color: rgb(208, 216, 220); font-family: Arial, sans-serif; font-size: 12px; line-height: 18px; margin-top: 2px;">This message came from outside your organization.<span class="Apple-converted-space"> </span></div></div><div style="height: 0px; clear: both; display: block; line-height: 0; font-size: 0.01px;"> </div></div><u></u><div dir="ltr"><div style="margin: 0in 0in 0.0001pt; line-height: normal;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(36, 36, 36); background-color: white;">Hello everyone,</span></div><p style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(36, 36, 36);"> </span></p><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(36, 36, 36);">I hope this email finds you well.</span></div><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(36, 36, 36);"><br><br></span></div><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(36, 36, 36);"> My Name is Sawsan Shatanawi,</span><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);"> and I was developing a Fortran code for simulating groundwater flow in a 3D system with nonlinear behavior. I solved the nonlinear system using the PCG solver and Picard iteration, but I did not get good results although I checked my matrix and RHS and everything, I decided to change my solver to Newton Rapson method.<br>I checked PETSc documents but I have a few questions:<br>1) My groundwater system is time-dependent, so should I use TS only instead of SNES?<br></span></div></div></div></blockquote><div><br></div><div>You could use TS, but it is not necessary. You could use SNES and your own timestepping. THe advantage of TS is that you can try many different timesteppers without recoding (just like you can</div><div>try many different linear and nonlinear solvers).</div><div> </div><blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="x_msg-4071540292083130403"><div dir="ltr"><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);">2) My system has its deltaT, would using deltaT as dt affect my solver, or is it better to use TS-PETSc dt? Also, would using PETSc dt affect the simulation of the groundwater system</span></div></div></div></blockquote><div><br></div><div>It sounds like your dt comes from your timestepper. If you use TS, you would use the dt from that.</div><div> </div><blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="x_msg-4071540292083130403"><div dir="ltr"><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);">3) I want my Jacobian matrix to be calculated by PETSc automatically</span></div></div></div></blockquote><div><br></div><div>PETSc can calculate a full Jacobian for smaller problems, or a finite-difference Jacobian for any problem (but this impacts the solver). It should be straightfoward to code up the analytic Jacobian. Is there a reason it would be a problem?</div><div> </div><blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="x_msg-4071540292083130403"><div dir="ltr"><p style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);"></span></p><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);">4) Do I need to define and calculate the residual vector?</span></div></div></div></blockquote><div><br></div><div>Yes.</div><div> </div><blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="x_msg-4071540292083130403"><div dir="ltr"><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);">My A-Matrix contains coefficients and external sources and my RHS vector includes the boundary conditions </span></div></div></div></blockquote><div><br></div><div>It is strange that your matrix would contain source terms. Do they depend on the flow fields?</div><div><br></div><div>Barry is right, you should consider PFlotran, and at least know why it would not work for your problem if you don't use it.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="x_gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="x_msg-4071540292083130403"><div dir="ltr"><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(36, 36, 36);"><br></span></div><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(36, 36, 36);">Please find the attached file contains a draft of my code</span><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);"><br><br></span></div><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);">Thank you in advance for your time and help.</span></div><div style="margin: 0in 0in 0.0001pt; line-height: normal; background-color: white;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);"><br></span></div><div style="background-color: white; padding: 0in; border-width: 1pt; border-style: solid; border-color: rgb(217, 217, 227);"><p style="margin: 15pt 0in; line-height: normal; background-color: white; padding: 0in; font-family: Calibri, sans-serif; font-size: 11pt;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);">Best regards,</span></p><p style="margin: 15pt 0in; line-height: normal; background-color: white; padding: 0in; font-family: Calibri, sans-serif; font-size: 11pt;"><span style="font-family: "Segoe UI", sans-serif; font-size: 11.5pt; color: rgb(55, 65, 81);"> Sawsan</span></p></div><div style="line-height: normal; background-color: white; margin: 0in 0in 0.0001pt; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><br></div><div id="x_m_-4071540292083130403appendonsend"></div><hr style="display: inline-block; width: 1102.5625px;"><div id="x_m_-4071540292083130403divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>><br><b>Sent:</b><span class="Apple-converted-space"> </span>Tuesday, January 16, 2024 10:43 AM<br><b>To:</b><span class="Apple-converted-space"> </span>Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>>; Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>>; Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>;<span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div dir="ltr"><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Hello all,<br><br>Thank you for your valuable help. I will do your recommendations and hope it will run without any issues.</div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><br></div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Bests,<br>Sawsan</div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"> </div><div id="x_m_-4071540292083130403x_appendonsend"></div><hr style="display: inline-block; width: 1102.5625px;"><div id="x_m_-4071540292083130403x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>><br><b>Sent:</b><span class="Apple-converted-space"> </span>Friday, January 12, 2024 8:46 AM<br><b>To:</b><span class="Apple-converted-space"> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>>; Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>>; Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>;<span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); background-color: rgb(255, 235, 156); padding: 1px;"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><div dir="ltr">Hi, Sawsan,<div> First in test_main.F90, you need to call VecGetArrayF90(temp_solution, H_vector, ierr) and VecRestoreArrayF90 (temp_solution, H_vector, ierr) as Barry mentioned.<br><div> Secondly, in the loop of test_main.F90, it calls GW_solver(). Within it, it calls PetscInitialize()/PetscFinalize(). But without MPI being initialized, PetscInitialize()/PetscFinalize()<b><span class="Apple-converted-space"> </span>can only be called once.</b></div><div><font face="monospace"> do timestep =2 , NTSP<br> call GW_boundary_conditions(timestep-1)<br> !print *,HNEW(1,1,1)<br> call GW_elevation()<br> ! print *, GWTOP(2,2,2)<br> call GW_conductance()<br> ! print *, CC(2,2,2)<br> call GW_recharge()<br> ! print *, B_Rech(5,4)<br> call GW_pumping(timestep-1)<br> ! print *, B_pump(2,2,2)<br> call GW_SW(timestep-1)<br> print *,B_RIVER (2,2,2)<br> call GW_solver(timestep-1,N)<br> call GW_deallocate_loop()<br> end do</font><br></div><div><br></div>A solution is to delete PetscInitialize()/PetscFinalize() in GW_solver_try.F90 and move it to test_main.F90, outside the do loop.<div><br></div><div><font face="monospace">diff --git a/test_main.F90 b/test_main.F90<br>index b5997c55..107bd3ee 100644<br>--- a/test_main.F90<br>+++ b/test_main.F90<br>@@ -1,5 +1,6 @@<br> program test_GW<br> <br>+#include <petsc/finclude/petsc.h><br> use petsc<br> use GW_constants<br> use GW_param_by_user<br>@@ -8,6 +9,9 @@ program test_GW<br> implicit none<br> integer :: N<br> integer :: timestep<br>+ PetscErrorCode ierr<br>+<br>+ call PetscInitialize(ierr)<br> call GW_domain(N)<br> !print *, "N=",N<br> !print *, DELTAT<br>@@ -37,4 +41,5 @@ program test_GW<br> end do<br> print *, HNEW(NCOL,3,2)<br> call GW_deallocate ()<br>+ call PetscFinalize(ierr)<br> end program test_GW</font><br></div><div><br></div><div>With that, the MPI error will be fixed. The code could run to gw_deallocate () before abort. There are other memory errors. You can install/use valgrind to fix them. Run it with valgrind ./GW.exe and look through the output</div><div><br></div><div><br></div><div>Thanks.</div><div>--Junchao Zhang<br></div><div><br></div></div></div><br><div><div dir="ltr">On Thu, Jan 11, 2024 at 10:49 PM Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>> wrote:<br></div><blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div dir="ltr"><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Hello,<br><br>Thank you all for your help.<br><br>I have changed VecGetArray to VecGetArrayF90, and the location of destory call. but I want to make sure that VecGet ArrayF90 is to make a new array( vector) that I can use in the rest of my Fortran code?<br><br></div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">when I run it and debugged it, I got </div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><br></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"> 5.2000000E-03</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"> 50.00000</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"> 10.00000</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"> 0.0000000E+00</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">PETSC: Attaching gdb to /weka/data/lab/richey/sawsan/GW_CODE/code2024/SS_GWM/./GW.exe of pid 33065 on display :0.0 on machine sn16</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Unable to start debugger in xterm: No such file or directory</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"> 0.0000000E+00</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgb(255, 255, 0);">Attempting to use an MPI routine after finalizing MPICH</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">srun: error: sn16: task 0: Exited with exit code 1</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">[sawsan.shatanawi@login-p2n02 SS_GWM]$ gdb ./GW/exe</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Copyright (C) 2013 Free Software Foundation, Inc.</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">License GPLv3+: GNU GPL version 3 or later <<a href="https://urldefense.com/v3/__http://gnu.org/licenses/gpl.html__;!!JmPEgBY0HMszNaDT!o4qS1zaFLg2L8PlawWJVyYsJpnwYHuL6SIIZsmzTO98RCzbP26HwTi_0-ipCS_D2SBG0X4gtEnM13-nbzFEFKvtraKaM$" originalsrc="https://urldefense.com/v3/__http://gnu.org/licenses/gpl.html__;!!JmPEgBY0HMszNaDT!o4qS1zaFLg2L8PlawWJVyYsJpnwYHuL6SIIZsmzTO98RCzbP26HwTi_0-ipCS_D2SBG0X4gtEnM13-nbzFEFKvtraKaM$" shash="dD9UMkMbISbCgb7EsF5VU/pWdxHuWXJTDZit9Slg3rR6lQikJK0uNaefF725RCjTRMnY4RVTDTKM2Pe+N3n4G9k/+it+MBN6i+lKY3x0SBtCj5Fup4YjApI4Ibtl0uYwQtMOJZw5Q3udX3rnx1x2oDNfY7PuU20zKVHIJcaf5wQ=" target="_blank">http://gnu.org/licenses/gpl.html</a>></span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">This is free software: you are free to change and redistribute it.</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">There is NO WARRANTY, to the extent permitted by law. Type "show copying"</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">and "show warranty" for details.</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">This GDB was configured as "x86_64-redhat-linux-gnu".</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">For bug reporting instructions, please see:</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><<a href="https://urldefense.com/v3/__http://www.gnu.org/software/gdb/bugs/__;!!JmPEgBY0HMszNaDT!o4qS1zaFLg2L8PlawWJVyYsJpnwYHuL6SIIZsmzTO98RCzbP26HwTi_0-ipCS_D2SBG0X4gtEnM13-nbzFEFKjO8jNfs$" originalsrc="https://urldefense.com/v3/__http://www.gnu.org/software/gdb/bugs/__;!!JmPEgBY0HMszNaDT!o4qS1zaFLg2L8PlawWJVyYsJpnwYHuL6SIIZsmzTO98RCzbP26HwTi_0-ipCS_D2SBG0X4gtEnM13-nbzFEFKjO8jNfs$" shash="KnSexQ0zvjaNqGYo92pH7/rx0HfvbhxEv4OHN5GBRZ+V7LJsabIhM6AH8IHM60Jr8E+ArxxERIEDpGkb/6B/8lSjyN27W1/xliyop2+ouRDx/3xdgoHihjpV7w1K6TL3kkIabqqn9llqlsNRCBKkQkUakS98w4LKaV7Zu5DM9rU=" target="_blank">http://www.gnu.org/software/gdb/bugs/</a>>...</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">./GW/exe: No such file or directory.</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">(gdb) run</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Starting program:</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">No executable file specified.</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Use the "file" or "exec-file" command.</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">(gdb) bt</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">No stack.</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">(gdb)<br><br>If the highlighted line is the error, I don't know why when I write gdb , it does not show me the location of error</span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">The code :<span class="Apple-converted-space"> </span><a href="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!o4qS1zaFLg2L8PlawWJVyYsJpnwYHuL6SIIZsmzTO98RCzbP26HwTi_0-ipCS_D2SBG0X4gtEnM13-nbzFEFKp8i33ur$" originalsrc="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!o4qS1zaFLg2L8PlawWJVyYsJpnwYHuL6SIIZsmzTO98RCzbP26HwTi_0-ipCS_D2SBG0X4gtEnM13-nbzFEFKp8i33ur$" shash="HvQgtqwy0UG3NBC2ZHHRn2pyoD5GQ6xgaCCqcKncnq00Ly5tbe7P2DZMPBTgRTieWcxmnOJXTR/gEnW+DzgQRakW/L3L4Jmh/F6BZclnnBwl2MCGIvXz/fyicD6CLQSGZSiRYLg4AEhLdSJr1crQJ7qOKwAIAlYlFOvI7wb1rB0=" id="x_m_-4071540292083130403x_x_m_6313238004422759188OWAf2a88c8d-1aa9-f2d9-963c-456c02baba7d" target="_blank">sshatanawi/SS_GWM (github.com)</a> <br><br>I really appreciate your helps<br><br></span></div><div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Sawsan</span></div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188appendonsend"></div><hr style="display: inline-block; width: 1085px;"><div id="x_m_-4071540292083130403x_x_m_6313238004422759188divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>><br><b>Sent:</b><span class="Apple-converted-space"> </span>Wednesday, January 10, 2024 5:35 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>>; Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>;<span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); background-color: rgb(255, 235, 156); padding: 1px;"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><br id="x_m_-4071540292083130403x_x_m_6313238004422759188x_lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On Jan 10, 2024, at 6:49 PM, Junchao Zhang <<a href="mailto:junchao.zhang@gmail.com" target="_blank">junchao.zhang@gmail.com</a>> wrote:</div><br><div><div dir="ltr">Hi, Sawsan,<div> I could build your code and I also could gdb it.</div><div><br></div><div>$ gdb ./GW.exe</div><div>...</div><div>$ Thread 1 "GW.exe" received signal SIGSEGV, Segmentation fault.</div>0x00007ffff1e6d44f in vecgetarray_ (x=0x7fffffffa718, fa=0x0, ia=0x7fffffffa75c, ierr=0x0) at /scratch/jczhang/petsc/src/vec/vec/interface/ftn-custom/zvectorf.c:257<br>257 *ierr = VecGetArray(*x, &lx);<br>(gdb) bt<br>#0 0x00007ffff1e6d44f in vecgetarray_ (x=0x7fffffffa718, fa=0x0, ia=0x7fffffffa75c, ierr=0x0) at /scratch/jczhang/petsc/src/vec/vec/interface/ftn-custom/zvectorf.c:257<br>#1 0x000000000040b6e3 in gw_solver (t_s=1.40129846e-45, n=300) at GW_solver_try.F90:169<br>#2 0x000000000040c6a8 in test_gw () at test_main.F90:35<div> </div><div><font color="#ff0000">ierr=0x0 <span class="Apple-converted-space"> </span></font><font>caused the segfault. See </font><a href="https://urldefense.com/v3/__https://petsc.org/release/manualpages/Vec/VecGetArray/*vecgetarray__;Iw!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ9Km12jA$" originalsrc="https://urldefense.com/v3/__https://petsc.org/release/manualpages/Vec/VecGetArray/*vecgetarray__;Iw!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ9Km12jA$" shash="GVwDaFfI8zuvcW8E/mfJTCHFsXpBSzTufGqeGRNP2Ecos2eU/YlfGhbtl5zgPsCHRhiFiCq/AHSiZQNtybXp8CtaaQ9UdqkNAzihQnxAkHp77lyb9PGZOSDxrLQ30pt9rfQPXNRDKg/vXoJXQCxJ1Ad+i/+mjiBLWMlYKx6Mn2k=" target="_blank">https://petsc.org/release/manualpages/Vec/VecGetArray/#vecgetarray</a>, you should use VecGetArrayF90 instead.<br></div><div><br></div><div>BTW, Barry, the code<span class="Apple-converted-space"> </span><a href="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM/blob/main/GW_solver_try.F90*L169__;Iw!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZh2eAi4o$" originalsrc="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM/blob/main/GW_solver_try.F90*L169__;Iw!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZh2eAi4o$" shash="LseuBTnsYf8JqXij894RA9RCkSYws6bhEWwpM/YdN7JnE483GQcC+IGyadCpZXDz0BFMpBbMLFYSPj/VvC1+bclLs67mlBD7oTNtBNFZNmPcpnFptsy4gzlA8aig6U73/HnFoxiVDkNDChYT+8R3YoZ2vMCPFduLPkOdPHwvJu4=" target="_blank">https://github.com/sshatanawi/SS_GWM/blob/main/GW_solver_try.F90#L169</a><span class="Apple-converted-space"> </span>has "call VecGetArray(temp_solution, H_vector, ierr)". I don't find petsc Fortran examples doing VecGetArray. Do we still support it?</div></div></div></blockquote><div><br></div> This is not the correct calling sequence for VecGetArray() from Fortran. </div><div><br></div><div> Regardless, definitely should not be writing any new code that uses VecGetArray() from Fortran. Should use VecGetArrayF90().</div><div><br><blockquote type="cite"><div><div dir="ltr"><div><br></div><div><div><div dir="ltr"><div dir="ltr">--Junchao Zhang</div></div></div><br></div></div><br><div><div dir="ltr">On Wed, Jan 10, 2024 at 2:38 PM Shatanawi, Sawsan Muhammad via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div dir="ltr"><div><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">Hello all,</span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"><br></span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">I hope you are doing well.</span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"><br></span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">Generally, I use gdb <the name of my exe.file> to debug the code.</span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"> I got the attached error message.</span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"><br></span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">I have tried to add the flag -start_in_debugger in the make file, but it didn't work, so it seems I was doing it in the wrong way</span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"><br></span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">This is the link for the whole code: <a href="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ1Veab3M$" originalsrc="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ1Veab3M$" shash="XUOLW89K8KuYcagpU3SVHrXnq/XBh+7Fpb0x5tLp+m8Gm/7Wl/OdrDtlbMiIwqZBKVPxitd3xV6b47n6LRdTCzTLeRu5TG6XyDRLsCcT4+e9wDLv/QeyIqfKl75SJEjF7/fu0UcS96VQnGNXNUrR0WUP+jx5lO6tj1TGGmDo1gU=" id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814OWA37645568-445c-55da-2f91-abc541b1f4d6" target="_blank" style="margin: 0px;">sshatanawi/SS_GWM (github.com)</a></span></div><div style="font-style: normal; font-weight: 400; font-size: medium; margin: 0px; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; word-spacing: 0px;"><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPBorder_GTaHR0cHM6Ly9naXRodWIuY29tL3NzaGF0YW5hd2kvU1NfR1dN" style="margin: 16px 0px; max-width: 800px; min-width: 424px;"><table id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPContainer867037" style="padding: 12px 36px 12px 12px; width: 600px; border-width: 1px; border-style: solid; border-color: rgb(200, 200, 200); border-radius: 2px;"><tbody><tr valign="top" style="border-spacing: 0px;"><td><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPImageContainer867037" style="margin: 0px 12px 0px 0px; height: 120px; overflow: hidden;"><a id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPImageAnchor867037" href="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ1Veab3M$" originalsrc="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ1Veab3M$" shash="o3OIa2jNqs7bmAkm+SI6q16pjTGH7pQy4mw2d/nCTboC2VOGdqusjyblSsuwY4JVK0YOadbwoS8H2njJldKjrnujvzit2BhYPLZHmqBX+biQezk9KQCCJVvHeeQZGXpv+y5/l2KhqvZf1gVSeOxPQB+1pDEqaaEAdlzh5CaETmI=" target="_blank" style="margin: 0px;"><img id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPThumbnailImageId867037" alt="" height="120" width="240" src="https://opengraph.githubassets.com/9eb6cd14baf12f04848ed209b6f502415eb531bdd7b3a5f9696af68663b870c0/sshatanawi/SS_GWM" style="margin: 0px; display: block;"></a></div></td><td style="width: 289.6px;"><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPTitle867037" style="font-weight: 300; font-size: 21px; font-family: wf_segoe-ui_light, "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin: 0px 8px 12px 0px;"><a id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPUrlAnchor867037" href="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ1Veab3M$" originalsrc="https://urldefense.com/v3/__https://github.com/sshatanawi/SS_GWM__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ1Veab3M$" shash="EUwalrUXD9n74JSZNhmDRIbt/m33Pf4ljWae5UwstjJ7N6s6nJkuCCcBaX2zz0DYuYozUV4VpsxnDgLRmZBu6Uh+qJr9H6cgHOAEzNQgTvs4bUXXG2km58s47XGz4wHtu43+h5DdxXfh1XGDFkZb7NTj6aagzgqRp8rHNT7bwFE=" target="_blank" style="margin: 0px; text-decoration: none;">GitHub - sshatanawi/SS_GWM</a></div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPDescription867037" style="font-size: 14px; font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin: 0px 8px 12px 0px; max-height: 100px; overflow: hidden; color: rgb(102, 102, 102);">Contribute to sshatanawi/SS_GWM development by creating an account on GitHub.</div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPMetadata867037" style="font-size: 14px; font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin: 0px; color: rgb(166, 166, 166);"><a href="https://urldefense.com/v3/__http://github.com/__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ8rcrPiA$" originalsrc="https://urldefense.com/v3/__http://github.com/__;!!JmPEgBY0HMszNaDT!tqBApprMfYxwNz4Zvnk8coNE5AeWjA9wSdAM7QJcIIVP1z0VDsVIalo4Sew2b0fW3bZtTAbPh-h0MUsZ8rcrPiA$" shash="a6tJtqtMqKXuqQilb4eSRJkwtpbdjtI/kTzOI33d3+gyUepZWZ44ki/O8t+eZgYKU4+KD3YNFUo07nr8ImH6U1DdT544g9sJnOZITD5QBZC3CrOq+9pPDCrtC1sBybUH5fjIFax0R05vW/Ka9tOeBssQ0nnOjeP0se6q0V+yRUA=" target="_blank">github.com</a></div></td></tr></tbody></table><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPCloseButtonContainer867037" title="Remove link preview" role="button"><i aria-hidden="true" id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814LPCloseButton867037"></i></div></div></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"><br></span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">You can read the description of the code in " </span><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400; color: rgb(12, 100, 192);">Model Desprciption.pdf</span><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">"</span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">the compiling file is </span><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400; color: rgb(12, 100, 192);">makefile_f90</span><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"> where you can find the linked code files</span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;"><br></span></div><div style="text-align: left; margin: 0px;"><span style="letter-spacing: normal; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; font-weight: 400;">I really appreciate your help<br><br>Bests,<br>Sawsan</span></div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814appendonsend"></div><hr style="display: inline-block; width: 1057.640625px;"><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>><br><b>Sent:</b><span class="Apple-converted-space"> </span>Friday, January 5, 2024 4:53 AM<br><b>To:</b><span class="Apple-converted-space"> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span>Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>>;<span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); background-color: rgb(255, 235, 156); padding: 1px;"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><div dir="ltr">This is a segv. As Matt said, you need to use a debugger for this or add print statements to narrow down the place where this happens.<div><br></div><div>You will need to learn how to use debuggers to do your project so you might as well start now.</div><div><br></div><div>If you have a machine with a GUI debugger that is easier but command line debuggers are good to learn anyway.</div><div><br></div><div>I tend to run debuggers directly (eg, lldb ./a.out -- program-args ...) and use a GUI debugger (eg, Totalview or DDT) if available.</div><div><br></div><div>Mark</div><div><br></div></div><br><div><div dir="ltr">On Wed, Dec 20, 2023 at 10:02 PM Shatanawi, Sawsan Muhammad via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div dir="ltr"><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Hello Matthew,<br><br>Thank you for your help. I am sorry that I keep coming back with my error messages, but I reached a point that I don't know how to fix them, and I don't understand them easily.<br>The list of errors is getting shorter, now I am getting the attached error messages </div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><br></div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Thank you again,<br><br></div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Sawsan</div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399appendonsend"></div><hr style="display: inline-block; width: 1040.078125px;"><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank">knepley@gmail.com</a>><br><b>Sent:</b><span class="Apple-converted-space"> </span>Wednesday, December 20, 2023 6:54 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>>;<span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); background-color: rgb(255, 235, 156); padding: 1px;"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><div dir="ltr"><div dir="ltr">On Wed, Dec 20, 2023 at 9:49 PM Shatanawi, Sawsan Muhammad via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br></div><div><blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div dir="ltr"><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif;">Hello Barry,<br><br>Thank you a lot for your help, Now I am getting the attached error message.</span></div></div></div></blockquote><div><br></div><div>Do not destroy the PC from KSPGetPC()</div><div><br></div><div> THanks,</div><div><br></div><div> Matt</div><div> </div><blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div dir="ltr"><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif;">Bests,<br>Sawsan</span></div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543appendonsend"></div><hr style="display: inline-block; width: 1022.515625px;"><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span class="Apple-converted-space"> </span>Barry Smith <<a href="mailto:bsmith@petsc.dev" target="_blank">bsmith@petsc.dev</a>><br><b>Sent:</b><span class="Apple-converted-space"> </span>Wednesday, December 20, 2023 6:32 PM<br><b>To:</b><span class="Apple-converted-space"> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>><br><b>Cc:</b><span class="Apple-converted-space"> </span>Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>>;<span class="Apple-converted-space"> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span class="Apple-converted-space"> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); background-color: rgb(255, 235, 156); padding: 1px;"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><div><br></div>Instead of <div><br></div><div><div> call PCCreate(PETSC_COMM_WORLD, pc, ierr)</div><div> call PCSetType(pc, PCILU,ierr) ! Choose a preconditioner type (ILU)</div><div> call KSPSetPC(ksp, pc,ierr) ! Associate the preconditioner with the KSP solver</div><div><br></div><div>do</div><div><br></div><div> call KSPGetPC(ksp,pc,ierr)</div><div> call PCSetType(pc, PCILU,ierr)</div><div><br></div><div>Do not call KSPSetUp(). It will be taken care of automatically during the solve</div><div><br></div><div><br></div><div><div><br><blockquote type="cite"><div>On Dec 20, 2023, at 8:52 PM, Shatanawi, Sawsan Muhammad via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:</div><br><div><div style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Hello, <br>I don't think that I set preallocation values when I created the matrix, would you please have look at my code. It is just the petsc related part from my code.<br>I was able to fix some of the error messages. Now I have a new set of error messages related to the KSP solver (attached)<br><br>I appreciate your help</div><div style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"><br></div><div style="font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">Sawsan</div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_appendonsend" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;"></div><hr style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; display: inline-block; width: 934.906px;"><span style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; float: none; display: inline;"></span><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_divRplyFwdMsg" dir="ltr" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;"><font face="Calibri, sans-serif" style="font-size: 11pt;"><b>From:</b><span> </span>Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>><br><b>Sent:</b><span> </span>Wednesday, December 20, 2023 6:44 AM<br><b>To:</b><span> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>><br><b>Cc:</b><span> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code</font><div> </div></div><div style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;"><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); background-color: rgb(255, 235, 156); padding: 1px;"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><div dir="ltr">Did you set preallocation values when you created the matrix?<div>Don't do that.</div></div><br><div><div dir="ltr">On Wed, Dec 20, 2023 at 9:36 AM Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>> wrote:<br></div><blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div dir="ltr"><div dir="ltr">Hello, </div><div dir="ltr"><br></div><div dir="ltr">I am trying to create a sparse matrix( which is as I believe a zero matrix) then adding some nonzero elements to it over a loop, then assembling it </div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_x_m_-677654408871201876ms-outlook-mobile-signature"><div><br></div>Get<span> </span><a href="https://urldefense.com/v3/__https://aka.ms/o0ukef__;!!JmPEgBY0HMszNaDT!uUJ_jeYf45gcXDGR_PeMjhU7hbd_fKcXJPn0pM9eb-YQihKNYuXMYM9x-hglsbXsCFIwNBWgHXdetHODupsOloE$" originalsrc="https://urldefense.com/v3/__https://aka.ms/o0ukef__;!!JmPEgBY0HMszNaDT!uUJ_jeYf45gcXDGR_PeMjhU7hbd_fKcXJPn0pM9eb-YQihKNYuXMYM9x-hglsbXsCFIwNBWgHXdetHODupsOloE$" shash="uMmZESpUbOlZpeINr1Zf8OgJU+pM+FlZE7xs4Jcr1HSzCimh4oEyUsNl+6LaNBhOqKd90VwHsu2pvHdNLUOVb44+d0G/iaMltDALYLQO6egvh7ip1Vg+n5B35mITyKS6zo5RNGwVNa8akHi7v7rGjvaqf1n9Ftv3MQC+PrwThsA=" target="_blank">Outlook for iOS</a></div><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_x_m_-677654408871201876mail-editor-reference-message-container"><hr style="display: inline-block; width: 917.344px;"><div id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_x_m_-677654408871201876divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif"><b>From:</b><span> </span>Mark Adams <<a href="mailto:mfadams@lbl.gov" target="_blank">mfadams@lbl.gov</a>><br><b>Sent:</b><span> </span>Wednesday, December 20, 2023 2:48 AM<br><b>To:</b><span> </span>Shatanawi, Sawsan Muhammad <<a href="mailto:sawsan.shatanawi@wsu.edu" target="_blank">sawsan.shatanawi@wsu.edu</a>><br><b>Cc:</b><span> </span><a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a><span> </span><<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>><br><b>Subject:</b><span> </span>Re: [petsc-users] Help with Integrating PETSc into Fortran Groundwater Flow Simulation Code<div> </div></font></div><div style="margin-top: 0px; margin-bottom: 0px; border: 1px solid rgb(255, 202, 202); padding: 1px; background-color: rgb(255, 235, 156);"><span style="font-size: 10pt;"><b>[EXTERNAL EMAIL]</b></span></div><div><div dir="ltr">I am guessing that you are creating a matrix, adding to it, finalizing it ("assembly"), and then adding to it again, which is fine, but you are adding new non-zeros to the sparsity pattern.<div>If this is what you want then you can tell the matrix to let you do that.</div><div>Otherwise you have a bug.</div><div><br></div><div>Mark</div></div><br><div><div dir="ltr">On Tue, Dec 19, 2023 at 9:50 PM Shatanawi, Sawsan Muhammad via petsc-users <<a href="mailto:petsc-users@mcs.anl.gov" target="_blank">petsc-users@mcs.anl.gov</a>> wrote:<br></div><blockquote style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><div dir="ltr"><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"></div>Hello everyone,<div><br></div><div>I hope this email finds you well.</div><div><span><br></span></div><div><span> My Name is Sawsan Shatanawi,</span><span><span style="display: inline; color: rgb(55, 65, 81);"> and I am currently working on developing a Fortran code for simulating groundwater flow in a 3D system. The code involves solving a nonlinear system, and I have created the matrix to be solved using the PCG solver and Picard iteration. However, </span></span><span>when I tried to assign it as a PETSc matrix I started getting a lot of error messages.</span></div><div><br></div><div>I am kindly asking if someone can help me,<span style="display: inline; color: rgb(55, 65, 81);"> I would be happy to share my code with him/her.</span></div><div><span><br></span></div><div><span>Please find the attached file contains a list of errors I have gotten</span></div><div><span style="color: rgb(55, 65, 81);"><br></span></div><div><span style="color: rgb(55, 65, 81);">Thank you in advance for your time and assistance.</span><br></div><div><span><p style="margin: 1.25em 0px; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; color: rgb(55, 65, 81);">Best regards,</p><p style="margin: 1.25em 0px; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; color: rgb(55, 65, 81);"> Sawsan</p><br></span></div><div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;"></div></div></div></blockquote></div></div></div></div></div></blockquote></div></div></div><span id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_cid:66357E60-6F0B-4E28-9883-5421A4F75577"><Matrix_RHS.F90></span><span id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_cid:FB95EDCC-3170-4E85-BBFB-5804756A9022"><out.txt></span><span id="x_m_-4071540292083130403x_x_m_6313238004422759188x_m_5694742774502394814x_m_1291163558337743399x_m_6596943672973964543x_cid:8AF6D377-8CA5-4FFF-81AC-12EFB1837766"><solver.F90></span></div></blockquote></div><br></div></div></div></div></div></div></blockquote></div><br clear="all"><div><br></div><span>--<span class="Apple-converted-space"> </span></span><br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.com/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!JmPEgBY0HMszNaDT!uskvAyF0pMMWDbMIexr9g4qN46V7Rea17GQdNIVG2vH_HMaX7mXgie4ZYgusmPpss_DS7H1_8vn8arGQNSkC$" originalsrc="https://urldefense.com/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!JmPEgBY0HMszNaDT!uskvAyF0pMMWDbMIexr9g4qN46V7Rea17GQdNIVG2vH_HMaX7mXgie4ZYgusmPpss_DS7H1_8vn8arGQNSkC$" shash="k8pOMQ+tzVRgJHZGcamUoso0iA9t/iVNv3tH7WSGJULzzQaRB+Iw5XTK0hfQ6j3ZAiYsGU/3xbVYgt632/FQ114dcs/MPhlqxDUTh7k+m053ZUUOYt+epWMTnZlgg8EX78Et40Z6Gh8dGorO5KXOsOmdZRCR8TJmWMvlkeHPvxA=" target="_blank">https://www.cse.buffalo.edu/~knepley/</a><br></div></div></div></div></div></div></div></div></div></div></div></div></blockquote></div></div></div></div></div></blockquote></div></div></blockquote></div><br></div></div></div></div></blockquote></div></div></div></div></div></div></blockquote></div><br clear="all"><div><br></div><span class="x_gmail_signature_prefix">--<span class="Apple-converted-space"> </span></span><br><div dir="ltr" class="x_gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener</div><div><br></div><div><a href="https://urldefense.com/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!JmPEgBY0HMszNaDT!uQ_CqaGbVrhAwdolHoSQABYOvUa_lcGbqSBo0R5vr6xf8cZbIdMji6gOsfHzJJg2Rl4pB9RLvcxAdNkAkI3m$" originalsrc="https://urldefense.com/v3/__http://www.cse.buffalo.edu/*knepley/__;fg!!JmPEgBY0HMszNaDT!uQ_CqaGbVrhAwdolHoSQABYOvUa_lcGbqSBo0R5vr6xf8cZbIdMji6gOsfHzJJg2Rl4pB9RLvcxAdNkAkI3m$" shash="fZjynGR7AEh9I6pdM2AQZ2mbuW0IEGctQTklRwj0MMiqxBaVGhIHy5/mZLZb87yM/TK17jm7w3+QZnlrbEQnWShCL7TGVAdIazzGfTijvWGz0wlBilHNZq79VVVeomf8ntOtHa99XT4xZ4C9M3GN08vM5C+88FYuuG2NOt6MI1Q=" target="_blank">https://www.cse.buffalo.edu/~knepley/</a></div></div></div></div></div></div></div></div></div></div></div></blockquote></div><br></body></html>