[petsc-users] Trouble with ISEmbed

Fri Oct 21 13:25:38 CDT 2022

> On 21 Oct 2022, at 10:46 AM, TARDIEU Nicolas <nicolas.tardieu at edf.fr> wrote:
> 
> Dear Pierre,
> 
> To complete my last post, in fact, the initial code (playing with the LGMap) was correct. It was my test case that was wrong.
> Once fixed according to your suggestion, everything turns out to be OK.
> I am nevertheless wondering if this IS embeding according to the global numbering should not be a native PTESc's feature ?

I’m not sure about what you are asking for here.
I believe the most efficient way to deal with your problem is to use a MatNest instead of a MatAIJ.
Then, there are several ways to extract submatrices of the global MatNest without having to deal with the embedding.

Thanks,
Pierre 

> Thank you again,
> Nicolas
> --
> Nicolas Tardieu
> Ing PhD Computational Mechanics
> EDF - R&D Dpt ERMES
> PARIS-SACLAY, FRANCE
> De : TARDIEU Nicolas <nicolas.tardieu at edf.fr>
> Envoyé : jeudi 20 octobre 2022 11:45
> À : pierre at joliv.et <pierre at joliv.et>
> Cc : petsc-users at mcs.anl.gov <petsc-users at mcs.anl.gov>
> Objet : RE: [petsc-users] Trouble with ISEmbed
>  
> Dear Pierre,
> 
> You fixed the problem! Thank you warmly for your precious help.
> 
> Best regards, 
> Nicolas
> --
> Nicolas Tardieu
> Ing PhD Computational Mechanics
> EDF - R&D Dpt ERMES
> PARIS-SACLAY, FRANCE
> De : pierre at joliv.et <pierre at joliv.et>
> Envoyé : mercredi 19 octobre 2022 22:22
> À : TARDIEU Nicolas <nicolas.tardieu at edf.fr>
> Cc : petsc-users at mcs.anl.gov <petsc-users at mcs.anl.gov>
> Objet : Re: [petsc-users] Trouble with ISEmbed
>  
> Sorry, I’m not very proficient in petsc4py, and there are a bunch of interfaces missing, e.g., ISShift(), so it may not be optimal, but I hope you’ll understand.
> First, you’ll need to regenerate the .bin by uncommenting the proper part of the code.
> That is because you were initially generating a 20x20 matrix, with 4 fields per unknown.
> That’s 5 unknowns, and so, with two processes, 10 rows per process is not consistent as 10/4 is not an integer — I don’t know how to force, in petsc4py, the local size to 12 on process #0 and 8 on process #1.
> The modified code generates a 16x16 matrices so it remains consistent.
> If you then run the first part of the program, you’ll get both B_uu and B_pp from B instead of A, with one, two, or four processes.
> Again, that should work for arbitrary number of processes, you just need to be careful that your local dimensions are consistent with the number of fields.
> 
> Thanks,
> Pierre
>  
> 
> 
>> On 19 Oct 2022, at 5:01 PM, Pierre Jolivet <pierre at joliv.et <mailto:pierre at joliv.et>> wrote:
>> 
>> 
>> 
>>> On 19 Oct 2022, at 4:32 PM, TARDIEU Nicolas <nicolas.tardieu at edf.fr <mailto:nicolas.tardieu at edf.fr>> wrote:
>>> 
>>> Dear Pierre,
>>> 
>>> Thank you very much for your answer. I have the same explanation as you for the code I sent. 
>>> But what I would like to do is the following : I have the full matrix A with fields u, p and t (which are interlaced in the real application). I want to extract B=A(u+p, u+p). *Then* I would like to extract the (u, u) block from B - let us call it B_uu.
>>> In fact, B_uu=A_uu but I really need to do the extraction from B.
>>> And I am missing something since I have to play with different numberings when switching the IS from A to B.
>>> 
>>> Is it clear enough ???? 
>> 
>> That’s cristal clear.
>> If the fields are interlaced, that’s actually easier to do, because you preserve the distribution, and there is less data movement.
>> I’ll try to fix your code in the case where the fields are interlaced if now one gives you another answer in the meantime.
>> 
>> Thanks,
>> Pierre
>> 
>>> Regards, 
>>> Nicolas
>>> --
>>> Nicolas Tardieu
>>> Ing PhD Computational Mechanics
>>> EDF - R&D Dpt ERMES
>>> PARIS-SACLAY, FRANCE
>>> De : pierre at joliv.et <mailto:pierre at joliv.et> <pierre at joliv.et <mailto:pierre at joliv.et>>
>>> Envoyé : mercredi 19 octobre 2022 14:51
>>> À : TARDIEU Nicolas <nicolas.tardieu at edf.fr <mailto:nicolas.tardieu at edf.fr>>
>>> Cc : petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov> <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>>
>>> Objet : Re: [petsc-users] Trouble with ISEmbed
>>>  
>>> On two processes, you have a different distribution for u and u+p.
>>> IS Object: 2 MPI processes
>>>   type: general
>>> [0] Number of indices in set 5
>>> [0] 0 0
>>> [0] 1 1
>>> [0] 2 2
>>> [0] 3 3
>>> [0] 4 4
>>> [1] Number of indices in set 5
>>> [1] 0 5
>>> [1] 1 6
>>> [1] 2 7
>>> [1] 3 8
>>> [1] 4 9
>>> IS Object: 2 MPI processes
>>>   type: general
>>> [0] Number of indices in set 8
>>> [0] 0 0
>>> [0] 1 1
>>> [0] 2 2
>>> [0] 3 3
>>> [0] 4 4
>>> [0] 5 5
>>> [0] 6 6
>>> [0] 7 7
>>> [1] Number of indices in set 7
>>> [1] 0 8
>>> [1] 1 9
>>> [1] 2 10
>>> [1] 3 11
>>> [1] 4 12
>>> [1] 5 13
>>> [1] 6 14
>>> ISEmbed() only works on local indices, so when you embed u into u+p, on the second process, you miss the row/column indices 5, 6, 7 of B = A(u+p, u+p).
>>> Thus, you end up with a matrix of dimension size(u) - 3 = 10 - 3 = 7, with just the row/column indices 8 and 9 being selected by the second process.
>>> What is it that you want to do exactly? Play with ISEmbed(), or get A(u, u) without using A but B instead?
>>> 
>>> Thanks,
>>> Pierre
>>> 
>>> > On 19 Oct 2022, at 12:00 PM, TARDIEU Nicolas via petsc-users <petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>>> > 
>>> > Dear PETSc Team,
>>> > 
>>> > I am trying to use IS embeding in parallel. 
>>> > In order to (try to) understand how it works, I have built a simple example, attached to this email.
>>> > 
>>> > I consider a 20X20 matrix. The dof (u, p, t) in global numbering are the following : 
>>> > u: 0..9     p: 10..14     t: 15..19
>>> > 
>>> > I have defined 4 IS to describe the dof  u, p, t and the agglomeration of u and p, called up.
>>> > I first extract the submatrix matrix(up,up), then  I would like to extract from it the (u,u) block.
>>> > 
>>> > The example runs OK in sequential but I do not obtain the (u,u) block on 2 processes.
>>> > 
>>> > I have a mistake in the build of the sub-IS but I cannot find it for days.
>>> > 
>>> > Best regards,
>>> > Nicolas
>>> > --
>>> > Nicolas Tardieu
>>> > Ing PhD Computational Mechanics
>>> > EDF - R&D Dpt ERMES
>>> > PARIS-SACLAY, FRANCE
>>> > 
>>> > Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
>>> > Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.
>>> > Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
>>> > ____________________________________________________
>>> > This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.
>>> > If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.
>>> > E-mail communication cannot be guaranteed to be timely secure, error or virus-free.
>>> > <is_pb.tgz>
>>> 
>>> 
>>> Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
>>> Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.
>>> Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
>>> ____________________________________________________
>>> This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.
>>> If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.
>>> E-mail communication cannot be guaranteed to be timely secure, error or virus-free.
> 
> 
> Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
> Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.
> Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
> ____________________________________________________
> This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.
> If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.
> E-mail communication cannot be guaranteed to be timely secure, error or virus-free.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20221021/eee1c817/attachment-0001.html>