itaps-parallel reasons against part=mesh instance

[Mark Beall]

I don't see how having part=mesh instance and having more than one  
part per process implies multiple threads. Given the current  
interface, that would seem to be true since the mesh instance is  
passed in as the "context" of all the collective calls since it  
basically means "all of the mesh on this process", it would just be a  
matter of having something else that means the same thing. Along the  
same lines, since an iMeshP_PartitionHandle is associated with a  
single iMesh_Instance why is it even necessary to pass both into those  
functions, the mesh instance is redundant since there is only one that  
can be the correct one, right?

mark


On Apr 21, 2010, at 12:03 PM, Tim Tautges wrote:

> Some of the following argument depends on how partitions are  
> handled.  However, assuming the partition is also associated with a  
> mesh instance (with the partition in each mesh instance coordinated  
> with those on other instances over the parallel job)... using one  
> part per instance, and multiple instances on a given process,  
> implies multiple threads of control, since many of the iMeshP  
> functions are collective calls.  Single threads of control are far  
> and away the most common mode for running codes at scale right now,  
> and I assert will continue to be for some time (5-10 yrs).
>
> Also, for the mode where an application is providing an iMeshP  
> implementation on top of its data structure so it can use services  
> implemented on iMeshP, I think the restriction of one part per  
> instance means that these apps will always restrict themselves to  
> one part per process.  I think your application is of this type.   
> So, I'd much rather have this be a restriction of your application  
> than a behavior bound at the interface level.  In fact, the latter  
> almost guarantees that MOAB will only support one part per process,  
> since that will cover 99% of the use cases.
>
> I think this goes back again to the runtime notion of a part being  
> confused with the unit of partitioning that's stored with /  
> associated to the mesh.  Maybe that's just a narrow view, though,  
> since I'm pretty sure Mark S. disagrees with that.
>
> - tim
>
> Mark Beall wrote:
>> All,
>> I was thinking over the call we had on Monday, specifically about  
>> what arguments were made against part=mesh instance. The only  
>> really compelling argument I recall (and sorry if I don't remember  
>> others, that's why I'm writing this email) was Tim's example of the  
>> overhead in partitioning a mesh into 100,000 partitions with 8  
>> elements each.
>> Well, it kind of struck me that Tim's example, while relevant in  
>> terms of the percentage of overhead isn't really that relevant in  
>> terms of total memory. The initial mesh there would be 800,000  
>> elements, maybe a few hundred MB. Even with much more than 100%  
>> overhead, I could easily do that on my laptop. Given that I can buy  
>> a computer with 96 GB of memory today for about $8000 (192 GB for  
>> $17000) ( a Dell 7500 with 3rd party memory in case you're  
>> curious), you could add a couple zeros to the number of partitions  
>> for that mesh before it should become an issue for someone that  
>> will be running that simulation on a supercomputer costing a few  
>> hundred million dollars.
>> What were the other compelling arguments against part=mesh instance?
>> mark
>
> -- 
> ================================================================
> "You will keep in perfect peace him whose mind is
>  steadfast, because he trusts in you."               Isaiah 26:3
>
>             Tim Tautges            Argonne National Laboratory
>         (tautges at mcs.anl.gov)      (telecommuting from UW-Madison)
>         phone: (608) 263-8485      1500 Engineering Dr.
>           fax: (608) 263-4499      Madison, WI 53706
>