[mpich-discuss] MPE tracing

[Georges Markomanolis]

Dear Anthony,

If I understand right the trick with the printf works when the output of 
the traces is the stdout. Moreover I have changed it also, I save one 
file per process and I am buffering the data in order to avoiding 
writing to the hard disk many times. The overhead is decreased enough 
with this way. I tried to test also the MPI_Pcontrol in my application 
but it didn't work because it should be implemented in the library. 
Maybe I will try it more if I have time. I don't know/think that it is 
easy to convert the output of the logging to my format. Of course with 
tracing libraries I can do it directly by changing the trace_mpi_core.c 
and it is really simple. Is there any way for the logging also?
For example in the tracing libraries instead of having: [1] Starting 
MPI_Send with count = 131220, dest = 0, tag = 1..
I save to the trace: p1 send to p0 131220*sizeof(datatype)

Moreover I am interested in the time for acquiring the traces and with 
this way the overheads seems to be less than 3%. Finally for the cases 
that the trace files are really big, I am trying to avoid having only 
one trace file like in logging. So if it is possible to have one logging 
file per process and moreover to be able to save it with my format 
before writing the normal logging files then it should be nice but maybe 
it needs a lot of time compared to the implementation of MPI_Pcontrol 
for tracing libraries.
The programs that I profile use only the MPI_COMM_WORLD but it would be 
nice to have the patch for any case.

Best regards,
Georges


chan at mcs.anl.gov wrote:
> The equivalent of customized logging for tracing is to add a 
> printf("Starting of block A...") before the selected block of codes
> and do another printf("Ending of block A...") after the block of codes.
> But that still produces a lot of output.  I think what should be done
> is to have MPI_Pcontrol() implemented in the tracing library, so that
> one can turn on or off tracing with MPI_Pcontrol.  I think the feature
> can be easily added to the trace_mpi_core.c ...  But it is not there yet.
>
> Given the tracing library does not have MPI_Pcontrol support and you may
> want to use logging library with MPI_Pcontrol instead.  BTW, are the
> MPI programs being profiled uses communicator besides MPI_COMM_WORLD ?
> If the answer is yes, your version of logging library may contain
> bugs in MPI_Pcontrol when used with MPI communicator functions.  Let
> me know, I will give you a patch...
>
> A.Chan
>
> ----- "Georges Markomanolis" <george at markomanolis.com> wrote:
>
>   
>> Thank you all for your answers,
>>
>> I used clog2print because I wanted to see only the traces, not the 
>> visualization as I need to convert the MPE traces into another format
>>
>> that I need. Moreover I changed the output of the tracing library in 
>> order to have the format that I mentioned and it seems to work very 
>> nice. One more question. Is it possible to have a selective 
>> instrumentation with the tracing library? I think something similar
>> with 
>> the customized logging but for tracing, I just need to know which MPI
>>
>> calls are executed by a block of code.
>>
>> Thanks a lot,
>> Best regards,
>> Georges
>>
>> chan at mcs.anl.gov wrote:
>>     
>>> ----- "Georges Markomanolis" <georges.markomanolis at ens-lyon.fr>
>>>       
>> wrote:
>>     
>>>   
>>>       
>>>> Dear all,
>>>>
>>>> I want to ask you about the MPE's libraries. While I use the
>>>>         
>> logging 
>>     
>>>> library for an application, I dump the clog file with the command 
>>>> clog2print (is the right way to see the content of the files?).
>>>>     
>>>>         
>>> No, you should use the command jumpshot to "view" clog2 file.
>>>
>>>   
>>>       
>>>> Although 
>>>> my application executes MPI_Bcast commands I can't see them. From
>>>>         
>> the
>>     
>>>> manual I read that nested calls are not taken into account, like 
>>>> MPI_Bcast. Is this why I can't see any MPI_Bcast? 
>>>>     
>>>>         
>>> No. nested call probably is irrelevant here.
>>>
>>>   
>>>       
>>>> Moreover when I link 
>>>> with the tracing library I can see the MPI_Bcast commands. So in
>>>>         
>> the 
>>     
>>>> case that I want to know about all the MPI commands I should use
>>>> tracing 
>>>> not logging? I don't need the timestamps and the visualization.
>>>>     
>>>>         
>>> If you have access to the source code and you don't care about the
>>>       
>> order
>>     
>>> of MPI calls being executed, you can simply use "grep" of string
>>>       
>> that
>>     
>>> is prefixed with MPI_, e.g.
>>>
>>> grep "MPI_[A-Za-z_]*(" *.c
>>>
>>> If you still need to do logging, you can send me how you
>>>       
>> linked/compiled
>>     
>>> to enable MPE logging (I suspect something wrong there).
>>>
>>> A.Chan
>>>   
>>>       
>>>> Thanks a lot,
>>>> Best regards,
>>>> Georges
>>>> _______________________________________________
>>>> mpich-discuss mailing list
>>>> mpich-discuss at mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>>     
>>>>         
>>> _______________________________________________
>>> mpich-discuss mailing list
>>> mpich-discuss at mcs.anl.gov
>>> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
>>>
>>>       
>
>