[mpich-discuss] Information about output of slog2print
Fernando Luz
fernando_luz at tpn.usp.br
Wed Feb 15 13:54:18 CST 2012
Hi Chan,
I started read the paper, thanks.
But I have another question about log files. I tried run a mpi execution
without insert mpe marks in source-code. It's works but the size of
*.clog2 file is very large and this implies in a large *.slog2 file.
Using clog2print, to look the ascii version of clog2 file, I noted in
file a lot of replicated information, such as
ts=0.001767 icomm=0 rank=0 thd=0 type=bare et=175
ts=0.001767 icomm=0 rank=0 thd=0 type=bare et=174
ts=0.001768 icomm=0 rank=0 thd=0 type=bare et=175
ts=0.001768 icomm=0 rank=0 thd=0 type=bare et=174
ts=0.001768 icomm=0 rank=0 thd=0 type=bare et=175
ts=0.001768 icomm=0 rank=0 thd=0 type=bare et=174
ts=0.001768 icomm=0 rank=0 thd=0 type=bare et=175
ts=0.001769 icomm=0 rank=0 thd=0 type=bare et=174
ts=0.001769 icomm=0 rank=0 thd=0 type=bare et=175
I think these information are unnecessary to save.
Using grep, I threw away every row with "bare" text
$ clog2print out.clog2 > out.clog2_ascii
$ grep -R --invert-match "bare" out.clog2_ascii >
without_bare.clog2_ascii
The sizes of files are:
out.clog2_ascii - 586 Mb
without_bare.clog2_ascii - 5.7 Kb
Have some form of avoid print unnecessary information in clog2 binary
file?
Regards
Fernando Luz
On Qua, 2012-02-15 at 12:02 -0600, Anthony Chan wrote:
> Reading the slog2 paper below
>
> "An Efficient Format for Nearly Constant-Time Access
> to Arbitrary Time Intervals in Large Trace Files"
>
> in http://www.mcs.anl.gov/research/projects/perfvis/publications/index.htm
> should answer your questions. If not, ask again.
> Also, not all fields in slog2 format is used in jumpshot.
>
> A.Chan
>
> ----- Original Message -----
> > Hi,
> >
> > I used MPE in my MPI application and I converted the output (clog2
> > file
> > to slog2 file) and after this I converted again to ascii file using
> > slog2print.
> >
> > I want to know if have a documentation which describes the output of
> > slog2print.
> >
> > I don't understand the meaning of:
> >
> > Some stuffs in Header, like as:
> > - TreeLeafByteSize;
> > - MaxTreeDepth;
> > - MaxBufferByteSize;
> > - FBinfo(<integer> @ <integer>)
> >
> > The categorymap is used to define the elements of MPI (send, recv,
> > etc..) used in MPI execution. Some info inside of brackets I don't
> > know
> > the meaning, like as:
> > - isUsed;
> > - width;
> > - info_fmt;
> > - vis;
> > - search;
> > - ratios;
> > - count;
> >
> > And, at least, Slog tree directory, make use of categorymap to provide
> > the graph of execution.
> > - How the tree elements makes the graph?
> >
> > I was ask about this because I'm planned to make a bash script to
> > obtain
> > the informations in a mpi execution.
> >
> > This is my output logfile of an execution.
> >
> > -----------------------------
> > GUI_LIBDIR is set. GUI_LIBDIR =
> > /home/fernando_luz/usr/local/mpich2/lib
> >
> > SLOG-2 Header:
> > version = SLOG 2.0.6
> > NumOfChildrenPerNode = 2
> > TreeLeafByteSize = 65536
> > MaxTreeDepth = 12
> > MaxBufferByteSize = 210702
> > Categories is FBinfo(978 @ 203746012)
> > MethodDefs is FBinfo(0 @ 0)
> > LineIDMaps is FBinfo(232 @ 203746990)
> > TreeRoot is FBinfo(2428 @ 203743584)
> > TreeDir is FBinfo(210702 @ 203747222)
> > Annotations is FBinfo(0 @ 0)
> > Postamble is FBinfo(0 @ 0)
> >
> > CategoryMap:
> > 1: Category[ index=0, name=message, topo=Arrow,
> > color=(255,255,255,255,true), isUsed=false, width=3, info_fmt=<
> > msg_tag=
> > %d, msg_size=%d >, vis=true, search=true, ratios=5.938042E-5,0.0,
> > count=12 ]
> > 2: Category[ index=71, name=MPI_Irecv, topo=State,
> > color=(152,251,152,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=4.6330633E-6,4.6330633E-6, count=6 ]
> > 3: Category[ index=76, name=MPI_Pack_size, topo=State,
> > color=(255,255,255,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=4.7874983E-6,4.7874983E-6, count=76 ]
> > 4: Category[ index=78, name=MPI_Recv, topo=State,
> > color=(0,255,0,255,true), isUsed=false, width=1, vis=true,
> > search=true,
> > ratios=1.2596459,1.2596459, count=6 ]
> > 5: Category[ index=281, name=CLOG_Buffer_write2disk, topo=State,
> > color=(176,48,96,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=0.13866395,0.13866395, count=46 ]
> > 6: Category[ index=73, name=MPI_Isend, topo=State,
> > color=(135,206,235,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=4.941934E-6,4.941934E-6, count=6 ]
> > 7: Category[ index=105, name=MPI_Unpack, topo=State,
> > color=(255,255,255,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=6.872377E-6,6.872377E-6, count=76 ]
> > 8: Category[ index=75, name=MPI_Pack, topo=State,
> > color=(255,255,255,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=5.8685464E-6,5.8685464E-6, count=76 ]
> > 9: Category[ index=252, name=MPE_Irecv_waited, topo=State,
> > color=(255,140,0,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=6.1774176E-7,6.1774176E-7, count=6 ]
> > 10: Category[ index=184, name=MPI_Free_mem, topo=State,
> > color=(160,32,240,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=5.7141115E-6,5.7141115E-6, count=60 ]
> > 11: Category[ index=81, name=MPI_Send, topo=State,
> > color=(0,0,255,255,true), isUsed=false, width=1, vis=true,
> > search=true,
> > ratios=6.254635E-6,6.254635E-6, count=6 ]
> > 12: Category[ index=25, name=MPI_Comm_rank, topo=State,
> > color=(255,255,255,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=2.779838E-6,2.779838E-6, count=17 ]
> > 13: Category[ index=88, name=MPI_Test, topo=State,
> > color=(255,165,0,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=0.56697184,0.50070196, count=6141049 ]
> > 14: Category[ index=183, name=MPI_Alloc_mem, topo=State,
> > color=(160,32,240,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=9.574997E-6,9.574997E-6, count=60 ]
> > 15: Category[ index=28, name=MPI_Comm_size, topo=State,
> > color=(255,255,255,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=2.0076607E-6,2.0076607E-6, count=20 ]
> > 16: Category[ index=-3, name=Preview_State, topo=State,
> > color=(255,255,255,191,true), isUsed=false, width=5, vis=true,
> > search=true, ratios=0.0,0.0, count=0 ]
> > 17: Category[ index=-2, name=Preview_Arrow, topo=Arrow,
> > color=(255,255,0,255,true), isUsed=false, width=5, vis=true,
> > search=true, ratios=0.0,0.0, count=0 ]
> > 18: Category[ index=302, name=MPE_Comm_finalize, topo=Event,
> > color=(255,165,0,255,true), isUsed=false, width=1, vis=true,
> > search=true, ratios=NaN,0.0, count=4 ]
> > 19: Category[ index=-1, name=Preview_Event, topo=Event,
> > color=(255,255,255,191,true), isUsed=false, width=5, vis=true,
> > search=true, ratios=0.0,0.0, count=0 ]
> >
> > SLOG-2 Tree Directory
> > ID(12,0) -> DirVal[ TimeBBox(7.040500640869141E-4,3.0883190631866455),
> > FBinfo(2428 @ 203743584) ]
> > ID(11,0) -> DirVal[ TimeBBox(7.040500640869141E-4,2.0137720108032227),
> > FBinfo(1290 @ 134789844) ]
> > ID(11,1) -> DirVal[ TimeBBox(2.0137720108032227,3.0883190631866455),
> > FBinfo(1542 @ 203742042) ]
> > ID(10,0) -> DirVal[ TimeBBox(7.040500640869141E-4,1.0034620761871338),
> > FBinfo(1050 @ 67395756) ]
> > ID(10,1) -> DirVal[ TimeBBox(1.0034620761871338,2.0137720108032227),
> > FBinfo(670 @ 134789174) ]
> > ID(10,2) -> DirVal[ TimeBBox(2.0137720108032227,3.016982078552246),
> > FBinfo(330 @ 202180286) ]
> > ID(10,3) -> DirVal[ TimeBBox(3.016982078552246,3.0883190631866455),
> > FBinfo(1400 @ 203740642) ]
> > ID(9,0) -> DirVal[ TimeBBox(7.040500640869141E-4,0.4991741180419922),
> > FBinfo(710 @ 33697324) ]
> > ID(9,1) -> DirVal[ TimeBBox(0.4991741180419922,1.0034620761871338),
> > FBinfo(670 @ 67395086) ]
> > ID(9,2) -> DirVal[ TimeBBox(1.0034620761871338,1.5079309940338135),
> > FBinfo(670 @ 101093858) ]
> > ID(9,3) -> DirVal[ TimeBBox(1.5079309940338135,2.0137720108032227),
> > FBinfo(330 @ 134788844) ]
> > ID(9,4) -> DirVal[ TimeBBox(2.0137720108032227,2.5104100704193115),
> > FBinfo(330 @ 168485310) ]
> > ID(9,5) -> DirVal[ TimeBBox(2.5104100704193115,3.016982078552246),
> > FBinfo(330 @ 202179956) ]
> > ID(9,6) -> DirVal[ TimeBBox(3.016982078552246,3.0883190631866455),
> > FBinfo(1400 @ 203739242) ]
> >
> >
> > Best Regards
> >
> > Fernando Luz
> >
> > _______________________________________________
> > mpich-discuss mailing list mpich-discuss at mcs.anl.gov
> > To manage subscription options or unsubscribe:
> > https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
> _______________________________________________
> mpich-discuss mailing list mpich-discuss at mcs.anl.gov
> To manage subscription options or unsubscribe:
> https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20120215/23f12270/attachment.htm>
More information about the mpich-discuss
mailing list