[MPICH] Help With I/O

Wed Apr 18 16:28:19 CDT 2007

I have taken out the cout statements, and I have not tried very hard to debug the for loop, b/c I can't make any assumptions about the relative time it takes to execute an instruction. For instance, if I put a cout in the for loop I get half of a processes output and then half of another's after that. 

Here is hopefully a butter description of what I am wanting to do:

Say in my test.dat file I have the following:
111111111111111111112222222222222222222233333333333333333333444444444444444444445555555555555555555566666666666666666666

Then I set my vector<bool> to have the following items (starting with the first element [0]): true, false, true, false, true, false. Say I run the program using 3 processes, I will have each process take 2 elements of the vector. Then each process will look and see if element[i] is true, if so seek in the test.dat file to position 20 * i and grab 20 characters, lastly, output those 20 characters to the output file. 

Hope that helps in the understanding of the problem.

Regards,
Erich

From: matthew.chambers at vanderbilt.eduTo: erichpeterson at hotmail.com; mpich-discuss at mcs.anl.govSubject: RE: [MPICH] Help With I/ODate: Wed, 18 Apr 2007 14:36:43 -0500

The cout school of debugging has a lot to recommend it, but I don’t see the cout statements in your code and you haven’t described in more detail how you’ve used them to debug your for loop.  For example, how many iterations does the loop go through before crashing?  Does it even get through one?  I’m still not sure what your code is supposed to do.  Are you trying to read 20 bytes each iteration?  I would think that Read_at overwrites the contents of buf instead of appending onto the end of where the last read stopped.  In that case, you were right to have the output happen right after the input (however, it would probably be better to have a secondary buffer which buf is appended onto after each read operation, and after all the reads, you can write that secondary buffer out once.  It would be helpful if you would describe what your file reading is supposed to do more clearly.

From: owner-mpich-discuss at mcs.anl.gov [mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of Erich PetersonSent: Wednesday, April 18, 2007 1:45 PMTo: mpich-discuss at mcs.anl.govSubject: RE: [MPICH] Help With I/O

Yeah you are totally right about the buf not being allocated, I think I have allocated it correctly now. I also think I should take the write function out of the for loop and place it at the end, and just let it write out what is in the buffer totally. Below is my revised code. It is still giving me the same errors though. Another problem is, I have no idea how to debug using Linux (I'm a .Net Windows guy), which is what I have to use for this project. I have like one week left of school, and I don't think I have enough time to learn gdb. So, I'm just been placing cout statements everywhere to see it I have correct file handles, etc...#include "RecordRetrieval.h"#include "mpi.h"#include "time.h"#include "iostream.h"void RecordRetrieval::QueryData(vector<bool> unencoded_vector, char * filename){    int num_processes;    int num_vector_elements;    float num_elements_per_rank;    int local_start_position;    int local_end_position;    char * buf;    int my_rank;    MPI::File input_file;    MPI::Status input_file_status;    MPI::File output_file;    MPI::Status output_file_status;    //MPI::Offset filesize;    char output_filename[30];    size_t i;    struct tm tim;    time_t now;    now = time(NULL);    tim = *(localtime(&now));    i = strftime(output_filename, 30, "%m_%d_%Y_%H_%M_%S", &tim);    /* Let the system do what it needs to start up MPI */    MPI::Init();    /* Get my process rank */    my_rank = MPI::COMM_WORLD.Get_rank();    /* Find out how many processes are being used */    num_processes = MPI::COMM_WORLD.Get_size();    num_vector_elements = unencoded_vector.size();    num_elements_per_rank = num_vector_elements / num_processes;    local_start_position = my_rank * (int)num_elements_per_rank;    if(my_rank == num_processes - 1)    {        if(num_elements_per_rank * num_processes == (int)num_elements_per_rank * num_processes)        {            local_end_position = local_start_position + ((int)num_elements_per_rank - 1);        }        else        {            local_end_position = (local_start_position + (int)num_elements_per_rank - 1) +                (((int)num_elements_per_rank * num_processes) - ((int)num_elements_per_rank * num_processes));        }    }    else    {        local_end_position = local_start_position + ((int)num_elements_per_rank - 1);    }   buf = (char *) malloc((local_end_position - local_start_position) + 1);     input_file = MPI::File::Open(MPI::COMM_WORLD, filename, MPI::MODE_RDONLY,                    MPI::INFO_NULL);    output_file = MPI::File::Open(MPI::COMM_WORLD, output_filename, MPI::MODE_CREATE | MPI::MODE_WRONLY, MPI::INFO_NULL);        // filesize = input_file.Get_size();    for(int i = local_start_position; i < local_end_position + 1; i++)    {        if(unencoded_vector[i])        {            input_file.Read_at(i * 20, buf, 20, MPI_CHAR, input_file_status);        }    }        output_file.Write_shared(buf, ((local_end_position - local_start_position) + 1), MPI_CHAR, output_file_status);       free(buf);    input_file.Close();    output_file.Close();                                              MPI::Finalize();}
_________________________________________________________________
Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy!
http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20070418/c64eb68f/attachment.htm>