[MPICH] Help With I/O

[Matthew Chambers]

The cout school of debugging has a lot to recommend it, but I don't see the
cout statements in your code and you haven't described in more detail how
you've used them to debug your for loop.  For example, how many iterations
does the loop go through before crashing?  Does it even get through one?
I'm still not sure what your code is supposed to do.  Are you trying to read
20 bytes each iteration?  I would think that Read_at overwrites the contents
of buf instead of appending onto the end of where the last read stopped.  In
that case, you were right to have the output happen right after the input
(however, it would probably be better to have a secondary buffer which buf
is appended onto after each read operation, and after all the reads, you can
write that secondary buffer out once.  It would be helpful if you would
describe what your file reading is supposed to do more clearly.

 

  _____  

From: owner-mpich-discuss at mcs.anl.gov
[mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of Erich Peterson
Sent: Wednesday, April 18, 2007 1:45 PM
To: mpich-discuss at mcs.anl.gov
Subject: RE: [MPICH] Help With I/O

 

Yeah you are totally right about the buf not being allocated, I think I have
allocated it correctly now. I also think I should take the write function
out of the for loop and place it at the end, and just let it write out what
is in the buffer totally. Below is my revised code. It is still giving me
the same errors though. Another problem is, I have no idea how to debug
using Linux (I'm a .Net Windows guy), which is what I have to use for this
project. I have like one week left of school, and I don't think I have
enough time to learn gdb. So, I'm just been placing cout statements
everywhere to see it I have correct file handles, etc...

#include "RecordRetrieval.h"
#include "mpi.h"
#include "time.h"
#include "iostream.h"

void RecordRetrieval::QueryData(vector<bool> unencoded_vector, char *
filename)
{
    int num_processes;
    int num_vector_elements;
    float num_elements_per_rank;
    int local_start_position;
    int local_end_position;
    char * buf;
    int my_rank;
    MPI::File input_file;
    MPI::Status input_file_status;
    MPI::File output_file;
    MPI::Status output_file_status;
    //MPI::Offset filesize;
    char output_filename[30];
    size_t i;
    struct tm tim;
    time_t now;
    now = time(NULL);
    tim = *(localtime(&now));
    i = strftime(output_filename, 30, "%m_%d_%Y_%H_%M_%S", &tim);

    /* Let the system do what it needs to start up MPI */
    MPI::Init();
    /* Get my process rank */
    my_rank = MPI::COMM_WORLD.Get_rank();
    /* Find out how many processes are being used */
    num_processes = MPI::COMM_WORLD.Get_size();
    num_vector_elements = unencoded_vector.size();

    num_elements_per_rank = num_vector_elements / num_processes;
    local_start_position = my_rank * (int)num_elements_per_rank;
    if(my_rank == num_processes - 1)
    {
        if(num_elements_per_rank * num_processes ==
(int)num_elements_per_rank * num_processes)
        {
            local_end_position = local_start_position +
((int)num_elements_per_rank - 1);
        }
        else
        {
            local_end_position = (local_start_position +
(int)num_elements_per_rank - 1) +
                (((int)num_elements_per_rank * num_processes) -
((int)num_elements_per_rank * num_processes));
        }
    }
    else
    {
        local_end_position = local_start_position +
((int)num_elements_per_rank - 1);
    }

   buf = (char *) malloc((local_end_position - local_start_position) + 1); 

    input_file = MPI::File::Open(MPI::COMM_WORLD, filename,
MPI::MODE_RDONLY,
                    MPI::INFO_NULL);

    output_file = MPI::File::Open(MPI::COMM_WORLD, output_filename,
MPI::MODE_CREATE | MPI::MODE_WRONLY, MPI::INFO_NULL);
    
    // filesize = input_file.Get_size();
    for(int i = local_start_position; i < local_end_position + 1; i++)
    {
        if(unencoded_vector[i])
        {
            input_file.Read_at(i * 20, buf, 20, MPI_CHAR,
input_file_status);
        }
    }
    
    output_file.Write_shared(buf, ((local_end_position -
local_start_position) + 1), MPI_CHAR, output_file_status);
   
    free(buf);

    input_file.Close();
    output_file.Close();
                                          
    MPI::Finalize();
}




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich-discuss/attachments/20070418/9f2824f9/attachment.htm>