<html>
<head>
<style>
P
{
margin:0px;
padding:0px
}
body
{
FONT-SIZE: 10pt;
FONT-FAMILY:Tahoma
}
</style>
</head>
<body>Yeah you are totally right about the buf not being allocated, I think I have allocated it correctly now. I also think I should take the write function out of the for loop and place it at the end, and just let it write out what is in the buffer totally. Below is my revised code. It is still giving me the same errors though. Another problem is, I have no idea how to debug using Linux (I'm a .Net Windows guy), which is what I have to use for this project. I have like one week left of school, and I don't think I have enough time to learn gdb. So, I'm just been placing cout statements everywhere to see it I have correct file handles, etc...<br><br>#include "RecordRetrieval.h"<br>#include "mpi.h"<br>#include "time.h"<br>#include "iostream.h"<br><br>void RecordRetrieval::QueryData(vector<bool> unencoded_vector, char * filename)<br>{<br> int num_processes;<br> int num_vector_elements;<br> float num_elements_per_rank;<br> int local_start_position;<br> int local_end_position;<br> char * buf;<br> int my_rank;<br> MPI::File input_file;<br> MPI::Status input_file_status;<br> MPI::File output_file;<br> MPI::Status output_file_status;<br> //MPI::Offset filesize;<br> char output_filename[30];<br> size_t i;<br> struct tm tim;<br> time_t now;<br> now = time(NULL);<br> tim = *(localtime(&now));<br> i = strftime(output_filename, 30, "%m_%d_%Y_%H_%M_%S", &tim);<br><br> /* Let the system do what it needs to start up MPI */<br> MPI::Init();<br> /* Get my process rank */<br> my_rank = MPI::COMM_WORLD.Get_rank();<br> /* Find out how many processes are being used */<br> num_processes = MPI::COMM_WORLD.Get_size();<br> num_vector_elements = unencoded_vector.size();<br><br> num_elements_per_rank = num_vector_elements / num_processes;<br> local_start_position = my_rank * (int)num_elements_per_rank;<br> if(my_rank == num_processes - 1)<br> {<br> if(num_elements_per_rank * num_processes == (int)num_elements_per_rank * num_processes)<br> {<br> local_end_position = local_start_position + ((int)num_elements_per_rank - 1);<br> }<br> else<br> {<br> local_end_position = (local_start_position + (int)num_elements_per_rank - 1) +<br> (((int)num_elements_per_rank * num_processes) - ((int)num_elements_per_rank * num_processes));<br> }<br> }<br> else<br> {<br> local_end_position = local_start_position + ((int)num_elements_per_rank - 1);<br> }<br><br> buf = (char *) malloc((local_end_position - local_start_position) + 1); <br><br> input_file = MPI::File::Open(MPI::COMM_WORLD, filename, MPI::MODE_RDONLY,<br> MPI::INFO_NULL);<br><br> output_file = MPI::File::Open(MPI::COMM_WORLD, output_filename, MPI::MODE_CREATE | MPI::MODE_WRONLY, MPI::INFO_NULL);<br> <br> // filesize = input_file.Get_size();<br> for(int i = local_start_position; i < local_end_position + 1; i++)<br> {<br> if(unencoded_vector[i])<br> {<br> input_file.Read_at(i * 20, buf, 20, MPI_CHAR, input_file_status);<br> }<br> }<br> <br> output_file.Write_shared(buf, ((local_end_position - local_start_position) + 1), MPI_CHAR, output_file_status);<br> <br> free(buf);<br><br> input_file.Close();<br> output_file.Close();<br> <br> MPI::Finalize();<br>}<br><br><br><br><blockquote><hr id="EC_stopSpelling">From: matthew.chambers@vanderbilt.edu<br>To: erichpeterson@hotmail.com; mpich-discuss@mcs.anl.gov<br>Subject: RE: [MPICH] Help With I/O<br>Date: Wed, 18 Apr 2007 13:24:23 -0500<br><br>
<meta http-equiv="Content-Type" content="text/html; charset=unicode">
<meta name="Generator" content="Microsoft SafeHTML">
<style>
.ExternalClass .EC_shape
{;}
</style>
<style>
</style>
<style>
.ExternalClass EC_p.MsoNormal, .ExternalClass EC_li.MsoNormal, .ExternalClass EC_div.MsoNormal
{margin-bottom:.0001pt;font-size:12.0pt;font-family:'Times New Roman';}
.ExternalClass EC_a:link, .ExternalClass EC_span.MsoHyperlink
{color:blue;text-decoration:underline;}
.ExternalClass EC_a:visited, .ExternalClass EC_span.MsoHyperlinkFollowed
{color:blue;text-decoration:underline;}
.ExternalClass EC_span.EmailStyle18
{font-family:Arial;color:navy;}
.ExternalClass EC_span.EmailStyle19
{font-family:Arial;color:navy;}
.ExternalClass EC_div.Section1
{page:Section1;}
.ExternalClass EC_p.MsoNormal, .ExternalClass EC_li.MsoNormal, .ExternalClass EC_div.MsoNormal
{margin-bottom:.0001pt;font-size:12.0pt;font-family:'Times New Roman';}
.ExternalClass EC_a:link, .ExternalClass EC_span.MsoHyperlink
{color:blue;text-decoration:underline;}
.ExternalClass EC_a:visited, .ExternalClass EC_span.MsoHyperlinkFollowed
{color:blue;text-decoration:underline;}
.ExternalClass p
{margin-bottom:.0001pt;font-size:12.0pt;font-family:'Times New Roman';}
.ExternalClass EC_p.ecmsonormal, .ExternalClass EC_li.ecmsonormal, .ExternalClass EC_div.ecmsonormal
{margin-bottom:.0001pt;font-size:12.0pt;font-family:'Times New Roman';}
.ExternalClass EC_span.EmailStyle19
{font-family:Arial;color:navy;}
@page Section1
{size:8.5in 11.0in;}
.ExternalClass EC_div.Section1
{page:Section1;}
</style>
<div class="EC_Section1">
<p class="EC_MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Oh, good point. I didn’t
notice that the num_elements_per_rank was a float. Still, there are
better, more transparent ways to do that (like using modulus: “if(
num_vector_elements % num_processes == 0 ) /* num_elements_per_rank will be an
integer */). But you seem to have ignored the rest of my post. I
said that it would be useful to see the debug output from your for loop (I
accepted your assertion that the program was ok until then). And what is
the status on the buf variable being allocated?</span></font></p>
<p class="EC_MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<div style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color blue; border-width: medium medium medium 1.5pt; padding: 0in 0in 0in 4pt;">
<div>
<div class="EC_MsoNormal" style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">
<hr align="center" size="2" width="100%">
</span></font></div>
<p class="EC_MsoNormal"><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> owner-mpich-discuss@mcs.anl.gov
[mailto:owner-mpich-discuss@mcs.anl.gov]
<b><span style="font-weight: bold;">On Behalf Of </span></b>Erich Peterson<br>
<b><span style="font-weight: bold;">Sent:</span></b> Wednesday, April 18, 2007
12:39 PM<br>
<b><span style="font-weight: bold;">To:</span></b> mpich-discuss@mcs.anl.gov<br>
<b><span style="font-weight: bold;">Subject:</span></b> RE: [MPICH] Help With I/O</span></font></p>
</div>
<p class="EC_MsoNormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<p class="EC_MsoNormal" style="margin-bottom: 12pt;"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">Hi, the reasoning behind the if
statements on the last process, is I might need to allocate 1 or 2 more
elements to the last process it the vector_size / num_processes is not an even
number. As far as debugging goes I have debugged it to the for loop, and
everything seems fine until it. I think it has something to do with needing to
have a lock on one or both of the file accesses. The problem is I have very
limited knowledge of MPI-IO and was hoping someone who does could easily see
what I am doing wrong with the I/O part.<br>
<br>
Thanks,<br>
Erich<br>
<br>
<br>
<br>
</span></font></p>
<div class="EC_MsoNormal" style="text-align: center;" align="center"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">
<hr id="EC_EC_stopSpelling" align="center" size="2" width="100%">
</span></font></div>
<p class="EC_MsoNormal" style="margin-bottom: 12pt;"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">From: matthew.chambers@vanderbilt.edu<br>
To: mpich-discuss@mcs.anl.gov<br>
Subject: Re: [MPICH] Help With I/O<br>
Date: Wed, 18 Apr 2007 09:39:50 -0500</span></font></p>
<div>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Hi Erich,</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Frankly the code leading up to that for
loop is a bit scary. I don’t really know anything about MPI I/O,
but I can get you a few tips on your C++:</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal" style="margin-left: 0.5in; text-indent: -0.25in;"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">-</span></font><font color="navy" size="1"><span style="font-size: 7pt; color: navy;"> </span></font><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Unless you’re using a horribly out of date compiler like MSVC
6, you should use the standard header names <iostream>, <vector>,
<ctime>, etc.</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal" style="margin-left: 0.5in; text-indent: -0.25in;"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">-</span></font><font color="navy" size="1"><span style="font-size: 7pt; color: navy;"> </span></font><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">If you are using a horribly out of date compiler like MSVC 6, you
should upgrade to the free MSVC++ 2005 Express Edition.</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal" style="margin-left: 0.5in; text-indent: -0.25in;"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">-</span></font><font color="navy" size="1"><span style="font-size: 7pt; color: navy;"> </span></font><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">In this case it’s a cosmetic fix, but you should probably
pass the vector<bool> parameter by reference instead of by value.</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal" style="margin-left: 0.5in; text-indent: -0.25in;"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">-</span></font><font color="navy" size="1"><span style="font-size: 7pt; color: navy;"> </span></font><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">You seem to be doing some mind boggling casting in order to
determine if num_elements_per_rank is too big to fit in an int (but only on
your last process?). You might get rid of that voodoo by using size_t
(usually at least an unsigned 4 byte int) for your position indexes instead
(vector<T>::size() returns vector<T>::size_type which usually boils
down to size_t).</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Beyond that, I would need to see some
debug output from your for loop. For example, what indexes are actually
being passed to the I/O calls by each process? Does MPI::File::Read_at()
allocate memory for the “buf” variable you pass it? If not,
you haven’t allocated any memory for it and that would lead to a crash
before you could say “new char.”</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Good luck,</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Matt Chambers</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<div style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color blue; border-width: medium medium medium 1.5pt; padding: 0in 0in 0in 4pt;">
<div>
<div class="EC_MsoNormal" style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">
<hr align="center" size="2" width="100%">
</span></font></div>
<p class="EC_ecmsonormal"><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> owner-mpich-discuss@mcs.anl.gov
[mailto:owner-mpich-discuss@mcs.anl.gov]
<b><span style="font-weight: bold;">On Behalf Of </span></b>Erich Peterson<br>
<b><span style="font-weight: bold;">Sent:</span></b> Wednesday, April 18, 2007
3:02 AM<br>
<b><span style="font-weight: bold;">To:</span></b> mpich-discuss@mcs.anl.gov<br>
<b><span style="font-weight: bold;">Subject:</span></b> [MPICH] Help With I/O</span></font></p>
</div>
<p class="EC_ecmsonormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"></span></font></p>
<p class="EC_ecmsonormal" style="margin-bottom: 12pt;"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">Hi all, I'm trying to write this
little routine which is part of my graduate project. What I'm trying to do is
pass a vector of bools in to the QueryData method. In that method I split the
vector up into equal parts among the number of processes, have each process
open a datafile which as 20 byte records (no new lines), read that record from
the file if the vector they are checking has "true" (basically
if vector[0] = true, it means grab the first record of the file), and lastly,
it should output that record into a new file.<br>
<br>
I have been able to determine it is messing up on the for-loop. The error is:<br>
<br>
[Student@cluster1 erich_test_area]$ mpiexec -n 3
/mnt/pvfs2/acxiom/erich_test_area/RecordRetrieval<br>
terminate called after throwing an instance of 'MPI::Exception'<br>
terminate called after throwing an instance of 'MPI::Exception'<br>
terminate called after throwing an instance of 'MPI::Exception'<br>
rank 0 in job 1 cluster1_33602 caused collective abort of all
ranks<br>
exit status of rank 0: killed by signal 6 <br>
<br>
If someone could please tell me or edit the code if they see what is wrong.
Thanks!<br>
<br>
Main.cpp:<br>
<br>
#include "RecordRetrieval.h"<br>
#include <vector.h><br>
int main()<br>
{<br>
vector<bool> vec;<br>
vec.push_back(true);<br>
vec.push_back(false);<br>
vec.push_back(true);<br>
vec.push_back(false);<br>
vec.push_back(true);<br>
vec.push_back(false);<br>
RecordRetrieval rec;<br>
rec.QueryData(vec, "test.dat");<br>
return 0;<br>
}<br>
<br>
RecordRetrieval.cpp:<br>
<br>
#include "RecordRetrieval.h"<br>
#include "mpi.h"<br>
#include "time.h"<br>
#include "iostream.h"<br>
void RecordRetrieval::QueryData(vector<bool> unencoded_vector, char *
filename)<br>
{<br>
int num_processes;<br>
int num_vector_elements;<br>
float num_elements_per_rank;<br>
int local_start_position;<br>
int local_end_position;<br>
char * buf;<br>
int my_rank;<br>
MPI::File input_file;<br>
MPI::Status input_file_status;<br>
MPI::File output_file;<br>
MPI::Status output_file_status;<br>
//MPI::Offset filesize;<br>
char output_filename[30];<br>
size_t i;<br>
struct tm tim;<br>
time_t now;<br>
now = time(NULL);<br>
tim = *(localtime(&now));<br>
i = strftime(output_filename, 30, "%m_%d_%Y_%H_%M_%S",
&tim);<br>
<br>
/* Let the system do what it needs to start up MPI */<br>
MPI::Init();<br>
/* Get my process rank */<br>
my_rank = MPI::COMM_WORLD.Get_rank();<br>
/* Find out how many processes are being used */<br>
num_processes = MPI::COMM_WORLD.Get_size();<br>
num_vector_elements = unencoded_vector.size();<br>
<br>
num_elements_per_rank = num_vector_elements /
num_processes;<br>
local_start_position = my_rank * (int)num_elements_per_rank;<br>
if(my_rank == num_processes - 1)<br>
{<br>
if(num_elements_per_rank *
num_processes == (int)num_elements_per_rank * num_processes)<br>
{<br>
local_end_position = local_start_position + ((int)num_elements_per_rank - 1);<br>
}<br>
else<br>
{<br>
local_end_position = (local_start_position + (int)num_elements_per_rank - 1) +<br>
(((int)num_elements_per_rank * num_processes) - ((int)num_elements_per_rank *
num_processes));<br>
}<br>
}<br>
else<br>
{<br>
local_end_position =
local_start_position + ((int)num_elements_per_rank - 1);<br>
}<br>
input_file = MPI::File::Open(MPI::COMM_WORLD, filename,
MPI::MODE_RDONLY,<br>
MPI::INFO_NULL);<br>
output_file = MPI::File::Open(MPI::COMM_WORLD,
output_filename, MPI::MODE_CREATE | MPI::MODE_WRONLY, MPI::INFO_NULL);<br>
// filesize = input_file.Get_size();<br>
for(int i = local_start_position; i < local_end_position
+ 1; i++)<br>
{<br>
if(unencoded_vector[i])<br>
{<br>
input_file.Read_at(i * 20, buf, 20, MPI_CHAR, input_file_status);<br>
output_file.Write_shared(buf, 20, MPI_CHAR, output_file_status);<br>
}<br>
}<br>
cout << "Error";<br>
input_file.Close();<br>
output_file.Close();<br>
<br>
MPI::Finalize();<br>
}</span></font></p>
<div class="EC_MsoNormal" style="text-align: center;" align="center"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">
<hr align="center" size="2" width="100%">
</span></font></div>
<p class="EC_ecmsonormal"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">Discover the new Windows Vista <a href="http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE" target="_blank">Learn more!</a></span></font></p>
</div>
</div>
<p class="EC_MsoNormal"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> </span></font></p>
<div class="EC_MsoNormal" style="text-align: center;" align="center"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">
<hr align="center" size="2" width="100%">
</span></font></div>
<p class="EC_MsoNormal"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">Invite your mail contacts to join your friends list with
Windows Live Spaces. It's easy! <a href="http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us" target="_blank">Try it!</a></span></font></p>
</div>
</div>
</blockquote><br /><hr />Connect to the next generation of MSN Messenger <a href='http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-us&source=wlmailtagline' target='_new'>Get it now! </a></body>
</html>