dropping columns/rows from matrix?

[Peter Schröder]

As part of a greedy basis pursuit algorithm I drop/undrop columns/rows 
from a matrix and resolve. I don't want to rebuild the matrix each time. 
Is there a quick way to do this?

Basically the setup is this. Consider a 2-manifold triangle mesh and a 
discretization (piecewise linear FE) of the Laplace-Beltrami operator 
over this mesh (symmetric positive (semi-)definite [constant vector is 
the only null space vector]). Fix boundary conditions (zero Dirichlet in 
my case). Solve for a given rhs (I am using CG and absolute Jacobi as a 
precon with good success). Based on the solution, take out a column (and 
the same row) and resolve. Repeat this a few times until, say, 10 
variables are dropped. Now pick one of them, say, i, and reintroduce it. 
Based on the solution replace i with i_new. Now visit another variable 
of the original 10 and "move" it. Etc.

Each one of the solves is quick (and I need to do hundreds for matrices 
with hundreds of thousands to millions of variables). I'd rather not 
rebuild the matrix each time... Any suggestions?

Thanks much!

Peter