The restarted Krylov subspace methods, including the Galerkin method and the leastsquares method, are popular and important for solving large linear systems of equations. However, the Galerkin method may suffer from serious breakdown, and the leastsquares method may encounter complete stagnation. To overcome the problems, a new restarting scheme, called the alternately restarting scheme, is proposed in this paper. The underlying idea is to use the Krylov subspaces generated by the coefficient matrix and its transpose alternately. We show that for an alternately restarted Krylov method, its residual tends to get the same reduction in every eigenvector direction, and therefore its convergence can be significantly improved. Numerical experiments are conducted, which indicate that the alternately restarted Krylov subspace methods are efficient and robust.
