This paper addresses a noise suppression problem, namely the estimation of non-stationary noise sequences. In this problem, we assume that non-stationary noise can be decomposed into stationary and non-stationary components. These components are described respectively as the bias factor and the residual signal between the bias component and noise at each frame. This decomposition clarifies the role of each component, thus enabling us to apply a suitable parameter estimation technique to each component. In this paper, the bias component is estimated by the EM algorithm with the entire observed signal sequence. On the other hand, the residual component is sequentially estimated by multiplying the extended Kalman filter with the EM algorithm. In the evaluation results, we confirmed that the proposed method improved speech recognition accuracy compared with the noise estimation methods without component decomposition.