An Improved Hybrid Model for Nonlinear Regression with Missing Values Using Deep Quasi-Linear Kernel

Missing values are ubiquitous in the nonlinear regression research, and may lead to bias and a loss of efficiency. Even in a large dataset, values drop-out can substantially reduce the available information for analysis. In this paper, we propose an improved hybrid model to solve the nonlinear regression problem under missing data scenarios, consisting of two parts: an overcomplete winner-take-all (WTA) autoencoder and a multilayer gated linear network. The WTA autoencoder is trained in an adversarial training process by taking advantage of gradually renewed teacher signals and the discrimination of missing values and observed values, and is designed to play two roles: (1) to impute missing components conditioned on observed samples; (2) to generate gate control sequences. On the other hand, the multilayer gated linear network with the generated gate control sequences implements a powerful piecewise linear regression model, whose parameters are optimized by formulating a support vector regression (SVR) with a deep quasi-linear kernel. Experimental results based on different real-world datasets demonstrate the effectiveness of our proposed hybrid model.