A Winner-Take-All Autoencoder Based Pieceswise Linear Model for Nonlinear Regression with Missing Data

Missing data is a prevailing problem in predictive analytics. In this paper, a winner-take-all (WTA) autoencoder-based piecewise linear model is developed to solve the nonlinear regression problem under the missing value scenario, which consists of two parts: an overcomplete WTA autoencoder and a gated linear network. The overcomplete WTA autoencoder is a stacked denoising autoencoder (SDAE) designed to play two roles: (1) to estimate the missing values; (2) to realize a sophisticated partitioning by generating a broad set of binary gate control sequences. Besides, an iterative algorithm with renewed teacher signals is developed to train the SDAE. On the other hand, the gated linear network with the generated binary gate control sequences implements a flexible piecewise linear model for nonlinear regression. By composing a quasi-linear kernel based on the gate control sequences, the piecewise linear model is then identified in the same way as a support vector regression. Experimental results have shown that our proposed hybrid model has a better performance than traditional models.