Behind the mask: Masquerading the reason for prediction

Ensuring the interpretability of the machine-learning-based prediction models is important for gaining users’ trust. The current representative algorithms for ensuring interpretability, LIME and SHAP, explain the prediction of a given black-box machine learning model based on a common perturbation mechanism to input data. In this study, we propose a masquerade layer to nullify this perturbation and hide the reason for prediction. Our proposed masquerade layer can be attached to any prediction models. It can also be attached without altering the prediction model itself, making it possible to manipulate the explanation provided by the interpretability algorithm in a manner that hardly changes the prediction model behavior. The experimental results show that existing representative perturbation-based interpretability algorithms have a critical weaknesses in terms of their reliability.