Image quality assessment (IQA) is an inherent research topic in image processing field for several decades. Recently, machine learning has achieved success in many multimedia tasks and can be applied in IQA. Especially, screen content images (SCIs) is greatly increasing in various applications, but the characteristics of SCIs makes it difficult to directly apply general IQA methods to predict qualities. In this paper, we propose a fast no-reference SCIs quality prediction method. First, we use the convolutional neural networks (CNNs) to predict the quality scores of each patch. Second, we present a SCIs-oriented quality aggregation algorithm for acceleration. Experimental results demonstrate that our method can achieve the high accuracy (0.957) with subjective quality scores, outperforming existing methods. Moreover, our method is computationally appealing, achieving flexible complexity performance by selecting different groups of patches.