A number of image quality assessment (IQA) metrics have been designed in recent years for natural images, leading to a desire to develop IQA approaches for screen content image which is composed of textual as well as pictorial regions and exhibits different visual characteristics from the natural image. In this work, a no reference IQA metric based on convolutional neural network (CNN) is proposed for screen content image, which fuses the quality scores of textual and pictorial image patches by taking the differences of their visual features into accounts. Experimental results on the benchmark screen image quality assessment database (SIQAD) verify the effectiveness of the proposed CNN based approach as compared with several state-of-the-art IQA metrics.