Abstract: In order to alleviate affective gap and large intra-class variance existing in visual sentiment analysis, firstly a new method is proposed, which exploits simultaneously not only deep latent correlations between visual and textual modalities, but also deep linear discrimination of visual modality and weak supervision of mid-level semantic features of images. The method uses multimodal deep network architecture to find a latent embedding space in which deep correlations between visual and textual modalities are maximized, and at the same time there is a deep discrimination on visual modality. In the latent space, the extracted semantic feature of texts can be transferred to the extracted discriminant visual feature of images. Secondly based on the usfulness of attention mechanism, an attention network is presented, which accepts the extracted features in the latent space as input and is trained as a sentiment classifier. Results of experiments conducted on real datasets show that the proposed approach achieves better sentiment classification accuracy than those state-of-the-art approaches.

Key words: sentiment analysis, latent correlation, linear discrimination, multimodal network, attention mechanism