Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Nicolaou, Mihalis; Panagakis, Yannis; Zafeiriou, Stefanos; Pantic, Maja
Publisher: IEEE Computer Society
Languages: English
Types: Conference object,Article
The problem of automatically estimating the interest level of a subject has been gaining attention by researchers, mostly due to the vast applicability of interest detection. In this work, we obtain a set of continuous interest annotations for the SE-MAINE database, which we analyse also in terms of emotion dimensions such as valence and arousal. Most importantly, we propose a robust variant of Canonical Correlation Analysis (RCCA) for performing audio-visual fusion, which we apply to the prediction of interest. RCCA recovers a low-rank subspace which captures the correlations of fused modalities, while isolating gross errors in the data without making any assumptions regarding Gaussianity. We experimentally show that RCCA is more appropriate than other standard fusion techniques (such as l2-CCA and feature-level fusion), since it both captures interactions between modalities while also decontaminating the obtained subspace from errors which are dominant in real-world problems.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • [1] Alex Pentland and Anmol Madan, “Perception of social interest,” in Proc. IEEE Int. Conf. on Computer Vision, Workshop on Modeling People and Human Interaction (ICCV-PHI), 2005.
    • [2] Bjo¨ rn Schuller, Ronald Mu¨ ller, Florian Eyben, J u¨rgen Gast, Benedikt Ho¨ rnler, Martin Wo¨ llmer, Gerhard Rigoll, Anja H o¨thker, and Hitoshi Konosu, “Being bored? recognising natural interest by extensive audiovisual integration for real-life application,” Image and Vision Computing, vol. 27, no. 12, pp. 1760-1774, 2009.
    • [3] Bjo¨ rn Schuller and Gerhard Rigoll, “Recognising interest in conversational speech-comparing bag of frames and supra-segmental features.,” in INTERSPEECH, 2009, pp. 1999-2002.
    • [4] Felix Arnold, Attention and interest: A study in psychology and education, Macmillan, 1910.
    • [5] Silvan S Tomkins, “Affect, imagery, consciousness: Vol. i. the positive affects.,” 1962.
    • [6] Paul J Silvia, Exploring the psychology of interest, Oxford University Press, 2006.
    • [7] Bjo¨ rn Schuller, Niels K o¨hler, Ronald Mu¨ ller, and Gerhard Rigoll, “Recognition of interest in human conversational speech.,” in INTERSPEECH, 2006.
    • [8] Martin Wo¨ llmer, Florian Eyben, Stephan Reiter, Bjo¨ rn Schuller, Cate Cox, Ellen Douglas-Cowie, and Roddy Cowie, “Abandoning emotion classes-towards continuous emotion recognition with modelling of long-range dependencies.,” in INTERSPEECH, 2008, pp. 597-600.
    • [9] Mihalis A Nicolaou, Hatice Gunes, and Maja Pantic, “Outputassociative rvm regression for dimensional and continuous emotion prediction,” Image and Vision Computing, vol. 30, no. 3, pp. 186-196, 2012.
    • [10] Angeliki Metallinou and Shrikanth S. Narayanan, “Annotation and processing of continuous emotional attributes: Challenges and opportunities,” in 2nd International Workshop on Emotion Representation, Analysis and Synthesis in Continuous Time and Space (EmoSPACE 2013), Apr. 2013.
    • [11] Angeliki Metallinou, Martin Wollmer, Athanasios Katsamanis, Florian Eyben, Bjo¨ rn Schuller, and Shrikanth Narayanan, “Context-sensitive learning for enhanced audiovisual emotion classification,” Affective Computing, IEEE Transactions on, vol. 3, no. 2, pp. 184-198, 2012.
    • [12] Hatice Gunes, Bjo¨ rn Schuller, Maja Pantic, and Roddy Cowie, “Emotion representation, analysis and synthesis in continuous space: A survey,” in Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on. IEEE, 2011, pp. 827- 834.
    • [13] Geovany A Ramirez, Tadas Baltrusˇaitis, and Louis-Philippe Morency, “Modeling latent discriminative dynamic of multi-dimensional affective signals,” in Affective Computing and Intelligent Interaction, pp. 396-406. Springer, 2011.
    • [14] James A Russell, Maria Lewicka, and Toomas Niit, “A cross-cultural study of a circumplex model of affect.,” Journal of personality and social psychology, vol. 57, no. 5, pp. 848, 1989.
    • [15] Mihalis A Nicolaou, Hatice Gunes, and Maja Pantic, “Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space,” Affective Computing, IEEE Transactions on, vol. 2, no. 2, pp. 92-105, 2011.
    • [16] Hatice Gunes and Bj o¨rn Schuller, “Categorical and dimensional affect analysis in continuous input: Current trends and future directions,” Image and Vision Computing, 2012.
    • [17] Peter J Lang, Mark K Greenwald, Margaret M Bradley, and Alfons O Hamm, “Looking at pictures: Affective, facial, visceral, and behavioral reactions,” Psychophysiology, vol. 30, no. 3, pp. 261-273, 1993.
    • [18] Tadas Baltrusˇaitis, Ntombikayise Banda, and Peter Robinson, “Dimensional affect recognition using continuous conditional random fields,” in IEEE FG, 2013.
    • [19] Mihalis A. Nicolaou, Stefanos Zafeiriou, and Maja Pantic, “Correlatedspaces regression for learning continuous emotion dimensions,” in Proceedings of the 21st ACM international conference on Multimedia. ACM, 2013, pp. 773-776.
    • [20] Caifeng Shan, Shaogang Gong, and Peter W McOwan, “Beyond facial expressions: Learning human emotion from body gestures.,” in BMVC, 2007, pp. 1-10.
    • [21] Nicolle M Correa, Yi-Ou Li, T u¨lay Adali, and Vince D Calhoun, “Fusion of fmri, smri, and eeg data using canonical correlation analysis,” in Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on. IEEE, 2009, pp. 385-388.
    • [22] Gary McKeown et al., “The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent,” IEEE TAC, 2012.
    • [23] Mihalis A. Nicolaou, Vladimir Pavlovic, and Maja Pantic, “Dynamic Probabilistic CCA for Analysis of Affective Behaviour,” in Proceedings of the 12th European Conference on Computer Vision, ECCV 2012., Florence, Italy, October 2012, pp. 98-111.
    • [24] Soroosh Mariooryad and Carlos Busso, “Analysis and compensation of the reaction lag of evaluators in continuous emotional annotations,” in Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on. IEEE, 2013, pp. 85-90.
    • [25] J. Orozco et al., “Hierarchical on-line appearance-based tracking for 3d head pose, eyebrows, lips, eyelids and irises,” Image and Vision Computing, February 2013.
    • [26] Zhihong Zeng, M. Pantic, G.I. Roisman, and T.S. Huang, “A survey of affect recognition methods: Audio, visual, and spontaneous expressions,” IEEE TPAMI, 2009.
    • [27] M. E. Tipping, “Sparse bayesian learning and the relevance vector machine,” JMLR, vol. 1, pp. 211-244, 2001.
    • [28] L. Vandenberghe and S. Boyd, “Semidefinite programming,” SIAM Review, vol. 38, no. 1, pp. 49-95, 1996.
    • [29] B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM J. Comput., vol. 24, no. 2, pp. 227-234, 1995.
    • [30] D. Donoho, “For most large underdetermined systems of equations, the minimal l1-norm near-solution approximates the sparsest nearsolution,” Communications on Pure and Applied Mathematics, vol. 59, no. 7, pp. 907-934, 2006.
    • [31] M. Fazel, Matrix Rank Minimization with Applications, Ph.D. thesis, Dept. Electrical Engineering, Stanford University, CA, USA, 2002.
    • [32] Z. Lin, R. Liu, and Z. Su, “Linearized alternating direction method with adaptive penalty for low-rank representation,” in Proc. 2011 Neural Information Processing Systems Conf., Granada, Spain, 2011, pp. 612- 620.
    • [33] D. P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Athena Scientific, Belmont, MA, 2nd edition, 1996.
    • [34] J. F. Cai, E. J. Candes, and Z. Shen, “A singular value thresholding algorithm for matrix completion,” SIAM Journal Optimization, vol. 2, no. 2, pp. 569-592, 2009.
    • [35] E. J. Candes, X. Li, Y. Ma, and J. Wright, “Robust principal component analysis?,” Journal of ACM, vol. 58, no. 3, pp. 1-37, 2011.
  • Inferred research data

    The results below are discovered through our pilot algorithms. Let us know how we are doing!

    Title Trust
  • Discovered through pilot similarity algorithms. Send us your feedback.

Share - Bookmark

Funded by projects

  • RCUK | Analysis of Facial Behavio...
  • EC | FROG

Cite this article