Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Yao, Yi
Languages: English
Types: Doctoral thesis
Subjects: QA76

Classified by OpenAIRE into

Human Computer Interaction has been relying on mechanical devices to feed information into computers with low efficiency for a long time. With the recent developments in image processing and machine learning methods, the computer vision community is ready to develop the next generation of Human Computer Interaction methods, including Hand Gesture Recognition methods. A comprehensive Hand Gesture Recognition based semantic level Human Computer Interaction framework for uncontrolled environments is proposed in this thesis. The framework contains novel methods for Hand Posture Recognition, Hand Gesture Recognition and Hand Gesture Spotting.\ud \ud The Hand Posture Recognition method in the proposed framework is capable of recognising predefined still hand postures from cluttered backgrounds. Texture features are used in conjunction with Adaptive Boosting to form a novel feature selection scheme, which can effectively detect and select discriminative texture features from the training samples of the posture classes.\ud \ud A novel Hand Tracking method called Adaptive SURF Tracking is proposed in this thesis. Texture key points are used to track multiple hand candidates in the scene. This tracking method matches texture key points of hand candidates within adjacent frames to calculate the movement directions of hand candidates.\ud \ud With the gesture trajectories provided by the Adaptive SURF Tracking method, a novel classi´┐Żer called Partition Matrix is introduced to perform gesture classification for uncontrolled environments with multiple hand candidates. The trajectories of all hand candidates extracted from the original video under different frame rates are used to analyse the movements of hand candidates. An alternative gesture classifier based on Convolutional Neural Network is also proposed. The input images of the Neural Network are approximate trajectory images reconstructed from the tracking results of the Adaptive SURF Tracking method.\ud \ud For Hand Gesture Spotting, a forward spotting scheme is introduced to detect the starting and ending points of the prede´┐Żned gestures in the continuously signed gesture videos. A Non-Sign Model is also proposed to simulate meaningless hand movements between the meaningful gestures.\ud \ud The proposed framework can perform well with unconstrained scene settings, including frontal occlusions, background distractions and changing lighting conditions. Moreover, it is invariant to changing scales, speed and locations of the gesture trajectories.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • [15] CATHERINE G. WOLF. A comparative study of gestural, keyboard, and mouse interfaces. Behaviour and Information Technology, 11(1):13{23, 1992.
    • [16] Microsoft. Kinect. http://www.microsoft.com/en-us/kinectforwindows/, 2009. [Online; accessed July-2014].
    • [17] ThalmicLabs. MYO armband. https://www.thalmic.com/en/myo/, 2014. [Online; accessed July-2014].
    • [18] LeapMotion Inc. Leap Motion Controller. https://www.leapmotion.com/, 2010. [Online; accessed July-2014].
    • [19] A.Jaimes and N. Sebe. Multimodal human computer interaction: A survey. Computer Vision and Image Understanding, 108:116 { 134, 2007.
    • [20] J. Nespoulous, P. Perron, and A. R. Lecours. The biological foundations of gestures: Motor and semiotic aspects. New Jersey London: Lawrence Erlbaum associates, 1986.
    • [21] William C Stokoe, Dorothy C Casterline, and Carl G Croneberg. A dictionary of American Sign Language on linguistic principles. Linstok Press Silver Spring, 1976.
    • [22] David Brien, British Deaf Association, et al. Dictionary of British Sign Language English. Faber & Faber, 1992.
    • [23] Chunli Wang, Wen Gao, and Shiguang Shan. An approach based on phonemes to large vocabulary chinese sign language recognition. In Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on, pages 411{416. IEEE, 2002.
    • [24] Quan Yuan, Stan Sclaro , and Vassilis Athitsos. Automatic 2d hand tracking in video sequences. In Application of Computer Vision, 2005.
    • [25] Alejandro Jaimes and Nicu Sebe. Multimodal human{computer interaction: A survey. Computer vision and image understanding, 108(1):116{134, 2007.
    • [26] Kai Nickel, Edgar Scemann, and Rainer Stiefelhagen. 3d-tracking of head and hands for pointing gesture recognition in a human-robot interaction scenario. In Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on, pages 565{570. IEEE, 2004.
    • [27] Toshiyuki Kirishima, Kosuke Sato, and Kunihiro Chihara. Real-time gesture recognition by learning and selective control of visual interest points. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27(3):351{ 364, 2005.
    • [28] Sebastien Carbini, Jean Emmanuel Viallet, and Olivier Bernier. Pointing gesture visual recognition for large display. In FG Net Workshop on Visual Observation of Deictic Gestures, 2004.
    • [29] Sanparith Marukatat, Thierry Artieres, and Patrick Gallinari. A generic approach for on-loine handwriting recognition. 2004.
    • [30] Beat Signer, Ueli Kurmann, and Moira C Norrie. igesture: a general gesture recognition framework. In Document Analysis and Recognition, 2007. ICDAR 2007. Ninth International Conference on, volume 2, pages 954{958. IEEE, 2007.
    • [32] Sebastien Marcel. Hand posture recognition in a body-face centered space. In CHI'99 Extended Abstracts on Human Factors in Computing Systems, pages 302{303. ACM, 1999.
    • [33] Sotiris Malassiotis and Michael G Strintzis. Real-time hand posture recognition using range data. Image and Vision Computing, 26(7):1027{1037, 2008.
    • [34] Lars Bretzner, Ivan Laptev, and Tony Lindeberg. Hand gesture recognition using multi-scale colour features, hierarchical models and particle ltering. In Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on, pages 423{428. IEEE, 2002.
    • [35] Agnes Just, Yann Rodriguez, and Sebastien Marcel. Hand posture classi cation and recognition using the modi ed census transform. In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on, pages 351{356. IEEE, 2006.
    • [40] Lars Bretzner, Ivan Laptev, and Tony Lindeberg. Hand gesture recognition using multi-scale colour features, hierarchical models and particle ltering. In Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on, pages 423{428. IEEE, 2002.
    • [41] Feng-Sheng Chen, Chih-Ming Fu, and Chung-Lin Huang. Hand gesture recognition using a real-time tracking method and hidden markov models. Image and Vision Computing, 21(8):745{758, 2003.
    • [42] Arnaud Lemoine, Sean Mcgrath, Anthony Vernon Walker Smith, and Alistair Ian Sutherland. Hand gesture recognition system and method, October 3 2000. US Patent 6,128,003.
    • [43] Zhou Ren, Jingjing Meng, Junsong Yuan, and Zhengyou Zhang. Robust hand gesture recognition with kinect sensor. In Proceedings of the 19th ACM international conference on Multimedia, pages 759{760. ACM, 2011.
    • [44] Zhou Ren, Junsong Yuan, and Zhengyou Zhang. Robust hand gesture recognition based on nger-earth mover's distance with a commodity depth camera. In Proceedings of the 19th ACM international conference on Multimedia, pages 1093{1096. ACM, 2011.
    • [45] Sung-Ho Im, Dong-Sun Lim, Tae-Joon Park, Kee-Koo Kwon, Man-Seok Yang, and Heung-Nam Kim. User interface apparatus using hand gesture recognition and method thereof, April 20 2010. US Patent 7,702,130.
    • [46] KS Chidanand Kumar. Segmentation and feature space analysis for hand gesture recognition under complex background. Science, 3(3), 2012.
    • [47] Thad E Starner. Visual recognition of american sign language using hidden markov models. Technical report, DTIC Document, 1995.
    • [48] Richard A Bolt. Put-that-there: Voice and gesture at the graphics interface, volume 14. ACM, 1980.
    • [49] Thomas G Zimmerman, Jaron Lanier, Chuck Blanchard, Steve Bryson, and Young Harvill. A hand gesture interface device. In ACM SIGCHI Bulletin, volume 18, pages 189{192. ACM, 1987.
    • [50] David J Sturman, David Zeltzer, and Steve Pieper. Hands-on interaction with virtual environments. In Proceedings of the 2nd annual ACM SIGGRAPH symposium on User interface software and technology, pages 19{24. ACM, 1989.
    • [51] Alexander G Hauptmann. Speech and gestures for graphic image manipulation. In ACM SIGCHI Bulletin, volume 20, pages 241{245. ACM, 1989.
    • [52] David L Quam. Gesture recognition with a dataglove. In Aerospace and Electronics Conference, 1990. NAECON 1990., Proceedings of the IEEE 1990 National, pages 755{760. IEEE, 1990.
    • [53] Alexander G Hauptmann and Paul McAvinney. Gestures with speech for graphic manipulation. International Journal of Man-Machine Studies, 38(2):231{249, 1993.
    • [54] Mahmoud Elmezain, Ayoub Al-Hamadi, and Bernd Michaelis. A robust method for hand gesture segmentation and recognition using forward spotting scheme in conditional random elds. In Pattern Recognition (ICPR), 2010 20th International Conference on, pages 3850{3853. IEEE, 2010.
    • [55] Bozon, Mark. Nintendo Sets the Record Straight. http://www.nintendo. com/wiiu;jsessionid=D5F2C11D626712CE04E2F54043807248, 2006. [Online; accessed July-2014].
    • [56] Anbumani Subramanian, Vinod Pathangay, and Dinesh Mandalapu. Hand gesture recognition. 2010.
    • [57] Richard O Duda, Peter E Hart, and David G Stork. Pattern classi cation. John Wiley & Sons, 2012.
    • [58] Neidle, C. Boston ASL dataset. http://www.bu.edu/asllrp/cslgr/, 2006. [Online; accessed July-2014].
    • [59] J Kenneth Salisbury and John J Craig. Articulated hands force control and kinematic issues. The International Journal of Robotics Research, 1(1):4{17, 1982.
    • [60] James M Rehg, Daniel D Morris, and Takeo Kanade. Ambiguities in visual tracking of articulated objects using two-and three-dimensional models. The International Journal of Robotics Research, 22(6):393{418, 2003.
    • [61] James M Rehg and Takeo Kanade. Digiteyes: Vision-based hand tracking for human-computer interaction. In Motion of Non-Rigid and Articulated Objects, 1994., Proceedings of the 1994 IEEE Workshop on, pages 16{22. IEEE, 1994.
    • [62] William C Stokoe. Sign language structure. 1978.
    • [63] Philippe Dreuw, Thomas Deselaers, David Rybach, Daniel Keysers, and Hermann Ney. Tracking using dynamic programming for appearance-based sign language recognition. In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on, pages 293{298. IEEE, 2006.
    • [64] Kikuo Fujimura and Xia Liu. Sign recognition using depth image streams. In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on, pages 381{386. IEEE, 2006.
    • [66] Richard Bowden, David Windridge, Timor Kadir, Andrew Zisserman, and Michael Brady. A linguistic feature vector for the visual interpretation of sign language. In Computer Vision-ECCV 2004, pages 390{401. Springer, 2004.
    • [67] Richard Bowden, Andrew Zisserman, Timor Kadir, and Mike Brady. Vision based interpretation of natural sign languages. 2003.
    • [68] Sushmita Mitra and Tinku Acharya. Gesture recognition: A survey. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 37(3):311{324, 2007.
    • [69] Rogerio Feris, Matthew Turk, Ramesh Raskar, Kar-Han Tan, and Gosuke Ohashi. Recognition of isolated ngerspelling gestures using depth edges. In Real-Time Vision for Human-Computer Interaction, pages 43{56. Springer, 2005.
    • [74] Hee-Deok Yang and Seong-Whan Lee. Robust sign language recognition with hierarchical conditional random elds. In Pattern Recognition (ICPR), 2010 20th International Conference on, pages 2202{2205. IEEE, 2010.
    • [75] Itay Katz. EyeSight Technology. http://eyesight-tech.com/, 2014. [Online; accessed July-2014].
    • [76] DANIEL VAN NIEUWENHOVE. Softkinetic. http://www.softkinetic. com/en-us/softkinetic.aspx, 2014. [Online; accessed July-2014].
    • [77] Mahmoud Elmezain and Ayoub Al-Hamadi. Ldcrfs-based hand gesture recognition. In Systems, Man, and Cybernetics (SMC), 2012 IEEE International Conference on, pages 2670{2675. IEEE, 2012.
    • [78] PointGrab Ltd. pointgrab. http://www.pointgrab.com/, 2014. [Online; accessed July-2014].
    • [79] Faisal Yazadi and Mark Schelbert. Cyber Glove Systems - worldwide leader in data glove technology. http://www.cyberglovesystems.com/index.php, 2010. [Online; accessed July-2014].
    • [80] Cem Keskin, Furkan K rac, Yunus Emre Kara, and Lale Akarun. Hand pose estimation and hand shape classi cation using multi-layered randomized decision forests. In Computer Vision{ECCV 2012, pages 852{863. Springer, 2012.
    • [81] Stan Melax, Leonid Keselman, and Sterling Orsten. Dynamics based 3d skeletal hand tracking. In Proceedings of Graphics Interface 2013, pages 63{70. Canadian Information Processing Society, 2013.
    • [98] Benjamin D Zarit, Boaz J Super, and Francis KH Quek. Comparison of ve color models in skin pixel classi cation. In Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 1999. Proceedings. International Workshop on, pages 58{63. IEEE, 1999.
    • [99] Stan Birch eld. Elliptical head tracking using intensity gradients and color histograms. In Computer Vision and Pattern Recognition, 1998. Proceedings. 1998 IEEE Computer Society Conference on, pages 232{237. IEEE, 1998.
    • [100] Wen-Hsiang Lai and Chang-Tsun Li. Skin colour-based face detection in colour images. In Video and Signal Based Surveillance, 2006. AVSS'06. IEEE International Conference on, pages 56{56. IEEE, 2006.
    • [101] D. Tang, H.J. Chang, A. Tejani, and T-K. Kim. Latent regression forest: Structural estimation of 3d articulated hand posture. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). IEEE, 2014.
    • [131] Alexandre Alahi, Raphael Ortiz, and Pierre Vandergheynst. Freak: Fast retina keypoint. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 510{517. Ieee, 2012.
    • [139] Mahmoud Elmezain, Ayoub Al-Hamadi, Jorg Appenrodt, and Bernd Michaelis. A hidden markov model-based continuous gesture recognition system for hand motion trajectory. In Pattern Recognition, 2008. ICPR 2008. 19th International Conference on, pages 1{4. IEEE, 2008.
    • [176] Richard C Rose, Edward M Hofstetter, and Douglas A Reynolds. Integrated models of signal and background with application to speaker identi cation in noise. Speech and Audio Processing, IEEE Transactions on, 2(2):245{257, 1994.
    • [177] A P Varga and RK Moore. Hidden markov model decomposition of speech and noise. In Acoustics, Speech, and Signal Processing, 1990. ICASSP-90., 1990 International Conference on, pages 845{848. IEEE, 1990.
    • [179] Danhang Tang, Hyung Jin Chang, Alykhan Tejani, and Tae-Kyun Kim. Latent regression forest: Structured estimation of 3d articulated hand posture. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 3786{3793. IEEE, 2014.
  • No related research data.
  • No similar publications.

Share - Bookmark

Cite this article