Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Liang, Hui; Chang, Jian; Deng, Shujie; Chen, C.; Tong, R.; Zhang, Jian J. (2017)
Languages: English
Types: Article

Classified by OpenAIRE into

With the rapid development of human-computer interaction technologies, the new media generation demands novel learning experiences with natural interaction and immersive experience. Considering that digital storytelling is a powerful pedagogical tool for young children, in this paper, we design an immersive storytelling environment that allows multiple players to use naturally interactive hand gestures to manipulate virtual puppetry for assisting narration. A set of multimodal interaction techniques is presented for a hybrid user interface that integrates existing 3D visualization and interaction devices including head-mounted displays and depth motion sensor. In this system, the young players could intuitively use hand gestures to manipulate virtual puppets to perform a story and interact with props in a virtual stereoscopic environment. We have conducted a user experiment with four young children for pedagogical evaluation, as well as system acceptability and interactivity evaluation by postgraduate students. The results show that our framework has great potential to stimulate learning abilities of young children through collaboration tasks. The stereoscopic head-mounted display outperformed the traditional monoscopic display in a comparison between the two.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • 1. Psomos P, Kordaki M. Pedagogical analysis of educational digital storytelling environments of the last five years. ProcediaSocial and Behavioural Sciences 2012; 46: 1213-1218.
    • 2. More C. Digital stories targeting social skills for children with disabilities multidimensional learning. Intervention in School and Clinic 2008; 43(3): 168-177.
    • 3. Garzotto F, Paolini P, Sabiescu A. Interactive storytelling for children. In Proceedings of the 9th International Conference on Interaction Design and Children, ACM 2010; 356-359.
    • 4. Hofer M, Owings SWAN K. Digital storytelling: Moving from promise to practice. In Society for Information Technology & Teacher Education International Conference, 2006(1): 679-684.
    • 5. Lu F, Tian F, Jiang Y, Cao X, Luo W, Li G, Zhang X, Dai G, Wang H. ShadowStory: creative and collaborative digital storytelling inspired by cultural heritage. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM 2011; 1919-1928.
    • 6. Thompson P. The digital natives as learners: Technology use patterns and approaches to learning. Computers & Education 2013; 65: 12-33.
    • 7. Barnes K, Marateo RC, Ferris SP. Teaching and learning with the net generation. Innovate: Journal of Online Education 2007; 3(4): 1.
    • 8. Jonassen DH. Computers as mind tools for schools: Engaging critical thinking, Prentice Hall, 2000.
    • 9. Smeets E. Does ICT contribute to powerful learning environments in primary education? Computers & Education 2005; 44(3): 343-355.
    • 10. Garzotto F, Forfori M. FaTe2: storytelling edutainment experiences in 2D and 3D collaborative spaces. In Proceedings of the 2006 conference on Interaction design and children, ACM 2006; 113-116.
    • 11. Mckinley B, Lee YL. Mystorymaker. In CHI'08 Extended Abstracts on Human Factors in Computing Systems, ACM 2008; 3219-3224.
    • 12. Russell A. ToonTastic: a global storytelling network for kids, by kids. In Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction, ACM 2010; 271- 274.
    • 13. Kelleher C. Motivating Programming: Using storytelling to make computer programming attractive to middle school girls. No. CMU-CS-06-171. CARNEGIE-MELLON UNIV PITTSBURGH PA SCHOOL OF COMPUTER SCIENCE, 2006.
    • 14. Widjajanto WA, Lund M, Schelhowe H. Wayang Authoring: a web-based authoring tool for visual storytelling for children. In Proceedings of the 6th International Conference on Advances in Mobile Computing and Multimedia, ACM 2008; 464-467.
    • 15. Vaucelle C, Ishii H. Picture this!: film assembly using toy gestures. In Proceedings of the 10th international conference on Ubiquitous computing, ACM 2008; 350-359.
    • 16. https://apps.leapmotion.com/apps/virtual-piano-forbeginners/windows. Access date: 19, Jan, 2016.
    • 17. https://www.oculus.com Access date: 19, Jan, 2016.
    • 18. Dufour T, Pellarrey V, Chagnon P, Majdoubi A, Torregrossa T, Nachbaur V, Li C, Ibarra Cortes R, Clermont J, Dumas, F. ASCENT: a first person mountain climbing game on the oculus rift. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play, ACM 2014; 335-338.
    • 19. http://zerolatencymedia.com Access date: 19, Jan, 2016.
    • 20. http://www.projectholodeck.com Access date: 1, Nov, 2015.
    • 21. Greer, R. Douglas. Designing teaching strategies: An applied behavior analysis systems approach. Academic Press, 2002.
    • 22. Vargas EA, Vargas JS. Programmed instruction: What it is and how to do it. Journal of Behavioral Education 1991; 1(2): 235-251.
    • 23. Skinner BF. The technology of teaching, New York: AppletonCentury-Crofts, 1968.
    • 24. Wai J, Lubinski D, Benbow CP. Spatial ability for STEM domains: aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology 2009; 101(4): 817.
    • 25. Franceschini S, Gori S, Ruffino M, Pedrolli K, Facoetti A. A causal link between visual spatial attention and reading acquisition. Current Biology 2012; 22(9): 814-819.
    • 26. Skinner RA, Piek JP. Psychosocial implications of poor motor coordination in children and adolescents. Human movement science 2001; 20(1): 73-94.
    • 27. https://www.photonengine.com. Access date: 19-01-2016
    • 28. Liang H, Chang J, Kazmi IK, Zhang JJ, Jiao P. Puppet Narrator: utilizing motion sensing technology in storytelling for young children. In Games and Virtual Worlds for Serious Applications (VS-Games), 2015 7th International Conference on, IEEE 2015; 1-8.
    • 29. Juanes JA, Gómez JJ, Peguero PD, Lagándara JG, Ruisoto P. Analysis of the oculus rift device as a technological resource in medical training through clinical practice. In Proceedings of the 3rd International Conference on Technological Ecosystems for Enhancing Multiculturality, ACM 2015; 19-23.
    • 30. Davis S, Nesbitt K, Nalivaiko E. A Systematic Review of Cybersickness. In Proceedings of 2014 Conference on Interactive Entertainment - IE2014, ACM Press 2014; 1-9.
    • 31. Yu C, Ballard DH. Understanding human behaviors based on eye-head-hand coordination. In Biologically Motivated Computer Vision, Springer Berlin Heidelberg 2002; 611-619.
    • 32. Pelz J, Hayhoe M, Loeber R. The coordination of eye, head, and hand movements in a natural task. Experimental Brain Research 2001; 139(3): 266-277.
    • 33. Arora, N., Agarwal, N., Reddy, S.R.N. FunPi: An Interactive Learning Experience Using Story Narration. In Proceedings of the Sixth International Conference on Computer and Communication Technology 2015, ACM 2015; 398-402.
    • 34. Song J, Cho S, Baek SY, Lee K, Bang H. GaFinC: Gaze and Finger Control interface for 3D model manipulation in CAD application. Computer-Aided Design 2014; 46: 239-245.
    • 35. Bolton J, Lambert M, Lirette D, Unsworth B. PaperDude: a virtual reality cycling exergame. In CHI'14 Extended Abstracts on Human Factors in Computing Systems, ACM 2014; 475-478.
    • 36. Cheng LP, Lühne P, Lopes P, Sterz C, Baudisch P. Haptic turk: a motion platform based on people. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems, ACM 2014; 3463-3472.
    • 37. Rheiner M. Birdly an Attempt to Fly. In ACM SIGGRAPH 2014 Emerging Technologies, ACM 2014; 3.
    • 38. Sweetser P, Wyeth P. GameFlow: a Model for Evaluating Player Enjoyment in Games. Computers in Entertainment (CIE) 2005; 3(3): 3-3.
    • 39. Ijsselsteijn W, Kort YD, Bellotti F. Characterising and Measuring User Experiences in Digital Games. In International conference on advances in computer entertainment technology 2007; 2: 27.
    • 40. Tan CT, Leong TW, Shen S, Dubravs C, Si C. Exploring Gameplay Experiences on the Oculus Rift. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play, ACM 2015; 253-263.
    • 41. Jennett C, Cox AL, Cairns P, Dhoparee S, Epps A, Tijs T, Walton A. Measuring and defining the experience of immersion in games. International journal of human-computer studies 2008; 66(9): 641-661.
    • 45. Santos BS, Dias P, Pimentel A, Baggerman JW, Ferreira C, Silva S, Madeira J. Head-mounted display versus desktop for 3D navigation in virtual reality: a user study. Multimedia Tools and Applications 2009; 41(1): 161-181.
  • No related research data.
  • No similar publications.

Share - Bookmark

Funded by projects

  • EC | ANIM

Cite this article