Minus-two: Multimedia, sound spatialization and 3D representation for cognitively impaired children

Abstract

Multimedia and Hypermedia technologies can be successfully used in education and rehabilitation for cognitively disabled persons. In particular, storytelling has always proved successfully in creating imaginary worlds that could both protect the child from the difficulties of the outer world and create the conditions to feed new stimuli without causing discomfort. However, traditional storytelling techniques (paper, pencil, the voice of the teacher) sometimes might be vastly improved by using multimedia and hypermedia technologies. The possibility to play different music, to explore spatialized sounds, to record sounds and voices, and the opportunity to play with 3D images within a 3D space using devices as simple as touch screens, may ensure an enhanced level of attention from the children and may give to the teacher new and powerful tools to convey his/her message through the common activity. In this paper we motivate the use of hypermedia educational techniques with cognitively impaired children, outlining the general requirements of an educational framework that might be provided to teachers. We then describe the functioning of a prototype we developed following these requirements, MINUS-TWO: a storytelling environment mainly characterized by a strict cooperation between the child and the teacher or the therapist, by multimodal interaction, sound spatialization and 3D representation. The paper also presents the results we obtained by using these techniques with more than twenty cognitively impaired children.