HCM - Haptic support of Cognitive Mapping

A multisensory (haptic, auditory) Force-Feedback-based environment for supporting blind people's cognitive mapping of real spaces, and their acquisition of spatial orientation and navigation skills.

Introduction
The ability to navigate in space independently, safely and efficiently is a combined product of motor, sensory and cognitive skills. Normal exercise of this ability has a direct influence on an individual's quality of life. Mental mapping of different spaces, and of the possible paths for navigating them, is essential to the development of efficient orientation and mobility skills. Most of the information required for this mental mapping is gathered through the visual channel. Blind people lack this crucial information. Consequently, they need a supply of appropriate information through compensatory channels at two main levels: perceptual and conceptual.
At the perceptual level, the deficiency in the visual channel should be compensated with information perceived via other senses. Touch and hearing become powerful information suppliers about known as well as unknown environments. Tactile information is commonly supplied by the cane for low-resolution scanning of the immediate surroundings, by palms and fingers for fine recognition of objects' form, textures, and location, and by the legs regarding surface information. The auditory channel supplies complementary information about events, the presence of other people (or machines or animals) in the environment, materials which objects are made of, or estimates of distances within a space.
As for the conceptual level, the focus is on supporting the development of appropriate strategies for an efficient mapping of the space and the generation of navigation paths.
Advanced computer technology offers new possibilities for supporting visually impaired people's acquisition of orientation and mobility skills, by compensating for the deficiencies of the impaired channel. Research on the implementation of haptic technologies within virtual navigation environments reports on its potential for supporting rehabilitation training.

Research goals
Two main goals guide this investigation:
1. Obtaining an understanding of the blind user's ability to construct a cognitive map by using a multisensory virtual environment.
2. Relating the cognitive map that the user constructed in the virtual environment to his ability to navigate in the real environment.

Research questions
Three central questions frame this research:

1. What characterizes a blind person's process of exploring an unknown environment following the experience with a multisensory virtual environment?
2. Does the walking in the virtual environment contribute to the construction of a cognitive map?
3. What are the user's abilities in navigating the virtual environment and the real environment, by using their cognitive map?
In addition to these main questions, we pose additional questions, which are related to the construction of the cognitive map and its application in a real environment.

The virtual environment
For this study we developed an adaptive multisensory virtual environment simulating real-life spaces. This virtual environment comprises two modes of operation: the Developer/Teacher mode, and the Learning mode.
The core component of the developer mode is the virtual environment editor. This module includes three tools: (a) a 3D environment builder; (b) a force feedback output editor; (c) an audio feedback editor.
The learning mode is the environment within which the user operates by using the force feedback joystick.

Materials and Methods
Materials
The study includes four main research tools: the environment within which the research takes place (a virtual and a real environment); tasks (exploration and recognition of the environment, a target-object task, a prespective-taking task); data collecting tools (computer's recording data, observations, interviews and physical models); evaluation tools (Orientation and Mobility, familiarization of an unknown environment, cognitive map construction, orientation tasks).
Subjects
31 subjects particpated in this research. They were selected based on the following seven criteria: total blindness; at least 12 years old; not multi-handicapped; have been in a rehabilitation program regarding orientation and mobility; Hebrew speakers; onset of blindness is at least two years prior to the experimental period; comfortable use of the computer or subjects that express willingness to use a computer. We defined two groups that were similar in age, gender and age of vision loss (congenitally blind or late blind):
1. The experimental group included 21 subjects. This group experimented with and explored the environment using the multisensory virtual environment.
2. The control group included 10 subjects. This group experimented with and explored the real environment.

Procedure
The study was carried out in five stages:
(a) Evaluation of orientation and mobility (all participants).
(b) Familiarization with the virtual environment (experimental group).
(c) Familiarization with the model's components (all participants).
(d) Tasks - For the exploration task the experimental group explored the virtual environment and the control group explored the real environment. Orientation tasks took place in the real environment for all the participants.
(e) Processing and analysis of the data.

Research Findings and Conclusions

Main Findings

Walking Patterns in the Virtual Environment - In order to get acquainted with the space, most subjects of both research groups used walking strategies that serve them in real space. When comparing findings of both research groups, 4 significant differences out of 7 evaluation variables were found. For example, most subjects in the experimental group first investigated the inner space, then the outer perimeter. They lengthened their walking path and took breaks 3 times more often than participants of the control group. In addition, a number of subjects created novel walking strategies executable only in the virtual environment and based on strategies used in real space.

Construction of Cognitive Maps as a result of Performance in the Virtual Environment - There were 23 evaluation variables. In 7 variables there were significant differences between the two research groups, in 5 other variables there was a tendency of higher evaluation scores within the experimental group subjects, while in 7 other variables no differences were found. These data demonstrated that the information resolution of the components of the cognitive map built by subjects of the experimental group was finer in details than the map built by subjects of the control group.

Performance of Orientation Tasks as a Result of Performance in the Virtual Environment - Most subjects of the experimental group successfully performed the orientation task in the virtual environment. About half of them performed while choosing a straight walking path and the "Object to Object" strategy. When examining the task of changing the target point in real space, most subjects of the experimental group successfully performed the task and more than half of them used the "Perimeter" strategy. Comparing findings of both research groups indicates a greater capability of subjects of the experimental group in performing the tasks.
Main Conclusions

Walking Patterns in the Virtual Environment - Walking in the virtual environment gave the blind person a stimulating, comprehensive and thorough acquaintance with the target space. The high degree of compatibility between the virtual system and the characteristics of the real space contributed to the user's sense of relaxed and safe walking . These conditions enabled the walker to perform walking strategies in a different manner and mass than the ones performed in the real world. Most of the experimental group subjects used the walking strategies that serve them in real space, but with modifications. They preferred to first explore the inner space and to use a wide variety of walking strategies with greater rotation between them when getting acquainted with the space. In addition, they developed new walking strategies, possible only in the virtual environment.

Construction of Cognitive Maps as a Result of Performance in the Virtual Environment - As a result of the intensive interaction with the components of the virtual environment, the users were exposed to a wide variety of feedbacks based on haptic and audio feedbacks. The information received allowed the user to devote all his or her attention and resources to the collection of spatial information alone, without a need to deal with recognition of the shape and function of the environment's components. On the other hand, the virtual environment did not supply measuring tools that could assist in estimating size and distance, nor did the user develop these tools. It is possibly that users of the virtual environment tended to describe the space using a description from the perimeter to the inner space, in a whole and holistic manner, because of the characteristics of the virtual environment.

Performance of Orientation Tasks as a Result of Performance in the Virtual Environment - The first real space walking experience of most subjects in the experimental group was a confident and resolved one. It was noticeable that this walking was based on spatial knowledge acquired as a result of their acquaintance with the virtual environment. The usefuleness of the virtual environment is further supported by subjects' use of "Object to Object" strategy. This strategy can be a factor indicating the quality of the cognitive map that was built, as well as the confidence of the subject in applying it in space. In the spatial manipulation task subjects utilized the "Perimeter" strategy, which depends more on space's components and choosing a walking path that passes next to the target point used in former tasks, in order to create a meaningful reference point for him or her in space.