Project Plan – Australian Sign Language Interpreter Interface

Table of Contents

Introduction – The Problem	2

Research	2

Development	3

Cost Accountability	3

Timetable	4

Gantt Chart...........................................................................................................................5

Introduction – The Problem

Sign language is not an integral part in communication learning of most Australian citizens. This forms a language barrier for those who must use sign language to convey messages to others; an especially difficult case are deaf persons talking to blind.

With advances in technology comes the ability to help overcome these problems.

When developing any form of sensor equipment to detect hand, body and head motion, there are several key factors to keep in mind. Such equipment should be portable, lightweight, durable and most importantly effective.

This project will only concentrate on the creation of any devices needed for calculating movement as defined by Australian Sign Language. While complete translation from movement into proper English forms is not a goal of this project, some work will be required to prove the validity of the design.

Research

Motion capture systems are commonly employed in digital character animation – typically used in areas such as movie post-production and computer game content creation. Technology used in these systems often revolves around the usage of multiple stationary cameras tracking lighted points on an actor's body. While this can be used to interpret sign language, the necessity of fixed camera locations and lighting control makes it unusable as a portable interpretation device.

Another alternative is the use of gloves and other items worn, each which contains attached sensors that can be used to calculate the position of the wearer's body. This is not as accurate as proper motion capture systems (as proof of this, simply see what is used in big budget movie making). However, the degree of accuracy provided by gloves may be more than adequate to deal with sign language interpretation – the dependence on subtle movements must be explored first.

Examples of movement to keep in mind are:

If the amount of bending of a finger important, or simply that it is bent.

Shaking of the head.

Elbow and shoulder positioning.

Wrist roll.

Also is important to bear in mind the minimum knowledge required to determine a general area of location of the hands. Above the head, in front of, behind, at chest height, all are examples where background information in inverse kinematic algorithms and bone joint movement could be useful.

Research is required into exactly what movement is required for the various sign language elements, and what sensors are needed to detect all of these movements. The key word to keep in mind here is required; redundant sensors may be helpful in measuring with greater accuracy, but they also add to the complexity of the system.

Looking at a sign language dictionary will be of importance with this aspect of the project. A dictionary will show all the movements, and from here it can be determined what are the major points of sign recognition (for example if fingers splayed a little means something different from fingers splayed to a greater extent). What movements require what sensors, and what can be inferred (or assumed) from sensor information is also important (for example a pressure sensor on top of the thumb could show if the thumb is clasped under or over other fingers. This is easier than trying to use bend sensors to calculate this state).

The main research goal is therefore the validation and improvement upon the basic requirements of determining motion, and feasibility of sensor control in these locations.

Development

Make the equipment. Any interfaces to computer terminals may be achieved through incorporating a programmable microchip. It can store sensor values in memory locations, which are easily retrievable through any type of serial connection.

There will be some crossover between the research and development sections. Research into knowing what is needed is vastly different from a hardware implementation. Hence various methods of detecting where a hand is relative to the body can be explored, with two of the major ideas being gyroscopes or using camera type equipment and motion capture notions (for example LED's and a web camera, or an infra-red sensor).

Different methods of detecting bend motion should be explored here in order to determine the most appropriate (taking into account such elements as sensitivity, range of motion, power consumption and durability). While industries do provide equipment for the use of detecting bending (most notably with virtual reality gloves) these may not provide the most elegant solution for sign language; indeed virtual reality gloves are designed with the idea that they will be properly cared for and looked after. This can not be assumed if sign language gloves are for everyday use.

Cost Accountability

While the cost and production of gloves in really an industry and not a research problem, there is still the underlying factor of ensuring sign language gloves can be made so that it is available for everybody. This means sensors should not be designed so they only work with expensive technology. Various microprocessors are available at low prices, and this should therefore not be an issue.

Timetable

Weeks	Tasks
1 – 5	Research what is needed, and theoretical solutions.
1 – 3	Bend and Pressure Sensor research
2 – 5	Wrist Location and Rotation
5 – 10	Development and testing of theoretical ideas.
5 – 8	Bend and Pressure Sensor Testing
8 – 10	Wrist Location and Rotation
10 – 20	Integration and testing.
10 – 15	Microcontroller Integration and Programming
15 – 20	Additional requirements as per final methods chosen.
The Rest	Thesis write-up.

The dates for each week are not given, however week one is taken from the first week of semester, and runs straight from then. It is expected (both by the “geekiness factor” and time constraints) that any official non-study or non-contact weeks will not be spent idle, but rather in work upon the project.

It should be noted that development and testing of theoretical ideas and integration and testing are similar in nature. Determining if a bend sensor works should not take five weeks, and so once its design is finalised it can be easily reproduced and attached to a glove. Integration and testing will therefore concentrate more upon the programming of the micro-controller and general interface integration.

Programming and debugging software is extremely time consuming – past experience has shown it to be more so than for hardware based sensors.

Weeks eight to ten depends upon the method chosen for final wrist measurement. If LED's and a CCD are used for example, then these two weeks will be more likely used in additional development of the bend sensors along with the beginnings of wrist location and rotation algorithm programming. This is also true of weeks fifteen to twenty.

2004-02-29T04:25:59