Department of Computer Science
Central Washington University
400 East University Way
Ellensburg, WA 98926-7520
Department of Computer Science
Central Washington University
400 East University Way
Ellensburg, WA 98926-7520
June 4, 2003
Department of Computer Science
Central Washington University
If you were asked how people communicate with one another, your first answer would probably be through speech. What if you were unable to speak? How would you communicate then? You would probably communicate by using sign language. However, if you could not speak and you lacked the motor skills necessary to sign, it becomes extremely difficult to find ways of expressing yourself.
This is a serious problem and is the basis for our research. As computer science majors, our CS 480-1 team was approached by local community members, Marcia and Paul [1], to create an augmentative alternative communication (AAC) software tool for their son Ron. Ron was in a terrible car accident about seven years ago shortly after serving in the U.S. Army. As a result of this accident, Ron now suffers from a traumatic brain injury (TBI), which has greatly impaired both his ability to speak and his motor skills. As you can imagine, this event has had a devastating impact on both Ron and his family. The most impacted is Ron who, because he cannot clearly communicate, has lost a great deal of control in his everyday decisions and independence.
Ron's current method of communication involves pointing to hand drawn images taped to his wheelchair tray. Although this is somewhat effective, it is not the most efficient solution. However, to buy a generic AAC software program for Ron would cost our clients between $600 and $2000. Given the cost of taking care of a disabled adult, our clients can not afford to spend this amount of money on a software program that would not necessarily be beneficial to Ron because of his limited motor skills. Hence, we proposed our software solution, which we call Enabler.
So what is AAC? An accepted clinical definition for AAC is defined below.
AAC is an area of clinical and educational practice that attempts to compensate, temporarily or permanently, for the impairment and disability patterns of individuals with severe communication disorders (Asha, 1989).
An AAC software tool, then, would be the device that one uses as an alternative communication method. Our team's plan was to research current AAC tools available on the market and create one that would be customized to Ron's specific capabilities.
After researching several products and discussing each with our clients, we decided we liked the look and feel of Prentke Romich's Springboard shown in Appendix A. This AAC tool is a program that displays a textbox and several image icons that have an associated phrase. When the user selects one of these icons, the complete phrase will be transferred to the textbox and the system will speak the phrase. This product seemed to be exactly what we were looking for. However, as will be discussed later in this paper, the user interface would need to be greatly altered in order to suit Ron's specific disabilities.
A great amount of time and effort was spent investigating what kinds of phrases Ron should be able to communicate. We again turned to the products available on the market to get a sense of what phrases were most important for people to say. We also spent a great deal of time collaborating with our clients on these decisions since they would best know what Ron would want to communicate.
This portion of our research was the most time consuming and during this period, major changes were made to our project specifications after examining how Ron interacted with our initial prototype. Below is a complete examination of both our initial prototype and the altered prototype, which reflects our current working program.
After approximately two weeks after meeting our clients, our team came up with a low fidelity prototype. This prototype demonstrated the four main activities that our software would provide. These activities included (1) speaking a common phrase, (2) building a sentence, (3) using a screen keyboard, and (4) asking a specific question. Each of these activities is discussed below.
Common Phrase. If the Common Phrases activity is chosen, Ron would be given four categories of common phrases to choose from: "I want ...", "I feel ...", Social, and Questions. Within each of these categories, Ron could choose from 10-15 phrases. For example, if the user chose "I feel ...", a new screen would display with several image icons, such as a sad face with the words "Sad" under it. When the user selects the icon, the system would speak that phrase: "I feel sad."
Sentence Builder. This activity would allow Ron to create his own sentence given a set of predefined words. For example, there would be several words available such as "I", "want", "to", and "play." These words could be combined as needed to form sentences. This module was to be thought of as one level up in difficulty from the Common Phrases module and could be used after Ron improved upon his motor skills.
Screen Keyboard. This activity would allow Ron to type freely using a mouse-hover event driven screen keyboard. This module would be thought of as the highest level of difficulty.
Question & Answer. Since Ron is disabled, our clients wanted Enabler to be designed to work with an assistant. The assistant could help Ron do the things that he can not, such as turn the computer on or begin the program. An assistant would also ensure that Ron learns and understands how to use and navigate within Enabler. The Question and Answer module would then allow the assistant to ask a specific question directed towards Ron. An example of this could be to ask Ron "Where do you hurt?" The assistant would also provide answer options for Ron to respond to such as "Head", "Back", "Leg", or "None of These."
Our team realized that this first prototype had major problems when we observed how Ron was interacting with the computer. Marcia and Paul were also concerned that the program was too difficult for Ron to use because of his motor skills and hand-eye coordination.
We observed that because there was not enough "white space" between selectable objects, Ron was having a hard time navigating through our program without accidentally selecting multiple objects. An example of this is the "I feel ..." module discussed above. Fifteen selectable phrases were too many options for Ron. We had to cut it down to six in order for the icons to be properly spaced. This also helped with Ron's poor eyesight. With more icons on the screen, the images had to be small in order to fit everything on the screen. Moving to only six icons on a screen allowed use to increase the size to make them more easily viewable for Ron.
After this meeting, we also re-thought the existing activities available to Ron. Because of the navigation problem, we had to reevaluate the importance of the Sentence Builder. If we could only give Ron a few words to choose from, how useful would this activity be? In addition to this problem, we also reevaluated the Question and Answer activity. This activity was specifically for the assistant. Therefore, we did not want to make it one of Ron's selectable activities. It should only be available to the assistant with a keyboard press.
We wanted at least four activities for Ron to choose from to stay consistent with the rest of our design. This meant that our team had to come up with two new activities to replace the Sentence Builder and the Question and Answer module. These new modules are discussed below.
Read a Short Story. When Ron chooses this activity, he will choose from 4 different short stories. These stories will use text to speech technologies so that the system will read each story page. Ron can read along until the entire page is read and then the system will prompt him to "turn the page."
Word Games. We added a word game feature so that Ron could work on his motor skills. This page displays eight letters and an image. The object of the game is for Ron to choose the right descriptive word of the image by choosing from eight letters given to him. Once he has chosen the letters, he will hover over the Submit button. If he spells the word, he is congratulated; otherwise, he can try again.
Due to the unique nature of this project, the user interface for our program is graphics and multimedia intensive and has a simplified look and feel. Also, as mentioned, this design was established to accommodate Ron’s disabilities. We will describe these design standards and the reasons they are appropriate in more detail below.
Ron is unable to click mouse buttons. Because Ron has limited motor skills, he is unable to click mouse buttons. our team created a special mouse-box housing unit that allows Ron to easily control movement, while protecting the mouse buttons from being pressed. See Appendix B for a picture of the mouse-box. We decided on this design after much research and experimentation with other input devices and after other designs had failed.
Ron will select all objects with mouse-hover events. Since Ron’s mouse movements can be erratic at times, it is critical that the mouse hover time be set accurately. By default, Ron can select an object by hovering over it for approximately 2 seconds.
Ron has poor eyesight. All selectable items, text, and images will have to be large enough to accommodate Ron's poor eyesight. All text displays in an Arial Black font with a size of no less than 20 point. The program also uses a large custom mouse cursor for easy viewing. In addition, all pictures, text, icons and other items will be displayed uniformly.
We want to provide feedback for Ron with every action. Since we are unsure how well Ron can see, it was important to incorporate audible, as well as visual, feedback within the program. Therefore, we designed the program so that every event prompted an audible and visual feedback combination. For example, when Ron hovers over an icon, the image’s border is highlighted and a sound clip plays to indicate that he has selected that icon.
We do not want to overwhelm Ron. Bright colors, flashing or fast-moving objects can be distracting and overwhelming to Ron. Therefore, we tried to use only bold, primary colors. Studies have shown that these bold colors help improve the educational environment and enhance learning. In addition, we avoided using anything that was flashy, and designed the screens so that they would not be too busy or crowded.
Since some of our modules require two-person interaction, our program will be designed so that each user can interact with it simultaneously. This will be achieved by allowing Ron to use the mouse and letting an assistant use the keyboard. With this setup, the assistant can provide help to Ron throughout the program, should he require it and will eliminate the need for the assistant to take the mouse away from Ron in order to assist him. It is important that Ron feels as though he is controlling the majority of the application. Therefore, we never want the assistant to have to take control of the mouse in order to provide assistance.
We are unsure of Ron’s current cognitive state. Because we are not sure how cognitive Ron is at this point, all graphics are clear and concise and are associated with the appropriate textual description. We also eliminated controls that allow the user to maximize and minimize the screen and avoided any scrolling.
We do not want to offend Ron with child-like images. It was imperative that we create a program that would be beneficial for Ron to use and was not demeaning to him. During much of our preliminary research, we discovered that the available AAC software tools were very child-like, with childish graphics and options. Therefore, we wanted to incorporate icons that would be useful and have more of an adult appearance. Although we used primarily clip-art images for the icons, we tried to choose ones that were not too “cartoon-like" and would not offend Ron.
Ron will be unable to handle exception messages. Since Ron is unable to recover from errors, it was critical that we developed a quality, error-free product. Therefore, our testing process was extremely rigorous and we minimized the use of error messages in our code so that Ron would not have to deal with these messages.
Once our team determined the content and the critical design features, implementation did not take long. We were fortunate enough to be developing directly on our clients' machine. This eliminated many problems such as system compatibility, or moving our application from a Windows XP development environment to a Windows 98 target environment. It also allowed us to install applications that helped speed development efforts. Hardware compatibility problems were also avoided by developing on the clients' machine. Our team did not need to worry whether or not the application will bog down our clients' computer since we have access to its capabilities.
We developed our program using the C# language in Microsoft's .NET environment. We chose this language for three reasons. First, we all wanted to learn C#, which was designed specifically with the .NET framework in mind. Second, .NET has an extensive object library. Lastly, it integrates nicely with XML documents.
As just mentioned, we chose to work with XML documents, intended to house our various sound and image files. Again, we chose this technology for three reasons. The first reason is because it is easy to work with. Since an XML file is just simple text, we can make adjustments in any text editor. This aspect also meant that we would not have to worry about compatibility or portability in regards to the database. Another reason an XML document made sense was that the .NET framework includes methods to work with the Document Object Model (DOM) that makes parsing through the XML file more manageable. Finally, an XML document made more sense than an Access Database because of the type of information we would store in it. We would not need a powerful relational database for images and sound files.
It is an extremely difficult task to try and assess whether or not our program is something that Ron can use. During our testing process, we were normally at Ron's home for about an hour due to our time constraints. In order for Ron to really be efficient at using our program, or doing any other new task for that matter, he has to practice over and over. Therefore, we never saw Ron use our program efficiently.
Since our project ended this past March, we have contacted Marcia. She has informed us that Enabler has become part of Ron's everyday physical therapy. He can also navigate without the use of the mouse box, which is a great improvement.
1. American Speech-Language-Hearing Association. (1989). Competencies for speech-language pathologists providing services in augmentative communication, Asha, 31(3), 107-110.
2. AbilityHub.com. Ability Hub, Assistive Technology Solutions. Established in 1997. Retrieved on April 27, 2003 http://www.abilityhub.com/aac/index.htm.
Used for a listing of available software to the public. A large portal to other sites including some for alternative input devices.
3. CNS, Traumatic Brain Injury Resource Guide. Retrieved on April 24, 2003 http://www.neuroskills.com/.
This website was used for a lot of general information on people with TBI.
4. Hill, Katya."The AAC Institute: A resource for evidence-based clinical practice." Closing the Gap, Computer Technology in Special Education and Rehabilitation. October/November 2002, Volume 21. Pages 1, 14-15.
We received this newspaper after writing an online company dedicated to AAC technologies. You can get the first one free but then you must subscribe thereafter. A good source of information with lots of advertisement of available software and input devices.
5. Birren, Faber. Light, Color & Environment. Second edition. West Chester, PA, Schiffer Pub., 1988.
Presenting a wealth of data on the biological and psychological effects of color, with detailed recommendations for practical color use, special attention to computer facilities, and a historic review of period styles.
6. Krych, David, Craig S. Persel, Chris H. Persel, & Mark J. Ashley. Working With Behavior Disorders: Strategies for Traumatic Brain Injury Rehabilitation. Academic Press, 1999.
This book gave us a look at how people with TBI begin the rehabilitation process. It may not be of much use but it did provide us more background knowledge on TBI.
7. Augmentative Resources. Last updated on January 23, 2003. Retrieved on April 24, 2003 http://www.augresources.com/.
This was another website used that acted somewhat as a portal to other sites. It also helped with images that we were considering using for the user interface.
[1] Last names have been omitted to protect the privacy of our clients.
Found at http://store.prentrom.com/catalog/prentrom/SB-AEN-IBM.html
