Skip to page content
Workplace RERC


State of the Science Conference
September 15-16, 2005

Conference Home

AudioMORPH: Automated Auditory Interface Adaptation

Slides On This Page

  1. AudioMORPH: Automated Auditory Interface Adaptation
  2. Access to Graphical User Interfaces in the Workplace
  3. Adapting a Graphical User Interface
  4. AudioMORPH: Automating Customization
  5. AudioMORPH Components
  6. AudioMORPH Process
  7. AudioMORPH Process
  8. Why not Macro Express?
  9. Example Application
  10. Without Customization
  11. Sample Path/Task
  12. AudioMORPH
  13. A JAWS Script
  14. Future Work
  15. Contact Information

Slide 1 of 15

AudioMORPH: Automated Auditory Interface Adaptation

Melody Moore (Director)
Dan Ratanasit (Software Engineer)

GSU BrainLab, Computer Information Systems, Georgia State University


Slide 2 of 15

Access to Graphical User Interfaces in the Workplace

Web based systems are well supported by commercial screen readers…
But what about proprietary, non-web based business applications?


Slide 3 of 15

Adapting a Graphical User Interface

  • Commercial screen readers can be used to navigate any software application, but they may be cumbersome without customization
  • Screen readers can be adapted for custom business applications via scripts
  • Script writing is usually performed by a rehabilitation engineer with a programming background
  • Fully customizing a screen reader for a complex application can take weeks

  • Slide 4 of 15

    AudioMORPH: Automating Customization

    Goals of AudioMORPH:
  • Reduce dependence on outside professional customization
  • Automatically generate screen reader scripts based on real usage of a system
  • Speed adaptation time for auditory interfaces
  • Remove barriers to employment

  • Slide 5 of 15

    AudioMORPH Components

  • Automated Audio Interface Generation
  • Target Domain Scripts

  • Slide 6 of 15

    AudioMORPH Process

  • User: Domain Expert
  • Automation

  • Slide 7 of 15

    AudioMORPH Process

    Image: A flowchart shows how a domain expert and employee interact with a computer application. The domain expert interacts with AudioMORPH, which in turn interacts with the application, scripts, and key mappings. The employee interacts with a screen reader, which interacts with the application, scripts, and key mappings.

    Slide 8 of 15

    Why not Macro Express?

    AudioMORPH can:
  • Add screen-reader specific commands
  • Handle context sensitivity
  • Automatically generate screen reader compatible scripts
  • Generate additional configuration files such as key mappings and documentation files

  • Slide 9 of 15

    Example Application


    Slide 10 of 15

    Without Customization

    Image: The same application is shown. Several points on the window are highlighted. Although the application can be navigated with the use of keyboard controls, such as the tab key for moving to the next field, sometimes for these applications, it is necessary to skip around between various parts of the application screen.


    Slide 11 of 15

    Sample Path/Task

  • Add a new customer to the database
  • Without customization - requires 17 hits of TAB key
  • With customization - can reduce this to 1 shortcut

  • Slide 12 of 15

    AudioMORPH

    Image: A screen capture image of the AudioMORPH interface shows a window with various Path Controls (Start Path, Rename Path, View Path, etc.), boxes for assigning a name and keystroke shortcut, and buttons to start and stop recording of the script.


    Slide 13 of 15

    A JAWS Script

    Image: A screen capture of a script file shows multiple short scripts of about 6 lines each that announce which window a person has selected for some type of media program. Each script includes lines of letters and Alt-character combinations, in brackets, that represent various JAWS or Windows keyboard commands. A SayString function is used in each script to provide spoken output of which window was selected


    Slide 14 of 15

    Future Work

  • Currently completing AudioMORPH prototype
  • Fall 05 – perform extensive study in real work scenarios with domain experts
  • Test usability of resulting scripts with workers with visual disabilities
  • Compare scripts to those generated by rehabilitation engineers

  • Slide 15 of 15

    Contact Information

    Melody Moore
    melody@gsu.edu
    404-463-7150

    Dan Ratanasit
    dratanas@cis.gsu.edu
    404-463-7121

    www.cis.gsu.edu/brainlab



    Top of Page