CS474: Human Computer Interaction - Modalities - Voice Prompts
Activity Goals
The goals of this activity are:- To identify alternative modalities for human-computer interaction
- To write a program that uses voice prompts for engagement
- To identify signifiers and affordances for a given application and modality
The Activity
Directions
Consider the activity models and answer the questions provided. First reflect on these questions on your own briefly, before discussing and comparing your thoughts with your group. Appoint one member of your group to discuss your findings with the class, and the rest of the group should help that member prepare their response. Answer each question individually from the activity, and compare with your group to prepare for our whole-class discussion. After class, think about the questions in the reflective prompt and respond to those individually in your notebook. Report out on areas of disagreement or items for which you and your group identified alternative approaches. Write down and report out questions you encountered along the way for group discussion.Model 1: Voice Prompts
Questions
- What is the difference between the two versions of this program?
- How might you adapt this code for use in a text-based program you've written in the past?
- What challenges might you anticipate when using a voice approach, particularly with respect to accessibility, and how might you address them?
- What other modalities can you think of?
- How might you indicate to a user that it is time to input a certain value, and indicate what kinds of values are permissible?
- How do you enable the user to to provide input and to understand output at the right time?
Adapted from Dr. Alvin Grissom’s 2020 HCI course
Try It: How Machines “Hear” — Audio Features for Voice Interfaces
Before a voice assistant can recognize words, it must decide whether anyone is speaking at all. This notebook builds a tiny voice activity detector from scratch on synthetic audio (no microphone needed): you’ll compute short-time energy and zero-crossing rate, pick detection thresholds, and read a spectrogram — the representation real speech recognizers start from. It connects directly to the pause_threshold and timeout choices in the code above.
Click the badge to run the notebook in your browser with Google Colab (no installation required), or download the notebook to run it locally with Jupyter.
Explore Further
- Amazon - Alexa Design Guide — a production style guide for voice interaction design; skim the sections on prompts and error handling and notice how many rules exist just to compensate for the missing visual signifiers.
- Nielsen Norman Group - Voice First: Designing Conversations — a written overview of how usability heuristics translate (and fail to translate) to voice-only interfaces.