Designing Emotion: Design Findings from Building Easy Speech AAC

Abstract

This project explores how an AAC platform can improve emotional expression and routine engagement by visualizing logged emotional data from users and caregivers. The findings suggest that accessibility is enhanced when design prioritizes emotional readability—such as routine clarity and low-cognitive-load cues—as much as verbal output.

Research Question

How can HCI and embedded behavioral tracking improve emotional expressiveness and routine engagement for nonverbal or neurodivergent AAC users?

Background & Motivation

Existing AAC tools often require costly add-ons or fragmented systems, so I designed behavior tracking to be embedded directly into daily communication workflows. Easy Speech AAC was developed to address these gaps by providing an integrated platform for communication, visual scheduling, and mood analytics.

Methodology

Data was collected through real-world use by AAC users and caregivers over several weeks, focusing on recurring interaction patterns. This approach emphasized observing how the app was used in daily life rather than in a controlled lab setting. The process involved:

• Designing visual-schedule and mood-tracking systems using accessible UX principles.
• Releasing iterative builds for real-world use.
• Collecting qualitative feedback (unsolicited, recurring) from caregivers and AAC users through direct messages, which frequently centered on emotional regulation during transitions.
• Identifying recurring emotional patterns linked to daily activities through the platform's integrated analytics.

Limitations

This work focuses on qualitative patterns and reflects experiences within a limited set of caregiving contexts. Findings should be interpreted as design insights rather than clinical validation. No personally identifiable data was collected or analyzed. This work is not intended to be generalizable to the entire autistic population, as needs and preferences vary widely.

Design Implications

• Engagement improved as the schedule became more cognitively lightweight through clearer task grouping, simplified labels, and simpler visual cues. These refinements supported routine clarity and led to more consistent task engagement.
• Behavior trends revealed emotional cycles tied to activities. For example, a caregiver noted that their child consistently logged "tired" after school on days without a scheduled break, providing a clear, actionable insight.
• Emotional signaling functioned as a core accessibility layer rather than a secondary feature, shaping how users engaged with routines, transitions, and communication.

Outcome & Deliverables

• Released the full AAC platform publicly in 2025.
• Developed a caregiver framework in “How to Track Behavior in Nonverbal Individuals with Autism.”
• Ongoing research includes pattern-informed assistive insights to support caregivers, not clinical diagnosis.

Impact

The platform has been shared within AAC caregiver and professional communities, generating ongoing qualitative feedback and repeated use by families.

Future Direction

Future research will focus on pattern-informed assistive insights that support caregivers, not clinical diagnosis.

Related Work

Research in AAC emphasizes symbol-based communication output, vocabulary organization, and access methods to support expressive language. Many AAC systems focus on message construction, symbol selection, and speech generation, with limited attention to behavioral or emotional context beyond communication needs.

Additionally, work in caregiver-driven behavior tracking systems has demonstrated the value of structured logging for identifying behavioral patterns and communication with professionals. These systems allow caregivers to record contextual information such as triggers and behavior frequency. However, these tools are separate from daily communication interfaces.

Together, these bodies of work establish the importance of communication access, observation, and routine structure, but these elements are typically addressed in isolation. Easy Speech AAC builds on prior research by integrating mood tracking, visual scheduling, and caregiver notes directly into one AAC workflow, allowing emotional and behavioral context to inform both communication and daily routines.