The Influence of Human Factors on Subjective Emotion in Voice-based Human-AI Interaction

The paper notes that emotional satisfaction (pleasure) is the top level of Aarron Walter's hierarchy of user needs, and that Kansei (emotional) engineering offers methods to quantify emotion in product use. Voice-based intelligent systems are now common in smartphones and AI speakers, but they have not yet developed into the kind of interaction users want, and many people still prefer conventional input. AI is not yet able to satisfy individual users with diverse characteristics. The authors propose identifying user groups with shared human-factor characteristics and optimizing interaction for each group. The study evaluates whether user groups differing in human factors feel different emotions when interacting with a voice-based intelligent system, and identifies which human factors and design parameters affect emotion.

• Human factors (two levels each), drawn from the authors' earlier studies: gender (male/female); interest in new technology (early adopter vs. majority), classified using Chasm theory and a Korean version of Hurt et al.'s (1977) innovativeness questionnaire; and personality (introversion vs. extroversion), measured with the extraversion items of a Korean version of John and Srivastava's (1999) Big Five questionnaire.
• Participants: 58, grouped by these three factors (Table 2).
• System: a Wizard of Oz voice system (a human operator simulates the AI behind the scenes) with four design parameters at two levels each: voice (child vs. announcer), pitch (normal vs. raised two steps using text-to-speech), answer structure (simple answer vs. answer with its source) and speed (1.0x vs. 1.2x). Combining them gave 16 interaction types.
• Measures and analysis: after each interaction, participants rated 30 emotional adjective pairs on a 7-point scale. Factor analysis grouped the adjectives into emotional factors, and ANOVA tested differences by human factor and design parameter, including interactions between human factors (gender, technology interest) and design parameters (voice, pitch, structure).

• Empirical: factor analysis produced three emotional factors: Stability (e.g., comfortable, cozy, natural, stable, easy to understand), Trustability (e.g., professional, detailed, reliable, secure) and Intimacy (friendly, affectionate, kind, familiar).
• Empirical (human factors): gender affected trustability (p = 0.003), and interest in new technology affected intimacy (p = 0.000). Personality showed no significant effect on any factor (p = 0.434, 0.544, 0.268).
• Empirical (design parameters): voice affected all three factors (p = 0.000 each); answer structure affected stability and trustability (p = 0.000) and intimacy (p = 0.029); pitch affected only trustability (p = 0.000); speed affected none (p = 0.700, 0.272, 0.171).
• Empirical (interaction): for trustability, men showed little difference between announcer and child voices, whereas women strongly preferred the announcer voice, a statistically significant difference (p = 0.007).
• Design implications (authors' interpretation): user personality and answer speed may be excluded from consideration when designing emotionally satisfying intelligent systems, and for women an announcer voice is desirable to secure trust.

The study quantitatively evaluated users' subjective emotions in voice-based human–AI interaction using emotional vocabulary. It concludes that the identified human factors and design parameters can be applied to design intelligent systems customized to individual users. The authors note that this study measured emotion after the interaction using emotional vocabulary, a basic Kansei engineering technique. They plan to analyze interactions between multiple human factors and multiple design parameters, and to analyze emotion in an integrated way using real-time signals such as bio-signals and facial expressions. They position this study as basic research for that integrated emotional evaluation.