Sentiment analysis of speech prosody for dialogue adaptation in a diet suggestion program

In recent years, programs have been developed which allow robots to engage in simple dialogues with hospital and aged-care patients in order to provide information on and give healthrelated advice. To enable the robot to be persuasive and be accepted by the patient, it must not only understand their responses, but also understand their emotional state [1]. This information can then be used to modify the robot's responses. In an example dialogue, the robot asks whether the patient believes they overeat, to which the patient might respond "I don't overeat". If the patient has responded in a negative emotional tone, this may indicate a refusal to acknowledge the problem rather than the absence of it. In addition, the robot needs to learn to avoid responses which may provoke the patient. At that stage the goal of the robot is to convince the patient to acknowledge the problem before developing ways to solve it.

One method of collecting data about patients' emotional state is to analyze prosodic features of their speech. Prosodic features are the patterns of frequency, energy (volume), and rate of speech. Prosodic features have been known for a long time to reflect the speaker's emotional state, as was first documented by Charles Darwin in The Descent of Man [2], which also showed that, even in other animals whose vocalizations contain no linguistic properties, feelings can be expressed.

The main motivation for the development of this software is to improve upon a diet-suggestion dialogue system currently being developed and tested in aged-care homes.1 The elderly subject engages in dialogue with a health care robot, which provides suggestions to that person's diet, whilst also raising their motivation levels, and improve their perception of the robotic agent.