CARES

Controlled Audio Reactions in Environmental Scenes

Marcel Gibier

Inria Paris

CARES is a dataset of two-speaker audio scenes in which an environmental sound is placed inside a conversation. Each sound carries a reaction type that the dialogue must realise, and the sound itself is never named. Recovering the type therefore means hearing the sound and reading what the conversation does with it.

Pipeline

The CARES pipeline: themes, template, scenario, dialogue, voice synthesis, mixing, audio scene.
Seven stages, from conversation themes to mixed audio. Everything structural (scene, roles, sounds, reaction types, train and test split) is drawn from a fixed seed before the first model call.

Reaction types

PivotThe sound takes over. It becomes the subject for several turns.
VerbalOne speaker acknowledges it in passing, then the conversation resumes.
BehavioralNobody mentions it, but it visibly disrupts the speaking.
AmbientNobody reacts at all.

Examples

Pivot

beach · watch_alarm at 0:48 · 1 min 35

AIn my notes. On my phone. Which is with me at all times, so.

a watch alarm goes off

BHang on, was that you? Something just went off right by me.

The sound becomes the subject. What they were arguing about stops.

Verbal

cafe restaurant · chair_scraping at 1:12 · 1 min 30

A…I've told Marco he's on potatoes and he's furious about the demotion.

a chair scrapes

BOof, that went right through my back teeth. Sharp. Anyway, poor Marco, twenty years of stuffing rights gone in one phone call.

Noted, then dropped. The word anyway does the work: the subject survives.

Behavioral

airport · security_conveyor_belt at 0:16 · 1 min 32

AManchester. Back Sunday. Actually, funny thing, I've got the Pixies next Friday. Dan sorted it.

a security conveyor belt starts

BWait, next Friday, at the… the big one, the one by the station, it's got the ugly roof, I can never… anyway, yeah, that place.

Never mentioned. It shows only in the way the sentence comes apart and is rebuilt.

Ambient

public park · baby_crying at 0:20 · 1 min 33

AShe sits out on that balcony with her tea and she's already got a list of things to tell me.

a baby starts crying

BA list. Of course she does. And you just let her run through it?

Clearly audible, and the conversation does not bend around it.

Pivot (rare sound)

home indoor · siren_ambulance at 0:33 · 1 min 39

AHe keeps saying he's just being direct. Which, fine, be direct in the huddle. Not while I'm trying to read a hitter.

an ambulance siren passes

BWhoa, that's right outside. That was RIGHT outside!

Rare sounds are always pivots, the one case where the sound outranks the subject.