Every voice in the town: ElevenLabs

Why ElevenLabs, and why it never runs in the game

The first voices in the game came from Kokoro, a small open-source speech model that runs on your own machine. Its British voices were approximations at best, and it could not do many of the accents a Midlands town needs. ElevenLabs has a much wider range of British voices and lets you design and tune your own, so it became the default for every human voice. Kokoro is kept for voices that are meant to sound synthetic, such as a machine or an answerphone.

ElevenLabs is a cloud service, so the game never calls it while it is running, and never will. Every line is generated once, during development, and shipped as an ordinary sound file. A line sounds the same every time you hear it, and the game needs no connection to speak.

The baking pipeline

1. Write the lines in a script file, one entry per line, each with an ID such as Charlotte_GiveDrink_01. For big casts the script is generated by code from one source of truth, so the audio can never drift from the written text. Toby's generator also checks that every number he calls has the right colour on the wheel.
2. Keep the voice, model and settings in the script's own settings block, so re-voicing a whole character is a one-line edit.
3. Dry-run it first. The baker prints the character count before it spends anything.
4. Bake. A short Python script, standard library only, sends each line, asks for raw 24 kHz audio, wraps it as a WAV and writes it to the character's folder. It stops at the first failure, so we read the end of its output rather than assume a quiet run worked.
5. Generate outside the game's Content folder, then move the files in. The running Unreal editor imports anything dropped into Content by itself, half-written files included.
6. Level the set. A performance varies by 6 to 10 dB from line to line, and the game plays a character at one volume, so a normalising pass evens the set out after the bake. On Barry it closed the spread from 15 dB to 8.

One pleasant surprise: renaming baked lines is safe. Rename the WAV files on disk and the editor renames the sound assets itself and keeps every reference to them.

Choosing a voice

Audition, and never trust the label. One library voice tagged British read as Australian; James caught it by ear.

Some voices are designed. Toby's is one of James's own, and it already carries the Yorkshire accent. Edward, a regular at the pub, has a voice James made too, slowed to 0.95 speed because Edward is 72 and in no rush.

A designed voice can vanish. Designed voices take a slot on the account, and when the slots were full, James deleted one to make room. It was Charlotte's, and every reference to it died with it. She moved to one of the service's own library voices, which take no slot and cannot disappear that way, and all 94 of her lines were baked again. We now check a stored voice against the live account before every bake.

One voice, two men. Wozza, who drives for the council, and Jack, who plays you at rock paper scissors, share a voice. ElevenLabs has no pitch control, so Wozza is pitch-shifted down after the bake. James auditioned two and three semitones and said two was too much; one semitone is enough to read as an older, heavier man who still sounds like the same actor. The raw bake is archived outside the game, so re-tuning never needs a fresh bake, and the shift script refuses any result whose length changes by more than 20 milliseconds.

Direction: tag the attitude, never the volume

With ElevenLabs' v3 model you can write direction into the text in square brackets, such as [laughs] or [sarcastic]. Attitude comes from the direction, not from the voice: a neutral voice directed well beats a voice that is only described as cheeky. What direction cannot change is age, accent and timbre. Those belong to the voice.

Barry's first bake, 139 lines, came back and James's verdict was that he sounded like he was mumbling. There were two faults inside that one word. The first: v3 acts its tags, so a tag that names a volume or a breath is an instruction to swallow the line, and the model obeys. A line tagged [low, serious] came back at -29.5 dBFS against a median of -18.7 for the set, eleven decibels down and inaudible on James's machine. [quietly], [under his breath] and [half-laughing] all ask for the same thing without looking like it. So the rule is to tag the attitude, never the volume or the breath. [calm, sincere] gets the same performance and can be heard.

The second fault was the settings: low stability with high style is a licence to blur the diction. We settled it by baking three lines in three settings, peak-normalising them all so loudness could not flatter one, and letting James choose. He picked the multilingual v2 model at stability 0.55 and style 0.15, with the tags stripped out, so the direction became a stage direction for the writing instead of model input. His later verdict: Barry does speak well.

Not every model acts tags. The faster turbo model reads them out loud as words, so tags and stage directions are stripped before a turbo bake.

The accent is in the voice, not the spelling. Eye-dialect such as "yer" or "wi'" makes the model stumble or read the punctuation aloud. Toby's lines are spelled normally, and the Yorkshire comes from the words he chooses: nowt, owt, summat, mardy, give over.

Write lines as replies. Edward's lines were written to answer the lines Charlotte already had, so the two play as a conversation instead of two recordings. "Duck", the Nottingham endearment, is in both their mouths.

Sound effects as well as voices

ElevenLabs also makes sound effects from a written description. We use it where a real recording is wrong or cannot be licensed: the first engine idle, start, doors and brake squeal on the player's car; the charger sounds; the slides and thuds of a thrown glass; and two birds, the linnet and the stock dove, for which Freesound had no recording we could use.

Picking by measuring has a trap of its own. Choosing between five generated thuds, the first measurement faded in the start of every file, which is exactly where the knock is, and nearly chose the wrong one. Measured again, one clean mid-range tock won.

One effect came back as raw stereo samples with no header and a misleading file extension. Comparing the average jump between neighbouring samples under each possible layout proved which layout it was, and it was then mixed down to mono, because Unreal will not place a stereo sound in the world. For the car's revs, one idle loop is enough: the game pitches it from 0.75 to 2.2 times with engine speed.

Who is voiced

• Charlotte, the barmaid at the Slag and Slurry: 94 lines, from taking your order to calling the bingo.
• Edward, a regular at the same bar, in a voice James designed.
• Toby, the roulette dealer at Last Tenner Bets: 250 lines, properly mouthy, and every number on the wheel. Yahya, a punter at the same table, has 24.
• Barry, the hustler at the Baulk Line: 139 lines, one picked for each stroke from what happened.
• Denise, the adviser at Bestholme Job Centre, who signs you on: 30 lines.
• Wozza and Jack, one voice between them.