A dancer on the beat, because the beat is a number and not a signal
The nightclub went in today: a station of fourteen tracks that only plays during club hours, a disco rig that reacts to the music, and one NPC on the dancefloor who dances in time with whatever is playing. There will be more dancers eventually. For now there is one, and she is the reference implementation.
The interesting part is not the rig. It is where the timing comes from, because there are two different questions hiding inside "react to the music" and this project answers them from two different places.
Written up after the fact and dated to the day the rig went in. Two follow-on dates are called out where they belong, because the entry would be dishonest without them.
What I was trying to do
A room with music in it is not the same as a room that is playing music. I wanted the club to read as the second thing: lights that move with the track rather than on a loop, colour that changes when the song does, and a body on the floor moving with it. This is a Blueprint-only project driven through the editor's automation servers — no C++, no engine plugin work — so whatever I built had to be cheap enough to run in a Blueprint tick on a machine with 8 GB of video memory.
The two questions
"Is this a loud bit?" is answered live. An envelope follower on the club audio produces a smoothed intensity value, SmoothPulse. The light schedule runs on a 0.04-second timer, buckets that value into three bands — quiet below 0.6, groove above it, crescendo above 1.2 — and maps (fixture group, band, track, beat) to visible or hidden. Quiet shows one mover. Groove brings in the track's palette plus the glow lights. Crescendo kills the movers and flashes a white strobe. The laser only ever runs at crescendo. Per-track colour is a bitmask stored as an integer array — six colours, one entry per track — indexed by the same cursor arithmetic the playlist itself uses to pick the next song, so the palette and the track can never drift apart.
Two details worth stealing. Lights are hidden, not dimmed: hiding an actor genuinely removes the light from the scene, so the show is also the performance strategy. And the intensity is max(live envelope, synthetic cycle), where the synthetic side is a 22-second per-track sawtooth. The radio is player-triggered by construction, so the envelope reads zero whenever nobody is in earshot — without the fallback the whole rig looks dead from across the room, which is exactly the distance you usually see it from. Every music-reactive system needs that fallback.
"When is the next beat?" is not answered live, and this is the whole point. The beat flash is fmod(gametime, 0.428571) < 0.18 — a fixed grid at 140 BPM, which is the tempo the club's tracks were written around, with a 180 ms flash window. It comes from a constant. It never touches the audio.
The trap: trying to get tempo out of the envelope
The symptom you will type into a search box is something like "my music-reactive lights follow the volume but never land on the beat", or "how do I get BPM from an audio envelope at runtime in Unreal".
The honest answer is that you cannot, and the reason is structural rather than a missing node. An envelope follower gives you one scalar per frame: how loud is it now. Tempo is not a property of a moment, it is a property of the intervals between moments — you need seconds of history and an autocorrelation over it before a number falls out. That is not a per-frame Blueprint job on this hardware, and it would be a waste even if it were, because BPM is a fixed property of a recording. Measuring it every frame is spending runtime compute to rediscover a constant.
So: envelope answers dynamics, a measured number answers timing. Different questions, different sources, and mixing them up is the mistake.
Measuring the number offline
The club needed one constant. The pub singer's set, worked on 1 August, needed eight different ones, so that is where the measuring tool came from — a small offline analyser using nothing but numpy, because scipy is not installed on this machine. Export the sound assets to WAV, point the script at the folder, get a table back. The pipeline is: mono, downsample to 11 025 Hz, spectral-flux onset envelope from an STFT with half-wave rectification (energy falling is not a beat), FFT autocorrelation, comb score across 55–200 BPM, then fold the answer into 65–165.
Three cleverer versions of the octave-correction stage each looked like an improvement and each made things worse:
- Taking the mean of the onset envelope over a candidate beat grid. A mean *rewards
sparsity* — a half-tempo grid samples only the strongest beats and skips the weak ones, so its average comes out higher. It halved every single track; a 138.5 BPM number came back as 69.
- Using beat-versus-offbeat contrast as the arbiter. This is genuinely the right
discriminator in principle, and it still overturned tracks the plain autocorrelation had already got right.
- Offering ÷3 and ×1.5 as candidates. This wrecked four correct tracks — one 84 BPM
reading came back as 56. An octave corrector must never be able to invent a tempo that was not in the signal.
What works is the dullest option available: trust the autocorrelation peak, and only fold it into range. That got seven of eight right on the first attempt.
The control that caught the eighth is the part I would keep in any future version. Score the beat grid at the chosen BPM, at 0.93× and at 1.07×. A real tempo beats its own ±7% detune; a spurious one does not. That ratio is reported as a confidence value, and anything at or below 1.0 means do not trust the number. One track failed it — and it turned out to be a track with no correct whole-track answer at all, reading 80 / 162 / 162 across its three thirds because it is a slow number with a double-time middle section. The tool now prints per-third readings so a tempo change is visible instead of silently averaged, and carries an overrides table, because ground truth from the person who chose the music beats any algorithm.
The dancer, and what "in time" actually means here
The dancer is an external controller Blueprint, not logic inside the character — the standard shape in this project. On BeginPlay it finds the dancer by tag, finds the envelope follower and the club's station brain, and caches the body mesh. On tick it reads the game clock for whether the club is open, computes the same max(envelope, synthetic) intensity the lights use, and picks one of three states: standing pose, ready-bounce, or the current dance clip. It only re-plays a clip on a state change — without that guard it restarts the animation every frame and the figure twitches in place. When the station reports a new song it advances a style index and forces one re-play, so each track gets a different dance.
Two things cost real time here.
The first was the dancer looking like she was sitting down while standing up. She is a dressed MetaHuman, and the garments are separate skeletal meshes that do not follow the body on their own and are not compatible with the retargeted body clips. Playing the clip on everything crumples the garments to wireframe. Playing it only on the body leaves the garments frozen in their own idle — a seated one — draped over an animating standing figure. The fix is to set every garment mesh's leader pose component to the body at BeginPlay, then play the clip on the body alone; leader-pose copies evaluated bone transforms, so no clip compatibility is needed. Pick one animation configuration for a dressed MetaHuman and never mix the two.
The second was every light staying hidden even with the deciding helper hardcoded to return true. A non-pure Blueprint function called inside an expression does not get its exec pin threaded, so it silently returns the type default no matter what its body says. The same function called at statement level works fine. Inline the maths or make the helper pure. That one cost an hour and will again.
What is not true yet
Being blunt about the limits, because the headline claim is easy to overstate:
- She is in time because the tracks and her clips were chosen to share one tempo, not
because anything at runtime aligns a clip to a beat. There is no phase lock. Nothing resets the animation on a downbeat. If a track at a different BPM went into the club playlist tomorrow, she would be wrong and nothing would notice.
- The proper upgrade is to drive the animation play rate as
track BPM ÷ clip authored BPM.
That only means anything if the clip loops at an exact beat count, and at least one clip in this project is an arbitrary 4.33 seconds because it inherited an idle's length. Those need re-authoring against a target tempo first. Not done.
- The lights' beat flash is still that hardcoded 140, not a lookup against the measured
number. It predates the measuring tool and has not been upgraded.
- The rig and the dancer were PIE-verified the day they were built, in the original rooftop
club. The whole club was later rebuilt at ground level in the open-world map, and that version has not been PIE-tested during club hours with a player in the room. The music needs both the clock inside the opening window and a pawn within earshot; neither has been observed together there.
- One dancer. Everyone else in the club is still furniture.
What to take from it
- Split "reactive" into dynamics and timing before you build anything. Loudness is a
live signal. Tempo is a stored fact. One source cannot serve both.
- Anything constant per asset should be measured once, offline, and baked. The runtime
cost then collapses to an array lookup when the asset changes.
- A correction stage must never be able to invent a value that was not in the input. It
may fold, clamp or reject. The moment it can multiply by 1.5, it will do so confidently and be wrong.
- Build the self-check that can fail. "Does this answer beat its own slightly-detuned
neighbour" was the only test that found the bad track; everything else agreed it was fine. A confidence number nobody can fail is decoration.
- When a fallback exists for the common viewing condition, use it. A reactive system
driven by a player-gated signal looks broken from the distance you normally see it at.
- Ground truth from the person who chose the content beats the algorithm, and the code
should have a place to write that down rather than being argued with.