Second Chance GazetteDevlogToken Use

A posed actor reverts to its A-pose after saving, and 29,832 passing tests miss a scoring bug, in Unreal Engine 5.8

unreal-engineblueprintstestingmetahumananimationsilent-failuresnookerpython

The frame that was never close

The scoreboard read 154-153 and it looked like a game.

It was not a game. It was two players being fined seven points each, every time either of them missed, and by the time anybody worked that out the afternoon had gone. Neither of them had made a break worth the name. The scores were level because the penalties were symmetrical.

A lit snooker hall with three full-size tables in a row on a grey tiled floor. The nearest table is part way through a frame - the reds are still roughly bunched but spread at the edges, the blue sits on its spot, and the brown, green and white are down near the baulk end. Blue stacking chairs and a bar counter stand against the right-hand wall, rows of red seating at the far end. A red engine warning across the top left reads that video memory has been exhausted.
The frame the scoreboard was being read off - reds still roughly bunched, the colours down at the baulk end. The red line across the top is the card complaining about video memory, not a fault in the table.

Second Chance has a snooker club in it, the Baulk Line, off a street in a Midlands town. You walk in, you pick a cue up off the rail, you play a frame. That is the argument the whole game is making in miniature: GTA 6 has a lot of buildings and this one has a full-size table built to the official rulebook, with the baulk line and the D in the right place and a rules engine underneath that knows what a re-spot is. Rockstar have a department for a thing like that. James had a Thursday.

The rule in question exists to handle one particular offence - playing at two reds in successive strokes, which is worth seven rather than the four a plain wrong-ball foul would cost. The engine held a flag for it. The flag was named for the last stroke's ball on, and it was being read as though it meant the last stroke had potted a red.

Those two readings agree almost everywhere. They come apart in the single commonest sequence in snooker: a player goes for a red, misses, and the turn passes. The phase stays on reds, so the next player also plays at a red. Two strokes in a row with a red as the ball on, no colour ever potted in between - and the engine solemnly awarded seven points to the opponent for it. Every miss. Both players. All frame.

The clue was sitting in the number. In that phase the ball on is a red, worth one, and a foul cannot go below four. A flat seven cannot be arrived at from those, which means it had come from a clause rather than from arithmetic - and there are only two clauses in the whole rulebook that give a flat seven. A penalty that does not follow from the ball on is a clause to look up, not a number to explain away.

Thirty thousand tests, all of them green, all of them beside the point

The rules engine had been checked against a reference implementation written in Python: 29,832 assertions, whole frames at a time, run with the game not even open. Every one of them passed while this was happening.

They passed because the reference model made the same mistake, in the same words, on line

  1. The gate was differential - it proves the Blueprint agrees with the Python. Nothing

in it ever asked whether the Python agreed with snooker. A differential test can only find a divergence. It cannot find a shared assumption, and a shared assumption is exactly what you get when the same person writes both sides in the same afternoon.

What found it was playing one frame and looking at the scoreboard.

The fix was one clause on one line, in both implementations at once. The part worth keeping is the test written afterwards, because the test was made to fail first: put the old line back, run it, and eight missed reds in a row score 28-21, which is the 154-153 scoreboard reproduced in eleven failing assertions. A test that has never been seen to fail is not evidence of anything. It might simply be incapable of failing, which is what the previous thirty thousand had quietly been.

A cue 45 centimetres above the table, pointing somewhere else entirely

Three things James called out watching the table play itself, and none of them was the snooker.

The first was that the cue was floating. Not drifting - sitting at a fixed height above the cloth, in the air, striking balls it was nowhere near. Measured in play it came to exactly 45 cm, which is exactly the distance the cue is raised as a follow-through on every strike. The cue is put back down by the aiming code, on every frame that anybody is aiming. Nothing was aiming. So it went up on the first shot of the evening and stayed up for the rest of it.

The second was better, because it was a feature working precisely as designed, on the wrong man. A ruling the day before had made the player's own position the coarse aim - walk round the table and the cue swings with you, which is a nice thing to have and it does exactly what it says. It was still doing it. The shot was going down one line and the cue the viewer could see was being pointed down a completely different one, by where a human happened to be standing at the time. Two lines, and a watcher only ever sees one of them.

Fixing that meant putting a pause between choosing the shot and playing it, so the cue comes down and visibly sits on the line first. Which in turn meant stopping the player's aiming code from overwriting it during that second - and the cheapest possible proof that a gate gates is a sentinel: set the variable to 12345, call the thing, and see what comes back. 12345 means blocked. Anything else means it ran. No PIE, no build, four seconds.

The balls were barely moving

The pause shipped broken and he had it inside a minute. The power of the shots seems to have dropped a lot - the balls are barely moving. The log agreed: power 0.0, every stroke.

The cause was in a comment in the project's own code, which the agent had read earlier the same day and not applied. The aiming tick zeroes the power whenever the left mouse button is up. It always had. The old arrangement was safe entirely by accident, because it chose a shot and struck it inside a single frame, so there was no window in which anything could be zeroed. Introduce a one-second pause and two values now have to survive that second - and only one of them had been protected, because only one of them had been thought about.

Take something that ran in one frame and spread it over time, and the job is to enumerate every variable it touches and ask who else writes them. There were two. One needed a gate and the other needed a latch, and reading the tick's own comment about the second one would have saved the round trip.

A hand the maths could not tell was rotated

The rest of the day went on a hand.

James had described the grip in some detail, in the way people describe things they have actually done: hand closed round the cue, elbow dropped slightly, the cue held down at about hip level, the palm against it and facing outward, thumb up and raked a little forward - a bit like a relaxed thumbs-up hand pose - with the shaft running through the gap near the palm rather than out at the fingertips.

A bearded man in a dark waistcoat, white shirt and maroon tie stands behind the baulk end of a snooker table in a dim room, holding a cue upright against his side with his right hand at hip height. The baulk line and the D are marked on the green baize in the foreground, with a radiator and a plug socket on the wall behind him.
The cue at hip height with the hand closed round it. This is the version that passed every numerical check and was still visibly wrong.

None of that was typed in as angles. It was solved: put the numbers in, sweep the joint until the fingertips land on a circle the size of a cue, check the result. The check came back at 0.06 cm root-mean-square, which is a sixth of a millimetre of error per fingertip, and the enclosure controls were opposite-signed exactly as they should be. Every control passed.

James looked at it and said his hand was rotated.

It was, and the reason is worth having. The fit scored the fingertips against a vertical line. A vertical cylinder looks identical from every direction - so every possible wrist rotation scored exactly the same, the sweep had nothing to choose between them, and it returned one. Not a wrong one. An arbitrary one.

A degree of freedom the error function cannot see does not come out unset. It comes out arbitrary, and on a human hand that reads as a broken rig rather than as a gap in the maths. The cure was to state the thing that had actually been described - which way the palm faces - and let the cue's position be the output instead of the input. That is the right way round anyway. It is the hand that has to be anatomically correct.

Then it happened again, immediately, in the same shape: his grip is angled slightly too far forward. The new fit measured distance in one plane only, so a wrist bowed forward scored precisely as well as an upright one, and the bow was now the parameter nothing could see. Two faults, same family, one after the other.

A close side view of the same waistcoated man standing beside a snooker table, seen from his left. His right hand is closed round the cue at hip height with the thumb up and the knuckles near vertical, and the butt of the cue rests on the floor beside his shoe. Behind him are a papered wall, framed pictures and the shaded lights over the further tables.
The hand as it finished - knuckles near vertical, thumb up, the butt of the cue resting on the floor at his shoe.

There were two smaller ones alongside. Fixing the rotation swung the cue into his leg, 0.7 cm off the ankle, which nobody noticed because the clearance test printed the number rather than asserting on it - a number nobody is forced to read is not a check, and it raises now. And the hand had failed to close at all, because the same bug had already been found and fixed once that morning on the other clip: stating which way a bone must point lets a solver reach it by bending the joint backwards, so that clip had been rewritten to curl the joints instead. The standing clip was left on the old form and nobody went back for it. When a fix lands on one of two parallel paths, go and look at the other one.

Correct on screen, correct in the readback, and not on disk

And then the one that cost the session.

The animation was set on the placed man. It read back by name. The save reported success. The editor's list of unsaved changes came back empty. Two screenshots showed him standing there correctly posed, cue in hand. Everything anybody could ask, answered green.

He reverted to a bare reference pose the next time the actor was rebuilt, because none of it had ever left memory.

The call that marks an object as changed had been made on the actor, and the property that was changed lives on a component hanging off that actor. Marking the actor does not cover its components. The tell was inside the saved file itself: it contained the tags - which are an actor property, on the object that had been marked - and no mention whatsoever of the animation, which is a component property on an object that had not been. Everything in memory was genuinely correct, and stayed genuinely correct for exactly as long as anybody was looking at it.

Two men stand at opposite ends of a full-size snooker table in a flatly lit interior, each holding a cue upright at his side. The nearer one wears a dark waistcoat and white shirt, the further one a plain dark t-shirt. Behind them are a counter with a till, a fire extinguisher and a glazed double door onto a sunlit street, with a second table at the left edge of the frame.
Two men at the table with a cue each, held upright at the side. The faces were swapped in the same session and have not been looked at in play.

The placement script now ends by opening the file it has just written and asserting the clip's name appears in the bytes. The readback is not the proof. The file is.

Two men, a cue each, on disk this time. The last screenshot of them went in just before eleven.

The fault

Every stroke that missed a red awarded seven points to the opponent. Both players, all frame, so the scoreboard stayed level and read as a close game - 119-118, then 126-132, then 154-153, with no break of any size behind any of it.

The flag driving the clause meant the last stroke's ball on was a red. It was being read as the last stroke potted a red. Those differ in the commonest sequence in snooker: a player misses a red, the phase stays on reds, the turn passes, and the next player also plays at a red.

Why the gate could not catch it:

  • the 29,832 checks are differential - they prove the Blueprint matches a Python

reference model, nothing more

  • the reference model contained the identical misreading
  • both sides therefore agreed, in detail, thousands of times, on the wrong answer

☞ The value was the clue. In that phase the ball on is worth 1 and a foul floors at 4, so a flat 7 cannot be arithmetic - it has to be a clause.

The fix

Gate every rule against the real world, not against the other implementation

The clause now requires that the striker continued - i.e. that a red was actually potted and a colour is due - rather than merely that a red was the ball on. Corrected in the Blueprint and the Python reference in the same pass, so a rebuild keeps it, and only the one affected function was re-authored rather than all 91.

The guard, and it is the part worth copying: a separate test that checks the rule against snooker, written so that it has been seen to fail. Reverting the single reference line reproduces the original symptom exactly - eight missed reds in a row scoring 28-21, in eleven failing assertions. Every rule with a real-world meaning needs at least one test that does not go through the other implementation.

The fault

A posed actor reverts to its A-pose after a save that reported success

An animation set on a placed character's mesh component never reached disk, while every available instrument said it had:

  • the property read back correctly, by name
  • the save call returned true
  • the editor's dirty-package list came back empty
  • two screenshots showed the character correctly posed

The change existed only in memory, and memory was accurate for as long as anybody inspected it. The character reverted the next time the actor was re-instanced.

The fix

Mark the component, not the actor, and prove it from the saved bytes

modify() on an actor does not cover that actor's components, so a component property edited without it is never recorded for the save.

body.modify()                       # the mesh component, not the actor
cue.static_mesh_component.modify()  # same trap, same shape

The guard: the placement pass now reopens the file it has just written and asserts the clip's name appears in the bytes. A readback re-reads memory and will happily confirm a change that was never recorded; the saved file is the only thing that cannot.

The kit

ToolWhy this one
A Python reference model of the rulesthirty thousand checks a night, with the game shut - and the reason the shared misreading survived so long
One played frame, watchedthe only instrument that found the scoring bug at all
A sentinel value (12345)proves a gate gates, on the class default object, with no PIE and no build
Unreal Engine 5.8where all five faults lived, and where four of them reported success

Plain English

TermWhat it means
BlueprintUnreal's visual scripting. Logic built by wiring boxes together with lines instead of typing code. This whole game is built in it.
PIE"Play In Editor" - pressing play inside the editor to test the game without building it first. Fast, and it behaves subtly differently from the real build.
oracleA second, simpler implementation of the same rules, written only to be argued with. The real one is run against it thousands of times, and wherever the two disagree, one of them is wrong.
negative controlThe opposite of a positive control: deliberately putting the fault back to check that the test you just wrote actually notices. A test that has never been seen to fail is not evidence of anything, because it may simply be incapable of failing.
CDOThe template a Blueprint stamps out copies from. Editing the template does not always reach the copies already placed in the level, which is a classic silent failure.
actorAnything placed in the game world - a lamp post, a person, a trigger volume, a camera. If it sits in the level, it is an actor.
MetaHumanEpic's system for realistic human characters - face, body, hair and clothing, ready to animate. Every named person in the game is one.
reference poseThe neutral stance a character model is built in, before any animation is applied. Seeing it in a finished shot means the animation is not running at all.
dirty WP packageOne square of that World Partition grid holding unsaved changes. "Dirty" just means changed-but-not-saved, and unsaved squares are how a day's work quietly disappears.
tickCode that runs every single frame. Cheap once, ruinous sixty times a second across a thousand actors.

Things we still need to fix tomorrow:

  • The two men have faces on now. Nobody has looked at them in play.
  • Nothing plays the cue clips off the state of the turn, so the men stand still and the

cue moves on its own.

  • The cue model does not follow the hand - it sits where it was placed.
  • With no way to nominate a colour, the engine still charges seven where a referee would

charge whatever the striker had called. That question is recorded and not started.

  • The clip the hand was solved in and the live character disagree by about 15 cm, so every

number solved against the clip is in a frame the game does not reproduce.

  • There is still no video of any of it.

← All devlog entries

Watch it get built. All of this goes up on YouTube as it happens - broken animations, buildings hovering a foot off the ground, the lot.

Subscribe on YouTube