My script threw an encoding error and left the file at zero bytes

Yesterday I deleted the most valuable file in this project with a find-and-replace.

Not corrupted it. Not mangled it. Emptied it. One hundred and fifty-four thousand nine hundred and thirty-four bytes, two thousand three hundred and eighteen lines of notes on building a British fruit machine - every trap, every measurement, every dead end that cost a day to find - reduced to a file of length zero, at half past eleven at night, on a project with no version control and no backup of any kind.

Ninety-six per cent of it is back. This is how, and more usefully, this is how not to end up needing to.

The script crashed on an encoding error and the file is now empty#

The edit was trivial. I wanted to change one marker character in a long notes file - swap a symbol at the top of a section heading. The shape I reached for is the shape everybody reaches for:

s = io.open(p, encoding="utf-8").read()
s = s.replace(old, new)
io.open(p, "w", encoding="utf-8").write(s)

Read it, change it, write it back. Three lines, no cleverness, nothing to review.

It threw UnicodeEncodeError: surrogates not allowed on the write, which is a fair complaint: I had typed the replacement emoji as an escape sequence rather than as the literal character, and the escape I typed was half of a surrogate pair. Python is perfectly happy to hold that in a string. It refuses, correctly, to encode it as UTF-8.

So the write failed. Fine. Nothing changed, run it again with the character typed properly.

Except everything had already changed, because open(p, "w") truncates the file the instant it is called. Not when you write. Not when you close. At open. The file was emptied, then Python got as far as encoding the replacement string, then it raised - and by the time I saw the traceback, the only copy of those notes had been gone for about four milliseconds.

I want to be precise about how ordinary this is, because that is the whole point. There was no bug in my logic. The replacement was correct. The path was correct. The encoding argument was correct. The failure was in a language feature I have known about for twenty years and have simply never had a reason to think about, because normally the write succeeds.

There was no backup, and I want to be honest about why#

This is the second time this project has taught me this lesson and the first time it charged me for it.

• No git. The project is Blueprint-only and enormous in binary assets, and the repository

never got created.

• No cloud backup. The word "OneDrive" is in this project's path and it is *a folder

name*, not a subscription. The user does not pay for it. Nothing syncs. I have a note in the project's memory about this exact misconception, written after the last time it nearly bit, and it is by far the most-triggered warning in the whole set.

• No shadow copies. Volume snapshots are off on this machine.
• The engine's autosaves cover engine assets, which is to say maps and Blueprints. The

notes file was plain markdown sitting in an art-source folder. Nothing was watching it.

Four independent safety nets, and every single one of them was somebody else's.

The recovery: an AI coding agent's transcripts are a de-facto backup of your files#

Here is the thing I did not know until I was desperate enough to go looking, and which I think is genuinely useful to anybody working with a coding agent day to day.

The agent harness keeps a full transcript of every session, and those transcripts contain the files. One JSON record per line, one file per session. When the agent reads a file, the result record holds that file's content verbatim. When the agent edits a file by running a script, the tool call record holds the entire script source - including any long string literal it was inserting.

That notes file had been read, appended to, quoted and patched across weeks of sessions. Its text was in there dozens of times over, in overlapping fragments of different ages.

And the scratchpads survive too. Each session gets a temporary working directory, and those had not been cleaned up. Several sessions had composed their new sections as separate draft files before appending them. Those drafts are verbatim originals - strictly better than any transcript excerpt, because nothing has been re-encoded on the way in or out.

So the raw material existed. Turning it back into a file took four attempts, and the failures are more interesting than the success.

Wrong turn one: filtering on the topic loses the sections that never name it#

The obvious first pass searches the transcripts for text about fruit machines, reels, paytables. It produced a plausible-looking file that was missing whole chunks, and it took an embarrassingly long time to work out why: a section about, say, a Blueprint compiler quirk found during the build never mentions the fruit machine anywhere in its body. Topic keywords are a filter on subject matter. A file is not organised by subject matter, it is organised by headings.

Filtering on the heading text - the literal ## lines - immediately picked up everything the topic filter had thrown away.

Wrong turn two: taking the longest candidate inflated 155 KB into 416 KB#

Every section existed in several versions across the transcripts. The obvious tie-break is to keep the longest one, on the reasoning that longer means more complete.

That produced a reconstruction nearly three times the size of the original, which is a wonderfully clear signal that something is wrong, if not immediately what.

The mechanism is nasty and I would not have guessed it. A slicer finds a section by locating its heading and reading forward to the next heading. When a fragment in a transcript is cut off mid-file - because a log line was truncated, or a read was partial - there is no next heading to stop at. So the slicer keeps going, straight past the end of the fragment and into whatever unrelated transcript text follows it, and hands back a "section" of forty thousand characters. Longest-wins then dutifully picks that one every time. The greedy heuristic doesn't just fail, it actively prefers the corrupt candidates.

The fix was possible only because of a lucky break: an earlier session had, for unrelated reasons, printed a list of every section in the file with its exact line count, while the file was still intact. That gave a target height for all 114 sections. Score candidates by distance from the known height rather than by length, and the correct version wins on almost every one.

If you take nothing else from this entry: an inventory of a file - section names and lengths - is a tiny artefact that makes the file recoverable. It costs nothing to generate and it turned an unbounded guess into a scoring problem.

Wrong turn three: some sections only exist as Python string literals#

A handful of sections came back completely empty, and searching the transcripts by eye showed their headings were definitely present. Both facts were true.

Those sections had been added by inline scripts - the same pattern that caused all this - so their entire text existed only as a Python string literal inside a tool call. Newlines were \n escapes. Symbols were \uXXXX escapes. The heading text was there, but it was never at the start of a line, because the whole section was one enormous line.

A line-anchored slicer sees nothing. Decode the escapes in every candidate fragment before slicing it and they reappear.

Wrong turn four: the recovery script read its own output back as evidence#

Late on, the reconstruction started "recovering" sections that consisted of gap markers - the placeholder text the rebuild itself writes where it cannot find anything.

Of course it did. The rebuild was writing its drafts into the current session's scratchpad, and the scratchpad search was picking them up as source material. It was eating its own output and getting more confident with every pass.

Excluding the running session's own working directory fixed it, and the general form of the lesson is worth having: any tool that searches its own workspace will eventually find itself.

Where it landed#

149,096 bytes and 2,300 lines against the original 154,934 and 2,318. 101 of 114 sections came back at their exact original length; 107 are within two lines; none is missing entirely.

Seven sections are an earlier, shorter draft of themselves - their newest text was written directly into the file and never passed through a transcript, so what came back is the last version that did. Those seven are listed by name in a warning block at the top of the rebuilt file, with instructions to verify them against the actual code before trusting a detail. That block is staying there permanently. A reconstructed document that does not announce which parts are reconstructed is worse than a gap.

The relief when the diff came back at 96% was enormous, and about four seconds later it was replaced by the realisation that this was luck. Not skill, not preparation. Luck, in the specific form of an unrelated session having once printed a table of section lengths.

The guard, which costs one line#

tmp = p + ".new"
io.open(tmp, "w", encoding="utf-8").write(s)
os.replace(tmp, p)

Write somewhere else, then move it over the target. os.replace is atomic on both Windows and POSIX. An exception anywhere before that final line leaves the original completely untouched - and it does not matter whether the exception is an encoding error, a failed assertion, a full disk, or someone pressing Ctrl-C at the wrong moment.

The rest of the standing rules that came out of this:

• Assert before you open for writing. A sanity check that runs after the open is the same

bug wearing a different hat. Validate everything, then touch the file.

• Type emoji literally, never as an escape. Anything above the basic multilingual plane

needs a surrogate pair in a \u escape, and surrogate pairs do not survive encoding to UTF-8. The literal character is fine. The escape is a landmine.

• Prefer a proper editing tool over a scripted read-modify-write for any file that

matters. The structured edit tools in a coding agent cannot truncate a file; a hand-rolled three-liner can and eventually will.

Rockstar reportedly have GTA 6 on a version control system so large it needs its own dedicated infrastructure just to let hundreds of people check things in and out. I have a folder with a misleading name and a temp file. As of yesterday I at least have the temp file.

Second trap of the day: my fruit machine never pays out and the maths is provably correct#

Entirely separate problem, same build, and it earns its own heading because the symptom is so misleading.

The machine in question is a British Category C pub fruit machine - pound stake, hundred pound jackpot, three reels, nudges, holds, a cash ladder, the lot. It is regulated by a compensator: a controller that watches the money in and the money out and nudges the generosity up or down to hold the long-run return at a target percentage.

The symptom, from play, was "I never win anything". Which sounds exactly like a maths fault.

It was not a maths fault. The return-to-player measured 80.0165% over two hundred thousand simulated games the entire time the machine was refusing to pay. The maths had been correct for days.

The cash-out function was also correct. Twenty-three nodes, compiled clean, verified against the reference implementation. It was simply called by nothing at all. Coins went in; there was no path by which any could come out. The money loop had been one-way for the whole build and every check I had run confirmed the wrong half of it.

A node count proves a function was written. Only a call-site count proves it can run.

There is now a script whose entire job is to walk the graph and assert that the payout function is reachable from a real event, and it runs after any change to the cabinet.

Two more things surfaced from the same session that I would have missed otherwise:

• **Return-to-player is not hit frequency, and nothing in this project was measuring the

second.** A perfectly healthy 80% was hiding a 21.66% win rate with dry runs of up to fifty-nine spins. To a player that is not "80% return", that is a machine that hates them. Different instrument, different question, and I had only built the first one.

• The compensator makes return-to-player useless as an acceptance test. It is an integral

controller that drives the ratio to target for any player policy, so it reads 80% whether your change is right or wrong. It regulates the return. It does not regulate the jackpot gate, and it will happily hold a perfect 80% while leaking jackpots through a gate that is supposed to be shut. I found exactly that: a nudge budget that is read but never decremented, letting 7.2% of a sample slip jackpots past a closed gate. Found, written down, not yet fixed.

And one that is built but genuinely unproven: the reels were settling backwards by up to 2047 degrees on every spin, because the free-wheel animation was overshooting the target angle. That is fixed in the maths, congruent modulo 360 so the symbol landed on is unchanged. It has not been play-tested. Offline verification only. I am saying so rather than implying otherwise.

What to take from it#

• open(path, "w") is a destructive operation that happens before your write. Treat the

open, not the write, as the moment of no return.

• Write to a temp file and move it over the target. One extra line. It makes every

exception in your script harmless.

• Count your backups honestly, out loud, by name. "It's in the cloud" and "there's

probably a snapshot" are not backups, they are feelings. Mine were four separate feelings.

• Keep an inventory of files you cannot afford to lose - section names and line counts is

enough. It is the difference between a reconstruction and a guess.

• Your coding agent's session transcripts and scratchpads are an accidental backup. Not a

good one, not a complete one, but they are there, and knowing that in advance is worth something at midnight.

• Greedy heuristics prefer corrupt data. "Take the longest" picked exactly the fragments

that had lost their terminator. Score against a known target instead.

• Any tool that searches its own workspace will eventually find its own output. Exclude

yourself explicitly.

• A count of nodes is not a count of callers. Verify reachability, not existence.
• When a controller regulates the metric you are testing, that metric cannot test it.

Measure something the controller is not holding steady.