302 tests found six bugs in my trading bot, and no edge

I run an algorithmic trading system. Multi-asset, several strategies, live for months. Until recently it had **zero tests**. It now has 302. They found six real bugs, and they did not make the system profitable — because those are two entirely different claims, and keeping them apart is most of what this post is about. Everything below is **paper trading**. Simulated money, no live capital. That matters for the second half. ## Why does money code end up untested? Not laziness. Three things, and they compound. **Testing it has side effects.** The first thing I discovered was that writing these tests fired seven real circuit-breaker alerts at my own phone — the halt function sends a Telegram message. Another module appends to a live operational log. Another writes rows into the database a running system reads. A test suite that pages you, pollutes your logs and mutates production is a suite you run once and never again. **Nothing forces the issue.** There is no compiler error for "this strategy is subtly wrong." The system ran for months, opened positions, closed them, and reported numbers. Nothing about that experience says *write tests*. **And the numbers looked plausible.** This is the trap. A trading system that is quietly broken does not throw. It produces a P&L, and the P&L looks like a P&L. ## The six bugs Four of the six were **the same shape**: logic written for long positions, applied to shorts. **Every short was rejected before it could open.** The risk manager validated stops with one check — a stop above the entry price is invalid. That is correct for a long and exactly backwards for a short, whose stop belongs above entry. Every short signal was refused as "invalid stop loss." The strategies had been generating them for months. **The exit path could not tell a short from a long.** The position dictionary handed to the exit check simply had no `direction` key, so every position was monitored as a long. Fixing the first bug alone would have opened shorts that closed themselves on the next tick, paying a round trip in fees each time. They had to ship together. **Open shorts were marked backwards.** Portfolio value used the long formula for every position, so a short winning $200 showed the portfolio $200 *down*. That number feeds the circuit breaker and position sizing — a profitable short could trip a drawdown halt, and a losing one could hide a real one. **The correlation gate blocked hedges.** It compared the absolute value of price correlation with no knowledge of direction, so two longs at −0.9 — a textbook hedge — were refused as "too correlated." Concentration is a property of *exposure*, not price. The other two were different but rhyme with the first four: a drawdown halt that said "STOP until manual review" and silently cleared on restart, because the flag was written to disk and never read back. And expired options booked at zero P&L regardless of premium, which recorded **$1,301** of losses as break-even and turned a trade that lost $872 into one that appeared to make $430. ## Why reading the code would never have found them I want to be precise about this, because "write tests" is cheap advice. Every one of these bugs is *invisible at the call site*. The stop check reads as obviously correct until you hold a short in your head. The missing dictionary key is an absence — there is nothing on screen to notice. The mark-to-market line is a plausible multiplication. What found them was not scrutiny. It was **asserting things that have to be true**: - Marking a position to a price, then closing at that price, must give nearly the same portfolio value. - A short whose price falls must be worth more, not less. - Risk taken must never exceed the configured cap. That mark-and-close assertion is what caught the inverted sign. It is three lines, it encodes no cleverness, and before the fix the two numbers differed by twice the P&L in opposite directions. ## The part that matters more The suite went green. All six bugs fixed, regression tests pinning each one. **And not one bot has a demonstrated edge.** On a deflated-Sharpe test across the fleet — which adjusts for how many strategies you tried before finding one that looked good — *not one bot clears PSR(0) ≥ 0.95*. Every record is statistically consistent with luck once the trial count is accounted for. That is not a disappointing result. It is the correct one, arrived at honestly, and it is what most people running a bot fleet do not check. If you try twenty strategies and keep the two that look good, you have not found an edge. You have found the two that got lucky, and the maths for that is well understood and routinely ignored. So the honest status of my system is two separate sentences: > The code is tested and does what it is supposed to. No strategy has a proven > edge. Conflating those is the easiest way to lie to yourself. **A green test suite is a statement about the code, nothing more.** It says the machinery works. It says nothing whatsoever about whether the idea does. ## What I would tell anyone testing a system that touches money **Isolate first, or you will not run them twice.** Before the first assertion, stub the alerting, the database and the operational logs. My suite runs in three seconds and touches nothing — that property is why it still gets run. **Assert invariants, not examples.** Examples pin today's behaviour; invariants catch the case you did not imagine. The interesting bugs live at the boundary where two correct-looking rules interact. **Write the tests before you trust the numbers.** I had months of P&L from a system with a sign error in it. The reports were confident and wrong, and no amount of staring at the dashboard would have said so. **Then test the edge separately, and harder.** Correctness is the cheap half. --- If you have a system where the failures are quiet — money moving, reports that look plausible, nothing throwing — the useful question is not "is the code right" but "what would it look like if it weren't." **[Run the free website check](/website-audit)** — unrelated to trading, and a fair sample of the same instinct: a public endpoint that drives headless Chromium at whatever URL a stranger supplies, with the boundary and the tests that implies. If you want another pair of eyes on the parts of your system that fail without telling you, that is the work I do. [Tell me what you are running](/contact) — and if it turns out your reports are fine, I will say so.