The Lab

Where the idea comes from

• A soak is a long, unattended run of software under real conditions, to see what breaks. Ours ran the bots on four Bybit demo accounts and one Hyperliquid testnet account.
• Splitting a result into the trading and the holding is the idea behind performance attribution: how much came from the decisions, how much from simply owning something. Gary Brinson, Randolph Hood and Gilbert Beebower (1986) set it out for funds. Here it is much simpler: bots against the coins that sat in the wallet.
• Marking to market values each open position at the current price, so a result counts whether or not it was closed.
• The worst fall (a drawdown) is the largest drop from a high to a later low.

How we tested

• The window: from the restart on the first audit's fixes, 13 October 05:54 UTC, to the stop, about 16:30 UTC on 21 October: eight and a half days. The rehearsal account (slow) started later the same morning, at 06:41.
• The phases:
  • A, to the restart with every grid and DCA widened, 21 October 06:26 UTC;
  • B, the ten hours from there to the stop;
  • C, the flatten and the sale of every coin for USDT, on its own line and never mixed into the soak's result.
• The sources:
  • each account's equity, read every two to five minutes, with each bot's position and price;
  • Bybit's own record of every fill, fee, funding payment, interest charge and transfer, with the balance after each;
  • the reading after the sell-down, 22 October 13:28 UTC;
  • coin prices from Bybit's public one-minute candles, and USDT in US dollars from Coinbase's public candles.
• No money went in or out inside the window. Each subaccount's opening grant came before its first reading. Checked row by row: no transfer inside.
• The split, fixed before the run:
  • the held coins are the wallet's coins that no bot was using, at the start of each phase;
  • their part is each coin times its price change;
  • the bots' part is all the rest: fills, fees, funding, interest and the open positions' marks.
• The control is doing nothing: the held part is what the account would have made with every bot switched off and its coins kept.
• The sudden steps: every move of 3% or more between readings under 30 minutes apart, set against what the open positions say it should have been.
• Added after the first run, and said so: demo's result split bot by bot (each fill carries its bot's name), and each account's fees, funding and interest summed. Neither changes the frozen numbers.

The maths, step by step

1. An account's result. Equity at the end over equity at the start, minus 1. From 100 to 103 is +3%.

2. The held part. An account of 1,000 holds 1 idle BTC that goes from 100 to 110: the held part is +10, or +1%. If the account ended at 1,030, the bots' part is the other +20, or +2%.

3. A bot marked to market. Add up the cash its fills paid and took, then add what its open position is worth at the end and take off what it was worth at the start. Buy 2 at 100, still holding at 105: −200 + 210 − 0 = +10. Then take off its fees and add its funding.

4. A position's step. A long of 20 coins on an account of 1,000, when the coin moves 0.5%: 20 × 0.5% of the coin's price. At a price of 100, that is 10, or 1% of the account. At 20 times the account's equity in positions, a 0.5% move in the coin moves the account 10%.

What we found

1. Four accounts ended up and one down; the market went nowhere.

• Over the same window Bitcoin rose 1.2%, Ether fell 0.9% and Solana ended flat. Bitcoin stayed between 4.4% below and 1.5% above its start; Ether reached 8.1% below.
• Read with the held coins at each phase's end instead of its start, no held part changed by as much as 0.01%.

2. Demo's +40% was two leveraged stress-test positions, not the grids.

• Demo ran a deliberate stress test: a Bitcoin long at 50x and an Ether short at high leverage, beside its ordinary grids.
• Its open futures were worth 15 to 28 times its equity: 19x at the start, 17x at phase B's start, 28x at the most.
• Bot by bot, over the whole soak:
  • the Bitcoin long made +30.6% of the start equity;
  • the Ether short made +7.3%;
  • the other 14 bots made about +3.0% together;
  • 0.67% is not explained by any bot.
• Fees took 17.1% of the start equity in eight and a half days, and funding 0.5%. Before fees, the trading made about 58%.
• The same account fell 40.4% from a high on 14 October 08:09 UTC to a low on 15 October 13:53, in 30 hours, then climbed back. A result that falls 40% on its way to +40% shows how far the account was leveraged. It does not show skill.

3. Demo's sudden steps were its positions, marked to market.

• There were 59 moves of 3% or more within about two minutes, all between 14 and 18 October.
• In all 59, the open positions moved the same way as the account.
• Summed over the 59, the positions account for 93% of the size.
• The readings of equity and of each price are taken a moment apart. At 20x, that gap alone leaves single steps between 16% and 169% explained.
• No other account had a single step of 3%.

4. Carry lost on costs and its own trading; tilt and slow mostly followed their coins.

• Carry, long spot and short perps, the book hedged: −3.3%.
  • Fees took 1.9%.
  • Its own trades and marks took another 1.6%.
  • Funding paid it 0.17%.
  • Loan interest cost 0.01%.
  • This was the stretch in which its spot side was taken to 10x on a loan and its hedges to the venue's highest leverage, by design, to test the book at its edge. Its worst fall was 3.7%, from 16 October 08:50 to 18 October 19:59 UTC.
• Tilt, the same book leaning with a regime reading, no loan: +0.68%.
  • Its bots added only 0.12%.
  • The rest was the grant's idle coins, mostly its one Bitcoin.
  • Worst fall 2.1%.
• Slow, the rehearsal for the real-money row (an Ether long grid, and a Bitcoin long bot added the same day): +1.35%.
  • Its two bots made 0.88%: Bitcoin's 0.74% and Ether's 0.09%.
  • The idle coins made 0.47%.
  • Worst fall 6.8%, on the same low as demo's, 15 October.
• Hyperliquid testnet, five bots: +5.3%, all from the bots.
  • +12.6% in phase A, then −6.5% in the ten hours of phase B.
  • Worst fall 7.9%, from 20 October 19:27 to 21 October 16:10 UTC.
  • Its futures ran at 1.4 to 3.3 times its equity.

5. The ten widened hours (phase B) were too short to judge.

• In phase B, Bitcoin, Ether and Solana each rose 2.4 to 2.8%.
• Ten hours of one rising afternoon is not a reading of the widened settings.

6. The flatten and sell-down cost 0.09 to 0.62% on Bybit.

• Demo's closing fees were 0.85%, against its leveraged book. Slow's, carry's and tilt's were 0.01 to 0.05%, and their sales of coins 0.05 to 0.10%.
• The coins were sold by 17:04 UTC on 21 October, before Bitcoin fell 3.3% by the next day's reading. Being in cash missed that fall.
• Hyperliquid's +2.9% is the whole step from the stop to the reading: without a transaction record, it cannot be split.

What changed along the way

The bots were not left alone. Each change below falls inside the window:

• 13 October: the rehearsal account started, at 06:41 UTC, with its Ether grid. A Bitcoin bot was added from the panel later that day. At 09:48 UTC the fleets were restarted on the fixes from the day's second audit; no account's equity shows a step there.
• 15 October: carry's spot side was allowed up to 10x on a loan, its margin floor set to 1%.
• 16 October: carry's first 10x step, from 07:19 UTC. The buys filled; the hedges were refused for margin, so the book stood long and unhedged until a fix put each hedge before its buy. The hedges went to the venue's highest leverage.
• 18 October: each portfolio row's leverage became settable from the panel; the hedges were set to 50x (Solana 40x, the venue's limit at its size).
• 20 October: removing a bot became a "retire" with gentler steps; nothing traded differently.
• 21 October, 06:26 UTC: every grid widened to 25% either side, 12 rungs, no slide; every DCA's safety orders 5% apart (phase B). A DCA's card began showing its exits apart from its safety orders: display only.
• 21 October, about 07:55 UTC: carry went to 1x, inside phase B.
• 21 October, about 16:30 UTC: every fleet stopped, flattened from 16:40, coins sold to USDT by 17:04.
• The restart of 12 October (08:31 UTC) came before the window, so it is not a boundary here.