Round 7 is finished: V1, T5 and F5 are merged on main
Hyper Crux: Round 7 is finished. V1, T5 and F5 are merged on main, all 76 planted bugs were caught, and both workflows passed on GitHub. The round used about 1.7 million tokens over about an hour and a half, including the timed benchmark run.
V1, vector arrays and Nearest: each table’s vectors are stored in memory, and Nearest searches them exactly. In rough timings on the shared machine, all three searches beat their targets:
| Search | Measured | Target |
|---|---|---|
| 10 nearest among 100,000 vectors of 384 values | 33 to 39 ms | 65 ms |
| The same, with a filter that lets a tenth through | 7 to 9 ms | 12 ms |
| 10 nearest among 10,000 vectors of 1,536 values | 13 to 15 ms | 26 ms |
These are the store’s own times. The public package can’t search yet; that comes with G2.
T5, benchmarks: 0.1’s benchmarks now run on 0.x and on the Beta side by side. I made the first timed run once nothing else was using the machine. It took 5½ minutes, and the results file is in the repo.
| Benchmark | 0.x | Beta | Beta’s target |
|---|---|---|---|
| Put one record, committed | 221 µs | 208 µs | no slower than 0.x ✓ |
| Get by key | 13.9 µs | 858 ns | 3 µs ✓ |
| Put with a vector, 1,000 per transaction | 40.1 µs | 15.5 µs | 5 µs ✗ |
| Open 100,000 records with vectors | 504 µs | 588 ms | 250 ms ✗ |
The two misses:
- Batched puts are three times over their target. That’s for the later task on file growth (F7) to look at.
- Opening is more than twice its target. Part of that is because the file is still an uncompacted log. But V1 found that loading the vectors alone takes about 300 ms, which is already past 250 ms. The integration task (I1) measures this properly, and it decides whether the optional vector-block task (V2) runs.
0.x’s open time looks tiny because SQLite only reads the file as queries need it. The searches and walks will be timed once G2 adds them to the public package.
F5, failed commits: a commit that fails is now cut back out of the file before the lock is released. If even that fails, the handle keeps the lock and refuses writes until it’s closed. It also adds one sync before the cut. Without it, a stuck handle could leave the file so that a later writer’s commit might be lost in a power cut. FORMAT.md and BETA.md now describe this. No byte of the file layout changed, so the format stays at version 1.
74 of the 133 hours are done. The next round’s briefs are in the repo:
- G2: the first working version.
Scan,Drop,Link,Unlink,Neighbours,WalkandNearestwithout a filter go into the public package. The differential test then runs 0.x against the Beta. I’d re-run the benchmarks afterwards to timeGetandWalk. - F6: following other processes, in the log.
- Q3: the SQL parser. I’ve put it ahead of Q1 and Q2, because their tests replay SQL text and can’t run without the parser.