0. Three sentences up front
- This homomorphic encryption inference engine runs — the full inference pipeline of a 28‑layer Transformer has been executed.
- It is reproducible — byte‑for‑byte comparison passed in two independent workspaces (one containing existing data, one a fresh clone).
- It is not secure — parameters are mechanism‑verification level (n=2048), far below the 128‑bit security standard, and must not be used to protect real data.
These three sentences are the premise for all conclusions in this series. Below we elaborate.
1. Security boundaries: three non‑security facts
All parameters in this project (n=2048, 112‑prime inner chain, 2100‑prime bootstrapping chain) are mechanism‑verification level, not production grade. Beyond that, there are three concrete non‑security facts:
| # | Fact | Consequence |
|---|---|---|
| 1 | The random number source is a prototype xorshift, not a cryptographic RNG | Security analysis does not hold |
| 2 | The private key sk.bin is circulated together with the ciphertext in the relay package | No key escrow, no segmented decryption |
| 3 | To make bootstrapping numerically feasible, the private key sparsity hw was reduced to 8 | The private key is sparser and less secure |
These three are not “items to be improved”; they are the current state. Any output that cites this project should also cite these three facts.
2. Blocking‑level: three most critical unresolved problems
2.1 Bit‑level reproducibility defect not yet fixed
Phenomenon: The output bits differ between thread counts T23_NT=4 and T23_NT=8.
Root cause: The _Thread_local RNG is seeded with a fixed constant, so random number consumption is tied to thread scheduling.
Solution: Seed by automorphism index k (the code already has an entry point for k), but it has not been implemented yet. The source code remains:
c static _Thread_local uint64_t ckks_rng_state = 0x243F6A8885A308D3ull;
Impact: All “bit‑level reproducible” claims must be bound to the thread count. If you reproduce with a different thread count, the results will not be byte‑identical.
2.2 Precision margin is not wide; the tail of the chain already has out‑of‑threshold terms
| Object | Result | Max error |
|---|---|---|
| u0 (layer 0) | PASS 8/8 | 1.27e‑03 |
| u0r112 (after bootstrap refresh) | PASS 8/8 | 1.27e‑03 |
| u26 / u26r112 (chain tail) | PASS 8/8 | 4.47e‑03 ~ 2.30e‑02 |
| u27 (chain tail) | ⚠️ 6/8 | t1h1=3.82e‑02, t3h1=7.97e‑02 (exceeds default tolerance 3e‑2) |
Two key points:
- Bootstrap barely changes the numerical values: 1.2712e‑03 → 1.2706e‑03. Bootstrap is a “top‑up”, not a “correction”.
- Error grows roughly one order of magnitude with depth: from 1.27e‑03 at layer 0 to 2.3e‑02 at the chain tail.
We do not adjust the tolerance to make it “pass”, but whether these two items count as failures is not yet settled.
2.3 No fallback for out‑of‑bounds ciphertext attention scaling
softmax uses “pre‑scaling” instead of max‑sub. Once real inputs cause scores to leave the statistical interval, the fitted polynomial’s behavior outside the interval is not guaranteed, and no error is reported.
This is the most worrying part of the entire series: it is silent and depends on the input. You may get a result that “runs”, but it is mathematically wrong.
3. Important technical gaps (ordered by severity)
| # | Gap | Description | Part |
|---|---|---|---|
| 1 | Bootstrap domain transformation not attributed | coeff_to_slot + slot_to_coeff accounts for 80% of bootstrap time, but “why is it expensive” is only speculative (suspected memory bandwidth), no micro‑architecture profiling | Part 15 |
| 2 | No closed‑loop precision budget | Per‑layer independent scaling + post‑hoc whole‑chain acceptance; no derivation that back‑propagates usable error per layer from 28 layers | Part 9 |
| 3 | No minimal usable chain length scan | Boot compiled to 2100, but how much is actually used and how much can be compressed was guessed | Parts 5, 7 |
| 4 | “Missing means default” mode not cleaned up | Reading the entire tree of similar files requires one‑by‑one review | Part 8 |
| 5 | DIRECT_LAYERS is a hardcoded parameter tuning | Not a derived rule; cannot automatically adapt when switching models | Part 8 |
| 6 | Protocol has no version number, no input fingerprint | When parameters change, old and new packages look identical; the next party cannot verify the input | Parts 11, 14 |
| 7 | Thread pool has no non‑reentrancy assertion, no A/B data | Non‑reentrancy is a silent failure; “how much was saved” has no same‑round measurement | Part 12 |
| 8 | NTT only does radix‑2 | Not SIMD‑ized; the two modular multiplication paths have not been cross‑validated | Part 3 |
| 9 | Preprocessing scripts not integrity‑paired | Only verified weights (9/9) and silu (26/26); the output of fitting scripts has not been byte‑for‑byte paired | Part 2 |
| 10 | Switching models requires a full rerun + manual judgment | Scaling, fitting, SiLU path selection all depend on statistics from the plaintext reference | Part 9 |
4. Performance ledger: knowns and unknowns
4.1 Known numbers (this project, AMD Ryzen 7 9800X3D, 8 cores, pure CPU)
- One hop (lay + boot) ≈ 1.8 hours (≈ 6205 s)
- Of which bootstrap (boot) ≈ 1.2 hours, intra‑layer forward ≈ 0.6 hours
- The two domain transformations of bootstrap (coeff_to_slot 1688 s + slot_to_coeff 1716 s) total ≈ 3400 s, ~80% of bootstrap time
4.2 Unknowns
- Why the domain transformations are expensive has no micro‑architecture attribution. We only measured “how long this section took”, not cache hits, bandwidth, instruction counts. To confirm “the bottleneck is bandwidth” requires an extra round of profiling — by project discipline, this is marked as an unverified hypothesis.
- The proportion of “memory bandwidth” in single‑hop time is unmeasured. If the bottleneck really is bandwidth, the optimization direction should be reducing fetches (shorter chain, smaller key volume), not accelerating computation.
4.3 Horizontal comparison with the frontier (for reference only, not data from this project)
| System | Hardware | Result |
|---|---|---|
| Cerium (arxiv 2512.11269) | NVIDIA GPU (multi‑GPU) | Bootstrap 7.5 ms; BERT‑Base 8 s; Llama‑3‑8B 134 s |
| Odin (arxiv 2609.12378) | Single H100 80 GB | Llama‑3‑8B 366.4 s (end‑to‑end) |
| Cachemir (arxiv 2602.11470) | GPU | Llama‑3‑8B < 100 s per token |
| PQC‑LLaMA‑3 (arxiv 2604.12168) | i9 CPU | 237 ms/step, up to 80 token/s, accuracy 98% |
This project does not claim any advantage or disadvantage over other solutions — absolute values across platforms and implementations are not comparable.
5. Known trade‑offs: what we deliberately chose
These are not bugs, but active trade‑offs. Listed to avoid being treated as oversights:
| Trade‑off | What we gained |
|---|---|
| hw = 8 (private key sparser, less secure) | Aliasing in bootstrap folding is smaller → sine approximation needs only a 9th‑degree polynomial |
| Keeping the BFV‑era implementation and not deleting it | It is the simplest correctness self‑check (T1–T6), and also the original evidence of “why this path doesn’t work” |
| Self‑built thread pool does not support nested parallelism | Lower scheduling overhead + deterministic partitioning |
| Data package not distributed with the repo (~7 GB) | Repo size stays manageable; the cost is “must be able to self‑generate” (verified byte‑for‑byte identical) |
| Not pursuing full pass at the chain tail | Honest accounting, rather than tuning tolerance |
sk.bin included in the relay package | Avoids key distribution protocol; the cost is no privacy property |
6. Documentation and consistency debt
| # | Debt |
|---|---|
| 1 | The “architecture” paragraph at the top of vllm_ckks.h still says “four 60‑bit primes, supports depth 3”, while CKKS_NPRIMES is already 32 by default — the description stays at the initial prototype |
| 2 | The exact relationship between out np=2083 in boot logs, compile‑time 2100, and on‑disk 112 is unverified; we explicitly note it in the documentation without asserting it |
| 3 | Phase 2 (l1) and phase 6 (lay) semantics overlap, a leftover from development |
| 4 | The criteria for the fin phase are described differently from earlier layers, not prominent enough — we hit this ourselves when reproducing |
| 5 | The “unverified items” noted in each part will become stale as the code evolves; a unified audit is needed |
7. If you want to continue: where to start
Sorted by benefit ÷ difficulty, the top three are:
- Implement RNG seeding by k (item 2.1). The structure is in place, the solution is clear, the impact is well‑defined. Only after this can “bit‑level reproducibility” be stated without an asterisk. While you’re at it, add a regression check in CI that compares bytes across different thread counts.
- Do a real profiling of the bootstrap domain transformations (item 3.1). 80% of the time is there. Even just confirming “is the bottleneck bandwidth or compute” will redirect optimization from “optimize convolution” to the correct direction.
- Make DIRECT_LAYERS a derived rule (item 3.5). The criterion (narrow value range → direct fitting) can in principle be automatically generated from plaintext reference statistics. Once this is done, “switching models” has a chance of automatic adaptation.
8. One thing I couldn’t accomplish
By the time this series ends, one thing must be made clear:
We have not yet made this thing “usable by strangers”. The repo, documentation, and Release assets are ready, but it has not reached the point of being independently reproducible by the outside world — because the publicly accessible entry point (repo push + Release upload) is not yet complete.
In other words, all conclusions of “reproducible” and “independently verifiable” in this series are currently only validated by us in two independent workspaces; no third party has run it yet.
This is exactly why those 21 hops are waiting to be claimed, and why this design exists.
If you’ve read this far and have a machine willing to spend 1.8 hours — feel free to take a hop.
Appendix: Index of all 16 parts
| Part | Topic |
|---|---|
| Parts 1–2 | Preprocessing and data pairing |
| Part 3 | NTT and polynomial multiplication |
| Part 5 | Chain length and parameter selection |
| Part 7 | Bootstrap pipeline |
| Part 8 | DIRECT_LAYERS and SiLU path |
| Part 9 | Per‑layer scaling and precision budget |
| Part 10 | Ciphertext attention scaling |
| Part 11 | Protocol and relay package |
| Part 12 | Thread pool |
| Part 13 | Bit‑level reproducibility |
| Part 14 | Relay verification |
| Part 15 | Performance profiling (the 1.8‑hour ledger) |
| Part 16 | Open list and known boundaries (this part) |
The series is complete. All 16 parts. If you find any discrepancy with the source code, the source code prevails — every assertion in this document is annotated with its basis and uncertainty.