Measures scoring speed and end-to-end query latency in your browser.
Offline evaluation on pre-built embeddings. Updated each time compare_eval.py is run.
| Method | Mode | MRR | Hit@1 | Hit@6 | Notes |
|---|---|---|---|---|---|
| Potion base int4 | Lite | 0.4290 | 22/67 | 38/67 | distilled-mxbai base, int4 scoring |
| Potion fine-tuned int4 | Lite | 0.4815 | 26/67 | 40/67 | distilled-mxbai fine-tuned, int4 scoring |
| Full pure binary (ITQ) | Full | 0.5308 | 28/67 | 45/67 | mdbr-leaf-mt, binary ITQ (mobile) |
| Full binary+int4 rerank | Full | 0.6025 | 35/67 | 50/67 | mdbr-leaf-mt, binary+int4 rerank (desktop) |
Benchmarks the raw vector scoring step only (no model inference). 20 iterations, random query vectors.
Full pipeline: query encoding (model inference) + scoring + ranking.
| Test | Status | Load (s) | Query avg (ms) | Notes |
|---|