Semantic Search over 1M+ arXiv Papers With No Backend

Papyrus is a semantic search engine over more than a million arXiv papers and the whole thing runs in the browser. There is no inference backend. The embedding model that turns your query into a vector runs on your machine. So do the search and the ranking.

That changes the problem a lot. A normal RAG app assumes a server with plenty of RAM, a hosted vector database and a GPU somewhere behind an API. I had none of that. Every byte of the index has to be downloaded to the user and every comparison runs on whatever consumer hardware they happen to have. The model also has to be small enough to load in a tab before people give up and close it. Most of the interesting engineering in Papyrus comes from taking that constraint seriously instead of working around it.

Making a million vectors fit

The core trick is binary quantization. Instead of storing each embedding as a list of 32-bit floats, Papyrus keeps one bit per dimension. That alone makes the index 32 times smaller than the float version. Search gets cheaper too, because comparing two binary vectors is Hamming distance: an XOR and a popcount per machine word. That's the reason scanning a million-document index on a CPU is realistic at all.

The price is precision. Keeping only the sign of each dimension blurs the finer distinctions between documents, so retrieval runs in two stages. Hamming distance pulls a shortlist of candidates fast, then a cross-encoder reranker reads the query and each candidate together and puts them in the right order. The cheap stage handles scale and the expensive one only ever sees a handful of documents.

The models run through Transformers.js on WebGPU when the browser supports it and fall back to WASM automatically when it doesn't. The same page works on a gaming desktop and on a machine with no usable GPU access. The heavy part is embedding the whole corpus. That happens exactly once, offline. I ran that pipeline on a 4-GPU cluster, so the browser only ever has to embed one short query, never a million abstracts.

Shipping the index like an image

Since the index is just a compact binary file, distributing it looks more like serving an image than running a database. Papyrus fetches it from Cloudflare R2 and caches it in IndexedDB so returning visitors don't download it again. A registered service worker keeps search working with no network connection at all.

What I like most is what's missing. There's no vector database to keep alive, no GPU instance billed by the hour, no per-query inference cost. Every new user brings their own compute. The thing that usually gets more expensive as a search product grows is running the model for every query. That cost simply isn't on my bill. R2 doesn't charge for egress either, so serving the file is the only real cost and it stays small.

Client-side search isn't the right call for every corpus. For a large, mostly static collection like arXiv the whole stack can live in the user's tab.

The code is on GitHub if you want to see how the pieces fit together.