Semantic search that runs on your device

"Semantic search" — finding things by meaning rather than by exact words — has quietly become table stakes. The catch that rarely gets mentioned: for most tools, semantic search means your documents are sent to a server, turned into vectors somewhere in the cloud, and matched there. That is fine for public web pages. It is a strange default for a private research library full of unpublished drafts, confidential sources, and your own half-formed ideas.

ScholarThread takes a different position. When you are working locally — signed out, or reading on your phone — the semantic search index is built and searched on your own device. Your library does not leave it to be searched. Here is what that means in practice, and why it is possible at all.

What "by meaning" actually buys you

Keyword search fails in a specific, frustrating way: it only finds the words you remember. If you highlighted a passage about "allocating weight across inputs" and later search for "attention mechanism", keyword search shrugs. The concept is the same; the strings are not.

Semantic search closes that gap. Under the hood, each note, passage, and abstract is turned into a vector — a compact numerical fingerprint of its meaning. Your query becomes a vector too, and the search finds the fingerprints closest to it. Wording stops mattering; the idea is what gets matched. For a research library, where you are forever half-remembering an argument rather than a phrase, this is the difference between "I know I read this somewhere" and actually finding it.

The usual bargain — and why we declined it

The reason meaning-based search is normally a cloud service is that turning text into good vectors used to require a large model and a beefy server. So the bargain on offer is: hand over your documents, get semantic search in return.

We think that is the wrong trade for a tool built around your private reading. Two things let us avoid it:

• The models got small enough. A modern embedding model compact enough to run on a phone is now good enough to power genuinely useful search. It does not need a data centre.
• The index is small too. Your library is yours — hundreds or a few thousand documents, not the whole internet. Searching it by brute force on-device is fast, comfortably faster than you can notice, even without any special hardware.

Put those together and the cloud round-trip simply is not needed for your own library. So we removed it from the local path.

What runs where

To be precise about it, because "private" claims deserve precision:

• Working locally or signed out. Everything — building the index and searching it — happens on your device. Your papers, notes, and highlights are embedded and matched right there. Nothing about the content of your library is sent away to perform a search.
• Signed in on the web. Search is served by your account's backend so results stay in sync across your devices. That backend is still yours: your library, your tenant, not a shared pool your documents get mixed into.

The local path is the one we are proudest of, because it means the privacy-sensitive case — the researcher who has not signed up, or who is working with material they cannot put in the cloud — gets full semantic search with nothing leaving the machine.

The offline dividend

Because the local index lives on your device, a useful side effect falls out for free: it works with no connection at all. On a plane, on a train, in a library basement with no signal, you can still search your papers and notes by meaning. The index was built on your device and it is searched on your device; the network was never part of the loop.

This is the same instinct behind the rest of ScholarThread's offline-first design — your library renders and your PDFs open without a connection — extended to search. The tool should not stop being useful the moment the wifi does.

It covers your notes, not just your papers

One thing worth calling out: on-device semantic search is not limited to the papers you have collected. It also indexes your own notes and the text of documents you have read. So a query by meaning can surface a synthesis note you wrote three weeks ago just as easily as a published abstract — often more usefully, because your notes are already phrased in the terms of your project. The search that finds the paper and the search that finds your thinking about it are the same search.

Doing this in ScholarThread

You do not have to turn anything on. Search your library and meaning-based ranking is already part of the results — fused with keyword matching so you get both precision and recall (we cover that pairing in our post on the two search scopes). If you are signed out or on your phone, that ranking was computed on your device; if you are signed in on the web, it is served from your own account backend and kept in sync.

The principle underneath is simple: searching your private library by meaning should not require handing that library to anyone. Small models and a small index make the private version not just possible but genuinely fast — and, as a bonus, available even when you are offline.