Why I built it
I wanted to find out what it actually takes to build a search engine from nothing, rather than to have an opinion about it. Not a wrapper around someone else's index — an actual crawler, my own extraction and ranking, and a real question about how a small, honestly-scoped index compares to a mainstream one when they sit next to each other on the same page.
That meant deciding early what the site would never pretend to be. It was never going to cover the internet on its own, so I paired my own crawl with a general-web provider, Brave, and made the split visible rather than blending the two into one anonymous list. A result you see here always tells you which of the two produced it.
How it works
A query goes to both sources at once. The general-web side asks Brave and returns results labelled as live web. The independent side searches a smaller index I built myself: a crawler that respects robots.txt, follows per-host politeness rules, and works to hard daily and monthly budgets so it never runs unchecked. Whatever it finds is processed, deduplicated and ranked by a deterministic scoring method I control, then stored as a versioned index rather than edited in place.
The two result sets are combined on one page, but never disguised as each other. If the independent index has nothing relevant, you simply see the general-web results; if the general-web provider is unreachable for any reason, the site falls back to whatever the independent index holds rather than failing outright.
Location matters to some queries — a weather or local search means something different depending on where you are asking from — so ambiguous place names default to UK results unless the query itself names another country or region. Nothing more precise than that coarse hint is ever used.
An automated pipeline watches how much storage the independent index is using and pauses new crawling before it would come close to its limits, rather than reacting after the fact. It did not catch the database-write runaway, and the reason is the lesson: that problem was about how often data was rewritten, not how much was stored, so it surfaced as unexpected usage on the hosting account rather than as a full disk. Watching size is not the same as watching work. The cause was found and fixed, and crawling was paused rather than restarted immediately, so the fix can be trusted before the index is allowed to grow again.
For developers: the crawl and search pipeline
The independent side runs as seeds moving through a policy gate, a frontier that schedules and leases URLs, a fetcher hardened against server-side request forgery, an extraction and deduplication stage, and a versioned document store. A query processor and a deterministic ranker then serve results from whichever index version is currently active. Crawling is asynchronous throughout: a live search only ever reads the active, already-built index, and never waits on a crawl in progress.
The general-web side sits behind a neutral provider interface, so Brave could in principle be swapped or supplemented without touching the rest of the system. Results from both sides are normalised into one shared shape before being deduplicated and reranked together, and every result keeps a field recording which source produced it, which is what lets the label on screen be trusted rather than asserted.
Several of the defects that mattered most were only found once the system was actually deployed and given real traffic, not in its automated tests. One crawler build, for instance, passed its network fetch function around unbound, so every attempted fetch failed with a runtime error and nothing was ever crawled, even though the automated suite passed because its tests substitute a mock fetch. That class of problem — correct in isolation, wrong once wired into the real runtime — is the specific reason a deployed, independently verified system is treated as a stronger form of evidence here than a green test run on its own.
Where it may go
The most immediate step is resuming scheduled crawling once I am satisfied the fix for the database-write runaway holds under real conditions, so the independent index can start growing again. Beyond that, an AI-generated answer feature and a more advanced ranking approach exist as boundaries in the system today but are deliberately switched off, and I am not committing to a date for either. If I do turn them on, it will be because I can state plainly what they do and do not do, in the same way the two current result sources are labelled now.
I am also not planning to grow the independent index toward internet-wide coverage. Its value, if it has any, is in being small enough to actually inspect and explain, and I would rather keep that property than chase scale for its own sake.