Survival LLM is a free, MIT-licensed desktop app that pairs a local AI model with an offline library of 494,574 articles, so you can ask emergency questions with no internet. It is a few days old (v0.4.1), runs on Mac and Windows, and lives at anonnerdfrenzy/survival-llm. The author published test results, so I checked the part nobody had: the library search with no AI model.
I ran Survival LLM’s own search on its own 22 test questions, with no model. An on-topic title, by the author’s own pattern, landed in the top 5 for 21 of 22 questions. But every key fact showed up for only 9 of 22. My verdict: great reading tool, bad doctor, and not something you can bundle in a product.
What did Survival LLM’s search find with no model?
First the setup, because that is where I can be wrong. I used the Essentials library from the October 28 release on an Intel N95 (4 cores, 16 GB of RAM, no GPU). I did not run the desktop app, and I ran no AI model, not one. I called the library’s own Library.search() from Python with each raw question, the way a person types it.
The author’s test file lists 22 questions, each with a title pattern for “on-topic” (like candle|outage|fire) and key facts as regular expressions, things like “20 feet” or “8 drops”. I used those as they are.
Results:
- 21 of 22 questions had a title matching the author’s pattern in the top 5 results. Only 15 of 22 had one in first place.
- 9 of 22 had every key fact in the top 5 snippets plus the first 5,000 characters of the top 3 articles. The mean share of facts found was 66.3%.
- Speed: from the saved run file, half the searches took under 0.15 s and half took 0.8 to 3.4 s.
Both lines are lenient, for the same reason: loose patterns. Read strictly, the same data says 18 of 22 had at least one key fact, 15 of 22 had the right article first, and 3 of the 21 “hits” were the wrong page. A pattern matching “month” or “effective” makes almost any answer look complete. And a regex match over 15,000 characters is a pile to read, not an answer.
Still, the dumb search does a decent part of a librarian’s job on a weak machine. Franchement, I did not expect that from a keyword index.
Where the plain search falls over
Four questions got zero key facts from the raw question: candles during an outage, saving phone battery, a downed power line, and a toilet with no water. The candle question returned a Christmas-tree page and the battery question a page about reconditioning batteries. Both count in the 21, because the pattern only wants “candle” or “battery” in the title. The downed line question returned a yard-dirt Q&A page.
Then, not a benchmark, only a hint: I typed a 2 to 4 word query for each of the four by hand, and all four returned some facts. That is me tuning on the answer key. But it points at one job for a model: rewriting a sentence into the words the index knows.
The author’s data adds a twist. In their README, the model that scored best (Nemotron 3.5 Super) is described as mostly answering from memory. Recomputing from their result files, that model opened an article on only 5 or 6 of the 22 questions, while Qwen3.8 27B opened one on 21 and Granite 4.2 8B on 21 or 22. An outside user filed issue #1, “Retrieval runs inverse to answer quality”. The maintainer closed it on October 28, replying that answering from memory is fine as long as the app says plainly when it did not search, and v0.3.4 changed that behaviour.
Fair. But look at Granite: same search, an article opened on nearly every question, fully right on 3 of 22. So retrieval is necessary, not sufficient: reading pages did not rescue the small model, and the best scorer mostly remembered. Size is a confound (120B against 8B).
The 16 GB default model: do not trust it for first aid
On a Mac or PC with 16 to 31 GB, Survival LLM downloads IBM Granite 4.2 8B. I did not run it. I recomputed the grades from the repo’s published judge files with its own score_judging.py. The judges are unnamed AI models and no human graded anything.
For Granite 4.2 8B on the 22 questions:
- Main point right (judge accuracy 2 or 3 on a 0 to 3 scale): 14 of 22.
- Fully right (a 3): 3 of 22.
- Scored 0: 3 of 22.
Qwen3.8 27B had its main point right on 21 of 22 and fully right on 10. Nemotron 3.5 Super: 21 of 22, 14 fully right. Those are 18 GB and 80 GB downloads, not a 16 GB laptop story.
The product page says “21 of 22”. That is true for the big models, and it means “main point right” by AI judges, not 21 correct answers.
The same README explains why the smaller Granite 4.2 3B is not offered: it told people not to use a tourniquet on an arm, and to stay 3 feet from a downed power line, where the README says the guidance is 35 feet. I am not a doctor, so no first aid advice from me. The project itself threw out a model for confident wrong answers, and the 8B sits one step above it. If you install it, use it as a library browser: read the source panel and ignore the paragraph on top.
The README also says its tests ran on an earlier library build. The numbers compare models. They do not guarantee the version you download.
Do not bundle the library in a product
The code is MIT. The content is not. Survival LLM’s SOURCES.md marks wikiHow, iFixit, the IFRC first aid guidelines and the WHO guides as non-commercial licences (CC BY-NC-SA, or “copying allowed for non-commercial use”). Not all are in the Essentials pack I tested, which has no iFixit and no Wikipedia. The file says plainly that anyone who wants to use the library commercially should rebuild it without them.
Same trap as the model licence I wrote about in Mistral Large 4 License: Use It, Host It, or Pay?. Read the content file, not only the LICENSE. I am not a lawyer, so ask one.
What I would copy for Trellis
I will not ship a Survival LLM clone, but the method is free. Trellis is a pile of Amazon seller questions and policy text.
Before I pay for a model to answer from our documents, I run plain search alone on 20 real seller questions, each with a pass/fail list of facts. Write tight fact patterns. Loose ones (“month”, “effective”) made this test look better than it was, and your retrieval test will lie to you the same way. Then grade the model’s answer too. These 22 questions are CDC and FEMA topics that big models already know. Seller policy text is not, so memory will not save you there.
What I did not check
- No model run. I did not run Granite, Qwen or Nemotron. Everything about model answers is the project’s judge files, recomputed, not my testing. My search is the library’s own
Library.search(), and the run is saved as a JSON file. - No app run. It ships for Mac and Windows only, so I only ran the library.
- The battery claim. The page claims about 100 W and 100 to 300 answers per charge on a MacBook Pro. I cannot back it up.
- The install size. The pack is a 1.28 GB zip. It took 3.23 GB on my disk, against 4.5 GB in the manifest and “3 GB” on the page. I could not explain the gap.
If you try the search yourself: the library needs SQLite 3.43 or newer. On Debian 12’s 3.40.1 my run died with unrecognized option: "contentless_delete", and pip install pysqlite3-binary fixed it.
Next I will run the same search-only test on 20 Trellis questions, and I will post how many of them have the right passage in the top 5.
