ren started because I am extremely susceptible to the sentence "I wonder if I can run this locally."
That was part of it, anyway.
The bigger reason was that I had been getting increasingly interested in computational biology and computational medicine, especially the former, and eventually staring at papers and models was not enough anymore. I wanted to actually build something with them. Not another notebook where I feed a model ten images, print an accuracy number, say "neat," and never touch it again.
I wanted a system.
My first sketch of ren was almost offensively simple:
Vision → Retrieval → Synthesis → Dashboard.
Image goes in. Vision figures out what is there. Retrieval finds relevant biomedical information. A local language model turns that into something useful. Human sees all of it through a workstation.
Easy.
Narrator: it was not easy.
The single constraint that infected basically every decision afterwards was that ren had to be local-first and offline-first. Not "we technically support local inference if you own a small datacenter." I mean something I could actually run on my own computer.
For reference, that computer has a Ryzen 5 5600, 32 GB of RAM, and an RX 6600 with 8 GB of VRAM.
It is fine.
That is probably the nicest description of it. Not a potato, not a workstation. Just aggressively normal.
Which made it a pretty good reality check.
The hardware gets a vote
A funny thing happens when you stop designing ML systems as if compute is an abstract unlimited resource: suddenly model selection becomes much more interesting.
I looked at Virchow for pathology embeddings. Great model. Also an absolutely comical fit for what I was trying to do on an RX 6600.
Path Foundation was much saner.
Same story on the synthesis side. I tested biomedical language-model options, including OpenBioLLM, before settling on MedGemma for the actual production path.
This was one of the lessons I already half-knew from messing with local inference but ren made painfully concrete:
the strongest model is not automatically the best component.
If Model A scores slightly higher but eats several gigabytes more VRAM, runs at a fraction of the speed, makes deployment miserable, and forces the rest of your architecture to contort itself around it, congratulations. You may have selected the worse model.
ren therefore does not try to keep the entire ML stack resident simultaneously like some kind of VRAM clown car.
The stages are intentionally separable. Vision runs. It produces artifacts. Those artifacts become somebody else's problem. Synthesis can come later.
That separation ended up becoming one of my favorite parts of the project.
ren is very UNIX-y.
Not literally pipes-everywhere UNIX, but philosophically: small components, boring interfaces, compose them into something much less small and boring.
Specimen ingestion owns ingestion. Vision owns vision. Review owns review. Retrieval owns retrieval. Synthesis owns synthesis. The dashboard stitches the workflow together.
Most boundaries are files and structured artifacts. The LLM gets one deliberately narrow local HTTP seam through llama-server.
If one thing catches fire, I can usually tell which thing caught fire.
This is underrated.
I did not want a medical oracle
There was another architectural constraint that had nothing to do with my GPU.
I really, really did not want ren to become:
computer says Thing™
therefore Thing™ must be true
That feels sketchy in ordinary software. In anything adjacent to medicine it becomes completely unacceptable.
So ren is diagnostic-assist.
That wording is not legal-decoration fluff. It actually changes the data flow.
The vision stage can produce findings, but those findings are not silently blessed as truth. A reviewer can confirm them, correct them, or reject them. Only confirmed or corrected findings are allowed to move forward into automatic retrieval and synthesis.
Rejected findings stay around because deleting your mistakes is a fantastic way to build an unauditable system.
The human is not a checkbox at the end. The human is part of the derivation path.
This also explains why I intentionally made parts of ren less "agentic" than they could have been.
Normally I enjoy giving LLMs tools and letting them cook.
Here? Absolutely not.
Automatic retrieval uses fixed mappings from reviewed findings rather than asking an LLM to hallucinate whatever biomedical search query vibes with it in that particular inference run.
Boring is good sometimes.
ren was also where I finally learned RAG properly
My first experience with something RAG-adjacent was much jankier.
I was building my own AI harness and gave the model web search. It could search, fetch pages, and shovel useful text into context.
Which works.
But that is not the same thing as sitting down and building retrieval as an actual subsystem.
ren uses MedCPT-style biomedical embeddings and combines vector retrieval with symbolic information. LanceDB handles vector storage/search. SQLite carries structured local data. NetworkX handles graph relationships.
That hybrid approach became fairly obvious once I started working with ontology data.
Semantic similarity is useful when language is fuzzy.
It is considerably less impressive when the thing you are looking for has an exact identifier, known aliases, and parent-child relationships that your symbolic layer can just resolve deterministically.
So I use both.
The vector side can answer "what biomedical concept resembles this?"
The symbolic side can answer "mate, we literally know what this node is."
Turns out those complement each other rather nicely.
And because this was ren, naturally I could not just make retrieval work. I had to benchmark it, freeze it, pin things, and make sure future-me could reproduce the result without performing digital archaeology.
Which brings me to the project that permanently damaged how I develop software.
Lawliet made me empirical-pilled
Chess engine development does something to your brain.
You write an optimization.
It looks smarter.
It feels smarter.
You run a couple games and it wins.
Fantastic.
Then you run a statistically meaningful match and discover your brilliant idea is worth approximately nothing.
Or it is worse.
Lawliet got me used to SPRT, controlled test conditions, reproducible benchmarks, and generally refusing to believe an improvement just because I personally thought it was clever.
Unfortunately, I carried that disease into ren.
So what began as "haha, cool computational pathology side project" accumulated hashes, frozen artifacts, held-out evaluations, regression tests, browser scenarios, reproducible environments, and enough verification machinery that at some point I had to admit I was taking the dummy project rather seriously.
Good.
If ren were actually being put in front of professionals, would I want half the pipeline to be held together by "works on my machine"?
No.
Would I want a model checkpoint silently changing underneath an evaluation?
No.
Would I want an artifact mismatch to produce a plausible-looking output anyway?
Absolutely the hell not.
So ren tries to fail closed.
Important models and artifacts are pinned. Provenance is recorded. Frozen components stay frozen unless there is an explicit process for replacing them.
I also put the environment in a Nix flake because apparently simply using NixOS was not enough punishment.
But reproducible artifacts and reproducible environments belong to the same philosophy for me. If I cannot recreate the conditions behind a result, the result becomes less useful.
The final workstation eventually got its own full browser-level stability suite: 69 scenarios, 167 checks.
Import. Vision. Review. Retrieval. Evidence. Synthesis. Provenance. Sign-off. Export. Reloads. Failure paths.
The release run went 100/100 with no crashes, skips, unexpected failed requests, console errors, or HTTP 5xx responses.
I am aware this is an unreasonable amount of testing for something I originally built because computational biology looked cool.
I regret nothing.
The rough edges are still very real
ren is not magically "done."
The biggest thing annoying me right now is vision generalization.
I learned fairly quickly that "pathology images" is a hilariously broad category.
Blood-smear microscopy and whole-slide pathology are not interchangeable chunks of pixels. WSIs are enormous pyramidal images with their own tiling, viewport, storage, batching, and inference problems.
ren v1 supports blood-smear stills.
No WSI.
Yet.
I absolutely want to add it because apparently I dislike free time.
Unseen blood smears are another problem. ren is noticeably weaker once I throw images at it that sit outside the distributions and artifacts it was built around. The happy path has nice spatial findings and boxes around detected cells. An arbitrary unfamiliar smear does not suddenly acquire ground truth because I asked nicely.
That is probably the most important technical weakness I want to attack next.
The original pipeline was also supposed to be more automatic than it currently is.
All four big pieces exist:
Vision → Retrieval → Synthesis → Dashboard
but there are still seams where orchestration is more manual than I originally pictured.
That bothers me enough that I will almost certainly fix it.
I also want proper NVIDIA/CUDA support eventually.
Minor obstacle: I do not own an NVIDIA GPU.
And I refuse to put "CUDA supported" in a README because somebody on the internet said it probably works.
Lawliet brain damage strikes again.
Biomedical software is hard. Like, actually hard.
That is probably the main thing ren taught me.
The models are almost the easy part.
The annoying stuff is everything around them.
Data quality. Distribution shift. Reproducibility. Provenance. Model lifecycle. Retrieval behavior. Review semantics. Failure modes. UI state. Whether a technically valid result is actually sensible to show somebody. Whether you can explain where the result came from six weeks after you produced it.
I learned ViTs properly from this project.
I learned that blood-smear datasets and giant WSI datasets live in quite different universes.
I learned RAG beyond "LLM has a search tool now."
And, for reasons entirely unrelated to pathology, I learned a bunch about Git workflows because ren became large enough that my previous development style stopped scaling nicely.
That side quest eventually became an agentic Codex workflow with branches, PRs, isolated implementation agents, reviewer agents, verification passes, and artifact handoffs.
Basically I got annoyed enough to simulate a tiny software company inside my repository.
Normal behavior.
The thing I keep coming back to, though, is that ren stopped being interesting to me as a collection of ML models fairly early.
The models are replaceable.
That was the point.
What I actually ended up caring about was the system around them: the boundaries, the evidence, the review path, the reproducibility, and whether I could swap one component without turning the rest into spaghetti.
ren started because I wanted to screw around with computational biology and local inference on my own PC.
It ended up teaching me something much more useful:
making an ML model produce an answer is easy.
Making a system where you can explain why that answer exists, what produced it, what evidence went into it, what happens when something fails, and whether you should trust any part of it at all is the actual engineering problem.