Designing Software for Things that Rot

The white mold was good. The green-grey mold was probably fine. The fuzzy black spot was... well, that's when I ended up in a midnight rabbit hole of forum posts debating whether that particular shade of black meant penicillium or something that would send me to hospital. I had photos, I had descriptions, I had twelve different opinions from twelve different strangers on the internet.

What I didn't have, was the confidence. "Is that safe to eat?" my neighbour asked, eyeing the salami hanging in my garage. Fair question: it's raw meat that's been sitting at room temperature for six weeks.

That's when it clicked: I'd been treating fermentation like cooking when I should have been treating it like infrastructure. But how did I end up here in the first place?

First, there was hardware ¶

I started fermenting almost a decade ago, before sourdough starters became everyone's pandemic tamagotchi. You know the drill: shy attempts at sauerkraut, then mozzarella, then progressively weirder things and Sandor Katz books on the shelves. Tracking went from paper scraps to text files to an Obsidian vault. None of it helped because the problem wasn't tracking, it was knowing what to track and when it mattered.

Things escalated when I moved into a detached house–the kind where neighbors won't hear you checking on your sausage at 2 AM (which, reading that back, sounds worse than it is).

I bought a £150 larder fridge on eBay, added two Inkbird controllers, a reptile mat left after koji experiments, and went on a quest for a dehumidifier that resumes automatically. Most need a button press, which is useless for automation, and for some reason sellers never mention it in descriptions; cue exciting returns and very odd Amazon ads I keep getting three years later.

Temperature was easy. Humidity, not so much. Without a humidifier, the dehumidifier would overshoot and stall around ~75% RH, the case-hardening sweet spot where you build a meat Matryoshka you can't trust. The fix was bidirectional control: add moisture gently below target, remove gently above. Haven't had a case-hardening scare since.

Then, came the dashboard ¶

I already had Home Assistant running, so getting temperature and humidity from the chamber into a snippet took an evening. But I didn't have a way to know if what I was seeing was actually a problem.

Let's say, you're curing coppa and on day 12 humidity drops to 68% for six hours because you forgot to refill the reservoir. Is that fine? A disaster? The answer depends on cure phase, temperature, weight loss, and a dozen other variables. It's context, not a threshold.

Most of my "monitoring" was staring at graphs and trying to remember forum posts from three years ago. I had observability but no intelligence.

I read once: if you can measure it, you can manage it. But often a measurement without context is just noise.

What could go wrong? ¶

The wake-up call came with 'nduja–spreadable salami, heavy on fat and chillies, delicious, and risky because of moisture content and long fermentation. Reading through the process I found someone mentioning HACCP plans: Hazard Analysis and Critical Control Points, the food industry framework for identifying what can go wrong and how to monitor it.

I'd assumed it was bureacracy paperwork. Turns out it's an elegant decision tree. Enumerate every hazard, figure out which you can control, define critical limits, document monitoring.

For 'nduja: biological hazard is Salmonella, Listeria, Clostridium. Control point is pH drop. Critical limit is pH 5.3 within 48 hours. Monitor at 24h and 48h. Corrective action: if pH > 5.3 at 48h, discard.

Obviously kraut, kimchi, quick pickles are forgiving: they'll look and smell wrong if they fail. Things change with low-acid, low-oxygen, long cures of meat and things like garlic-in-oil, where something poisonous won't have off-smells or flavours. HACCP isn't paperwork; it's how you don't roll dice you can't see.

Building the decision tree ¶

Naturally, as an iOS developer I built a tracker. I started with projects and logs, then added "fermentation profiles" with hazards, limits, and check schedules.

The interesting bit was context: 65% humidity might be fine on day 30 but a problem on day 5, so the app has to know where you are in the process. Each fermentation profile is basically a small state machine where phases progress in order (ferment → dry → age), and each phase carries its own set of constraints that matter during that window.

struct Constraint {
  let metric: Metric          // pH, humidity, temp, weightLoss
  let range: ClosedRange<Double>
  let window: Interval        // e.g. 0..48h since phase start
  let severity: Severity      // info/warn/critical
  let action: () -> Void      // log verification / flag deviation
}

struct Phase {
  let name: String
  let constraints: [Constraint]
  let nextPhase: Phase?       // nil for terminal phase
  let transition: Transition  // time or value based
}

The first version checked every constraint on every measurement, which was fine for a dozen control points but felt wasteful. The fix was simple: when a batch enters a new phase, grab just the constraints relevant to that phase and cache them. Most phases only care about three to five things–during fermentation you're watching pH and temperature, during drying it's relative humidity and weight loss, during aging it's mostly "is anything growing that shouldn't be". So when a new measurement arrives you only validate against the cached constraints for the current phase.

Some checks are time-bound ("verify pH by 48 hours" for 'nduja), so I keep those in a sorted list of due tasks. Nothing fancy, just enough to know what needs checking next without scanning everything.

LeetCode youth finally paid off: turns out all those "rebalance a binary search tree" problems were preparing me for salami, not FAANG interviews.

Automating compliance ¶

Once checks worked, I figured I might as well generate the actual HACCP document, so the next time a neighbour suspeciously eyes me up and asks if the homemade Roquefort is safe to eat, I can pull up a binder that wouldn't look out of place in a Michelin kitchen.

Take the profile, enumerate hazards, build the control table, export as PDF. The useful bit was making it bidirectional–if you're logging anyway, the document fills itself. pH 5.2 at 48 hours? CCP-1 verified. Temperature excursion? Flagged with corrective note.

Could I have added blockchain for tamper-proofing? Sure. Am I going to give people that idea? Absolutely not.

A lot of tools in this space are beautifully precise and wonderfully narrow–great if you only brew beer, or only bake bread. I don't. I swing from grashopper garum to deer prosciutto to cedar mugolio, because you live only once and also have you ever tried to find a grasshopper garum in a corner shop? What I needed wasn't a tighter ruler; it was a truer diary.

Traceability over precision ¶

In the world where most apps chase precision, I only wanted traceability: a record of what really happened, not what should have.

HACCP matters for edge cases: long windows, low-acid environments, meat–anywhere botulism is theoretically on the table. Not everything I ferment needs this rigour. For the rest, just keep an honest log: what did I do, what did I measure, what did I observe.

I've got batches where the note says "forgot to log pH, but it smelled right." That's data. Photos often help more than numbers. "Slightly more sour than batch #3" beats pH to three decimals.

The manual reality ¶

Chamber monitoring runs in Home Assistant automatically. But the tracker doesn't talk to Home Assistant (yet?), so when I weigh a salami or measure pH I'm typing it in.

Dream setup: weigh the piece, measurement auto-logs, gets checked, flags me if the curve is wrong. But I don't have smart scales that talk to my phone, and even if I did I'd need to solve "which piece is on the scale". Barcode tags? RFID? A Vapor server with Apple's new Foundation models and VisionKit to run real-time OCR?

It might sound odd from someone who just spent a weekend reverse-engineering PureGym's API because I couldn't be bothered to remember an 8-digit PIN. But typing six numbers once a week? That's fine. Apparently my threshold is somewhere between "weekly number entry" and "daily PIN recall".

Designing for things that rot ¶

Web services are designed around uptime and consistency, i.e deterministic behaviour, same in test as production. Fermentation is the opposite: controlled drift where things change slowly within acceptable boundaries. Your job isn't to prevent change, it's to make sure it happens in the right direction at roughly the right speed.

It's all about watching trends, not absolute values. A 2°C spike for ten minutes is irrelevant, but a 0.5°C drift over a week matters. And you can't rollback. Once you've over-salted or let it dry too fast, that's it.

Every batch teaches you something, every deviation is data. I've got batches going back years: elderflower capers at ~7% brine (lower went mushy), ättika penny buns where blanch-then-brine kept their snap, dried barley koji where airflow beat temperature, duck prosciutto I ignored for ten days that still behaved, and the blue that didn't–because I didn't sanitise the box. All logged, all useful.

Designing software for things that rot means optimising for variance, memory, and timing–not perfection. It turns out the hardest part of software isn't keeping things alive. It's knowing when to let them age.

Smaller scale, same habits ¶

These days I'm settling in Canada with most of the gear in storage across the pond. No chamber, no controllers, no midnight humidity checks–just a jar of pinecones buried in sugar slowly turning into mugolio, some freshly picked BC chanterelles in a quick brine, my wife's sourdough starter living on the counter and getting fed when she remembers. The app reminds her; she ignores the app; the starter survives anyway because sourdough is remarkably forgiving.

Different scale, same principles: note what you did, watch what changes, let time do its thing. Turns out you don't need the chamber to benefit from the habits.

I ended up calling the tracker Fermento. It keeps track of salt percentages, culture lineages, HACCP plans. Mostly it just tries not to get in your way.

Most of what people ferment at home doesn't need HACCP plans–kombucha and pickles will tell you loudly if they've gone wrong. For those, Fermento is just a diary with reminders. But when you're doing long cures or low-acid environments where bad outcomes are silent, having the decision tree makes the difference. The app scales to what you're making.

If you've read this far, you're probably my kind of person. The app is here. It's free for basic tracking and HACCP generation is in the commercial tier ^[1].

The best incident reports end with dinner. May yours end with something cured, crunchy, or quietly fizzing.

Got weird hardware and weirder requirements? I make unreasonable things work on iOS. work@drobinin.com