My Phone Replaced a Brass Plug

For months, I spent my Wednesday evenings in a tin tunnel just outside Edinburgh, wearing a ridiculously looking (and equally uncomfortable) jacket. I'd lie on the floor and count breaths, then walk down the range, ducking under ceiling beams. The floor says DUCK in white paint every five metres, the beams have posters saying "DUCK" as well, but occasionally I still hit my head, too busy checking the scoring cards.

If you're unlucky, a shot lands near a ring line and you need help. You walk up to a tray of Greggs sausage rolls^[1]1. Best gourmet pastries this side of the pond (and they also do doughnuts!). - we're in the North, and so are our sponsors - find a wooden box which holds brass plugs in every size, choose the right one, carefully push it into the hole (ideally only once, to avoid tearing), and where it sits is your score.

The shooting part is fun. The score-counting-head-hitting-plug-pushing ritual had to end.

The reason I was there was cooking.

I got into it decades ago and gradually became more obsessed: from shy attempts at recreating dishes from every fine-dining restaurant I'd visited to building automated curing chambers. Not buying koji but growing the mold, hydrating ramen dough in a chamber vacuum, heating protease and grasshoppers in an immersion circulator to make garum.

Then I got into charcuterie, which meant getting whole animal carcasses and butchering them myself. As I decided to get serious about cooking meat I figured I should learn to hunt. I'd never really held a gun, and while in the UK we love licences and don't like guns (we prefer knives), deer hunting requires neither a hunting nor a rifle licence^[2]2. The Firearms Act lets a landowner hand you one of theirs if they "supervise" you using it, which is how folks have hunted on their estates for centuries (on that note, it's deer stalking - hunting is for rich twats on horses, shooting is for rich twats in tweeds).. Red deer are essentially pests - they eat woodland faster than it regrows and have no natural predators, so culling them comes with almost no restrictions.

You do need the rifle though, and preferably know how to use it^[3]3. I shot myself in the foot too many times writing code, imagine what I could do with a firearm. - so there I was, on a mat twice a week. Not quite the same discipline as stalking a deer: shoot, change cards, have a doughnut, repeat. Half a year later I had gained a few pounds on my way to a venison steak I was yet to shoot, and spent most evenings searching for the right-sized scoring gauge.

Bored as I was, I figured I might as well automate it.

Negative space ¶

I am an iOS engineer, so I started with vanilla iOS: Apple's Vision framework is around for a while, and has ready-to-use detectors for objects, person segmentation, text recognition, and even barcode scanning, but it kept tagging random parts of the image as bullet holes - from the dot in the target's centre to pieces of one of the scoring rings.

A bullet hole is a negative space. Object detectors are trained on the thing that should be there, so it's not very straightforward how to use them for finding something that was there before and then got removed.

I tried a few more obvious things: grayscale, inverting the image, adding and removing noise, but even when everything else worked, shots landing on ring lines were turning into fragments too small to register.

A better approach would be to treat the target as an object with known geometry: find the ring structure first, then look for holes inside it. I accepted I won't be reinventing the wheel this time, and looked up alternatives.

Port and whatnot ¶

A 2012 paper ¶

Scored shooting targets are boring enough as a computer-vision problem that somebody has published on it. I found Automatic Scoring of Shooting Targets with Tournament Precision, by Rudzinski and Luckner at Warsaw University of Technology, published in 2012 and promising 99% of holes detected.

There were a few caveats: the approach was optimised for low-resolution pictures but required flat ISSF targets ^[4]4. Comparing to NSRA, ISSF targets have different size of bulls and amount of rings, different rings, and different background
, low camera angle, annotations not similar to holes, and was generally designed for pellet shooting. Air-rifle pellets make a cookie-cutter hole in paper, while a .22 bullet at twenty-five yards leaves ragged edges.

I reproduced the paper step-by-step. If you don't fancy reading the publication, it boils down to four steps: erase the ring lines, flood-fill to find the hole shapes, run a Prewitt edge detector, and fit circles with a Hough transform.

Vision framework doesn't have a Prewitt edge detector, so I brought in OpenCV as well, and the first three steps worked well. But step four has a catch. NSRA cards print ring scores at the cardinal points - a "9" north, east, south, and west of the 9 ring - and Hough fits circles to those digits too.

Also occasionally the crescents after ring erasure would be too small for flood-fill, so I ended up using a V-value radial-intensity profile - pick a strip from the bull's centre outward, sample brightness along it, and look for the spikes where the white ring lines cross. The spike positions are the ring radii.

Bundled with Vision's VNDetectContoursRequest and a perimeter filter, this got me an average of four shots per card of five - so 80% accuracy, still a long way to go, and we hadn't even got to edge cases like overlapping shots yet.

Want Vision & CoreML in your app too? Let's chat →

Adding machine learning ¶

Shooters do things to target cards. They put names, add dates, circle around close shots. Most cards have torn staple holes, sometimes multiple bullets tear the paper so close to each other, it turns in a single flood-fill region.

My first attempt was to add an heuristic for each, come back every week with another set of shot cards, and keep tuning again, but that was hardly scalable and/or sustainable.

So I went back to Google and came across a paper published in late 2023 ^[5]5. Z. Ali et al., "Application of YOLOv8 and Detectron2 for Bullet Hole Detection and Score Calculation from Shooting Cards", AI, vol. 5, no. 1, 72-90, 2024. 10.3390/ai5010005.. The authors promised 96.5% average precision but focused on hole detection and read scores off the bounding-box class.

I couldn't afford manually preparing bounding-box classes for my cards, but I already had a working geometry, thanks to Rudzinski and Luckner. What I didn't have was reliable hole detection. The YOLO paper did the hole detection well but left geometry to an assumption that the card is perfectly aligned.

Naturally, I merged two approaches: OpenCV does the structural geometry - bulls, ellipses, rings, the perspective transform. YOLOv8 does hole localisation, same architecture as the MDPI paper, fine-tuned on my own dataset. The learned model's class prediction is discarded at inference; score comes from distance to the bull centre compared to geometric ring radii.

Once labelled and trained, I used coremltools to export the model to CoreML - the final package weighs 22.4 MB after Xcode imports it.

Scoring ¶

Mapping back ¶

With both pieces wired up, I expected scoring to be the easy bit. Mostly it was, except I kept losing time to the coordinate spaces and completely forgot about perspective.

Vision returns bounding boxes in 0-1 with the origin at the bottom-left but UIKit has the origin at the top-left, so it's very easy to get off-by-a-ring errors that look plausible enough to blame anything but the transformation method.

Once the coordinates line up, the math looks simple. The scoring gauge exists for a reason though - the detected bullet hole is smaller than the bullet that made it, because paper tears and gets pushed inwards. CoreML returns a bbox around the visibly torn paper, not the bullet, so for scoring I need the centre of the hole plus the bullet's radius - the furthest point the bullet reached.

Bullet radius ¶

A .22 bullet is 0.22" across (duh), and an NSRA .22 card is 2.05", so geometrically the bullet radius should be 10.87% of the bull diameter.

The paper doesn't tear that cleanly though. I also did terribly bad in both physics and machine learning in the past, so after re-reading both papers I gave up and started to tune the multiplier empirically - that is, changing the constant and re-running tests on cards I scored manually. 30% is what worked for me (so 14.13% of the bull's diameter), but I'd love to learn a proper way to do this - please drop me a message if you know it.

Beyond the gauge ¶

Six months in, I still think I am bad at shooting but now most of the time I know why - posture, trigger pressure, or breathing through the shot. These are things you feel rather than measure though, which is why beginners shoot grouping cards in the first place. The shape of the group on a card tells you a lot - from common issues like trigger pull and breathing through shot to issues with the rifle ^[6]6. A clean tight group sitting a few rings off the bull usually means you'd hit the bull perfectly if it wasn't for the tune - an explanation I find very comforting (and my mates disagree with)..

Competition cards are shot bull-by-bull in a known order, so once you know which bull went first you can plot accuracy against position to spot trends. I often see myself drifting in the middle and then tightening back up at the end, when I notice the card is running out.

I embarked on this quest trying to bring down the gauge (and stop bringing home piles of shot cardboard), but by the time the scoring worked I got way more interested in trying to automate not only the scoring but the feedback - after a couple of months' worth of scored cards I could stack all shots on a cumulative heat map to see the trends, or compare my performance across all four communal rifles.

I could even finally prove that eating a doughnut minutes before my turn on average drops my performance by 7 points out of 100 - this one might be placebo, but my working theory is that it raises my sugar levels (as you can tell, I am not only doing bad at physics but also at biology - I am doing my best though).

I ended up wrapping it up as a small offline-first app - originally for my mates back at the club, but really for anyone keen to make their cooking shooting routine a bit more fun. I did learn that most of the world doesn't care much for NSRA targets though, so I keep slowly adding support for other disciplines and cards.

I left for Canada before I felt confident enough to go deer stalking.

Somebody else lies on that mat^[7]7. I'll be back to it eventually.
in Prestonpans now on Monday and Wednesday evenings, ducking the same beams and counting breaths between cards. The wooden box of scoring gauges is still on the table - most of the lads said it's not the first time someone has reckoned they could do better than the good ol' brass plug.

The scoring gauge's real job is settling disputes. When two shooters disagree on a borderline shot, the score is whatever the gauge says it is, because that's the rule. Neither of them care for my state-of-the-art computer vision models.

The chances are, you've got your own equivalent somewhere - a tool that's been doing its job for longer than you've been around and would be silly to try to replace.

We might fail to retire them. But I aspire to build things that end up on somebody else's table twenty years from now, lasting long enough that somebody's silly enough to try to replace them too.

Fancy adding an on-device ML to your app? I can help →