When the alarm is armed, it reacts to person detections from my Frigate cameras. It missed people, and at night false detections set it off and caused quite some havoc at home, so my trust in the detections went down. Frigate runs on the same Raspberry Pi 5 as Home Assistant (I tuned it in January), with a Coral USB TPU doing the inference.
In August I pulled 30 days of person events to see why. SSD MobileDet, the model Frigate ships for the Coral, was never very confident. Nearly all person detections landed just above the cut-off, and a sun bed with a pale pillow under the IR lights got practically the same confidence as a real person (one night it stayed a “person” for almost six minutes). Raising the cut-off would drop real people as well, and lowering it would keep the sun bed. The cut-off alone could not fix it. Missing people and seeing people that are not there are two sides of the same problem, the model is weak and was not prepared for cameras mounted high and at an angle like mine.
YOLO isn’t new to me, I had Darkflow running light YOLO weights on a Pi 3B+ in 2018. Since version 0.17 Frigate can run YOLOv9 on the Coral, the detector code is upstream and I only needed the model file from dbro’s repo. First I tried the 512×512 version to see if the extra resolution helps detection. Inference went from 9.9 to 22–25 ms per frame and the detector CPU from 8% to 23–29%.
By the afternoon the Pi was struggling. Two cameras were pushing about 9 detections/s each, the Coral hit its ceiling of about 40 detections/s, host load climbed to 7–10 (against the usual 2–4) and frames got skipped. 512 probably detects better, but on my setup it is a no-go, so I switched to the 320 version of the same model. It runs at 10.6 ms, about what the old model cost, and I have not noticed a quality drop. The new model needed its own cut-off. I started permissive, and kept it strict on the two cameras that had given me the most false positives.
These are person detections per day, before and after:
| camera | SSD MobileDet | YOLOv9 |
|---|---|---|
| house-facing | 5.8 | 25.2 |
| backyard | 1.4 | 19.0 |
| front | 5.0 | 11.9 |
| shack | 3.8 | 7.5 |
| car shade | 6.3 | 7.1 |
*Not all cameras are included; some saw less of an improvement.
More detections could just as well mean more false alarms. I had Claude Code pull and label 120 snapshots, weighted towards the detections the model was least sure about. All 120 were real people.
The extra detections are the people the old model missed, and getting those was the whole point for me. The gain is biggest on the angled cameras, with people crouching or bending and seen steeply from above. The sun bed hasn’t come back either (I added a size filter on that camera just before the swap, so the model does not get all the credit).
At night, when a camera spots a person, the outside lights for that part of the yard turn on, so the camera gets a lit picture instead of IR to confirm what it sees. The Coral is a legacy accelerator by now and any of the newer ones would run the 512 model with ease, but for my setup 320 on the Coral does the job, and I am very happy with it. Highly recommended if your Frigate runs on a Coral as well.