[{"content":"A clean ClamAV scan means nothing matched a known signature. It does not mean the file is safe. I run a scan gate in front of my media-download pipeline: everything that lands from the download clients gets checked by a ClamAV daemon before it\u0026rsquo;s allowed into the library. For a long time I treated a clean verdict as the end of the question. It isn\u0026rsquo;t. ClamAV is a signature engine, and signature engines only catch what someone has already seen, fingerprinted, and shipped a rule for. Zero-days and packed or obfuscated executables walk right past it, and the gap is worse than it sounds because ClamAV is open source. Anyone can download the exact detection logic and test their malware against it before release. That\u0026rsquo;s not a hypothetical; researchers have measured samples built specifically to dodge open-source detectors evading ClamAV on the order of 70 to 85 percent of the time, without even needing inside knowledge of the engine.\nHere\u0026rsquo;s the full layered gate, in the order a file actually passes through it:\nflowchart TD A[File lands from download client] --\u003e B[clamd signature scan + extension blocklist] B --\u003e C[PUA detection: DetectPUA flag] C --\u003e D[\"Third-party signature feeds:Sanesecurity, SecuriteInfo, URLhaus, MalwarePatrol\"] D --\u003e E[YARA-Forge Core rules, native in clamd] E --\u003e F{Borderline verdict?} F --\u003e|Yes| G[\"SHA-256 hash lookup:VirusTotal / MetaDefender, hash only\"] F --\u003e|No| H[Entropy / packer check: Detect It Easy] G --\u003e H H --\u003e I[Verdict: clean / flagged / infected / blocked] Signature scanning only catches what\u0026rsquo;s already been seen # Every ClamAV signature exists because someone already found and analyzed that malware sample. A brand-new keygen or crack, repacked or lightly modified, has no signature yet and sails through clean. Packed and obfuscated binaries make this worse: the payload is scrambled until runtime, so a static signature scanner has nothing to match against even for a known threat. My original scan gate had one static layer: clamd plus a blocklist on file extensions like .exe, .scr, .bat, and a handful of others. That layer stops the laziest attacks and nothing else. The real threat model for a media pipeline isn\u0026rsquo;t a nation-state implant. It\u0026rsquo;s commodity crack and keygen malware bundled into an executable that a downloader was told to run, and that category is exactly the kind of thing built to slip past exactly this kind of scanner.\nPUA detection targets the actual threat, with a real tradeoff # ClamAV has a flag, DetectPUA, that flags potentially unwanted applications: adware, riskware, and, most relevant here, keygens and cracks. Turning it on is a one-line config change to clamd.conf, no code touched. It\u0026rsquo;s also not a free lunch. PUA signatures are less rigorously curated than core malware signatures, so expect more false positives on legitimate but aggressively-bundled installers. ClamAV\u0026rsquo;s own category-exclusion filtering for PUA is currently broken in the shipped version I\u0026rsquo;m running, so I can\u0026rsquo;t cleanly say \u0026ldquo;flag keygens but ignore adware\u0026rdquo; and trust the exclusion list to hold. I\u0026rsquo;m turning it on anyway and tuning against real false positives as they show up, because the alternative is leaving the single most on-target detection knob switched off.\nThird-party signature feeds close known gaps for free # ClamAV\u0026rsquo;s own database misses a lot that other groups have already catalogued. clamav-unofficial-sigs is a maintained aggregator that pulls in Sanesecurity, SecuriteInfo, URLhaus, and MalwarePatrol feeds and drops them straight into the same database directory clamd already reads. No changes to my scan gate\u0026rsquo;s code, no new dependency in the pipeline logic. It\u0026rsquo;s a cron job and a shared volume. This is the highest ratio of detection gained to effort spent of anything I added, because it\u0026rsquo;s pure config and it widens the signature set clamd already checks against on every scan.\nYARA rules run inside clamd, but only the trimmed kind # Clamd loads .yar files natively from the same database directory, no separate engine required, and gets to apply YARA rules against files it has already unpacked from archives and installers. That\u0026rsquo;s a real advantage over running YARA standalone, since clamd\u0026rsquo;s decomposition sees inside the containers a raw file scan would miss. The catch is that clamd\u0026rsquo;s YARA support is a subset of full YARA: no imports, no external variables, a 64-string cap per rule, minimum two-byte string segments. Community rule packs written for full YARA often won\u0026rsquo;t load as-is. I\u0026rsquo;m using YARA-Forge\u0026rsquo;s curated \u0026ldquo;Core\u0026rdquo; tier instead of pulling raw rules from wherever, because unvetted community rules have a documented history of tanking scan performance: one bad rule reportedly took a three-hour scan job to seven. Curation here isn\u0026rsquo;t optional polish. It\u0026rsquo;s the difference between a scan gate that finishes and one that doesn\u0026rsquo;t.\nA hash lookup adds a second opinion without uploading anything # Signature and YARA scans both run locally against files I already have. A hash lookup asks a different question: has anyone else already seen this exact file and scored it? I compute a SHA-256 of anything the local scan flags as borderline and check it against VirusTotal\u0026rsquo;s or MetaDefender\u0026rsquo;s free tier, hash only, never the file itself. That distinction matters for a pipeline that occasionally handles cracked software. Uploading the actual file to a public multi-scanner makes it permanently visible and searchable by anyone, which is exactly the kind of exposure I don\u0026rsquo;t want for downloads that were never meant to be public. This isn\u0026rsquo;t shipped in my scan gate\u0026rsquo;s code yet. It needs a new verdict state that plugs into the same aggregation logic the gate already uses, so a \u0026ldquo;flagged, pending second opinion\u0026rdquo; result sits in the same priority chain as infected, blocked, and clean.\nEntropy and packer detection catch what hashes can\u0026rsquo;t # A hash lookup only works if someone else has already seen the file. A packer or entropy check doesn\u0026rsquo;t need that. Detect It Easy, and its CLI diec, identifies packers and protectors on executables and reports per-section Shannon entropy. A section reading above roughly 7 bits of entropy is the standard first signal that it\u0026rsquo;s packed or encrypted rather than plain code. This is a heuristic, not a verdict, and I plan to route it to quarantine-and-alert rather than a silent auto-block, because plenty of legitimate installers are also highly compressed and I don\u0026rsquo;t want to nuke a real release over a false positive I can\u0026rsquo;t explain later.\nWhat I\u0026rsquo;m deliberately not building # A self-hosted dynamic-analysis sandbox, actually detonating suspicious files in an isolated VM to watch what they do, is technically doable in a home lab. CAPEv2 runs fine on a single box with nested virtualization. I\u0026rsquo;m not building it, because it\u0026rsquo;s a heavyweight answer to a threat model that\u0026rsquo;s mostly commodity keygen and crack malware, not a targeted attacker who needs behavioral analysis to unmask. If one of the layers above misses something in an actual incident, that\u0026rsquo;s the trigger to revisit sandboxing. Building it preemptively against a threat that doesn\u0026rsquo;t need it is effort spent on the wrong risk.\nThe honest residual gap # None of this closes the gap completely, and I don\u0026rsquo;t think any config change could. A sufficiently novel packer that mimics legitimate compression entropy, paired with a payload built against ClamAV\u0026rsquo;s public signature set and PUA rules specifically, can still get through every layer I\u0026rsquo;ve described. The hash lookup only helps once a file is already known to someone; a first-seen sample gets a pass there by definition. What changed isn\u0026rsquo;t that my scan gate is now airtight. It\u0026rsquo;s that I stopped treating a clean verdict as proof of safety and started treating it as one data point among several, none of which is trustworthy alone. That\u0026rsquo;s a more honest place to operate from, even if it\u0026rsquo;s a less comfortable one.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/clamav-clean-scan-doesnt-mean-safe/","section":"Blog Posts","summary":"A clean ClamAV scan means nothing matched a known signature. It does not mean the file is safe. I run a scan gate in front of my media-download pipeline: everything that lands from the download clients gets checked by a ClamAV daemon before it’s allowed into the library. For a long time I treated a clean verdict as the end of the question. It isn’t. ClamAV is a signature engine, and signature engines only catch what someone has already seen, fingerprinted, and shipped a rule for. Zero-days and packed or obfuscated executables walk right past it, and the gap is worse than it sounds because ClamAV is open source. Anyone can download the exact detection logic and test their malware against it before release. That’s not a hypothetical; researchers have measured samples built specifically to dodge open-source detectors evading ClamAV on the order of 70 to 85 percent of the time, without even needing inside knowledge of the engine.\n","title":"A Clean ClamAV Scan Doesn't Mean the File Is Safe","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/ai-agents/","section":"Tags","summary":"","title":"AI Agents","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/ai-infrastructure/","section":"Categories","summary":"","title":"AI Infrastructure","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/ai-infrastructure/","section":"Tags","summary":"","title":"AI Infrastructure","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/automation/","section":"Tags","summary":"","title":"Automation","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/blog/","section":"Blog Posts","summary":"","title":"Blog Posts","type":"blog"},{"content":"Anthropic will not tell you how many tokens or messages Claude Max 20x actually gives you, and I had to build a throttle for it anyway. I run several personal research projects on background schedules through claude -p, unattended fires that call Claude Code from cron and systemd timers while I\u0026rsquo;m not watching. Those fires draw from the exact same quota as the interactive Claude Code sessions I use to do actual work. If a background job burns the pool at 2pm, my 2:15pm session pays for it. I wanted those jobs to back off automatically as usage climbed, and hand back the room the moment I sat down to work. The problem is that Anthropic gives you nothing to calibrate that against.\nAnthropic publishes a ratio, not a ceiling # The Max plan documentation defines the 20x tier as \u0026ldquo;20 times more usage per session than the Pro plan,\u0026rdquo; and that\u0026rsquo;s the entire spec. No token count, no message count, no per-window number anywhere in Anthropic\u0026rsquo;s own docs. Everything else is qualitative: usage scales with conversation length, model choice, and effort level, and none of those get a formula. I went looking for a hidden number to hardcode against and confirmed there isn\u0026rsquo;t one, at least not one Anthropic publishes.\nThat absence isn\u0026rsquo;t a documentation oversight. It\u0026rsquo;s a load-bearing consequence of the ceiling itself moving. Anthropic doubled the 5-hour rate limit for Claude Code on Pro, Max, and seat-based Enterprise plans on 2026-05-06, and removed a peak-hour reduction that had applied to Pro and Max accounts on the same date. Any number I\u0026rsquo;d baked into a governor before that date would have been wrong the moment it shipped, silently, with no changelog entry pointing at my config file. A governor built against a fixed assumed ceiling is a governor built to go stale.\nOne pool, two independent windows, and a hidden sub-cap # The quota itself isn\u0026rsquo;t even one thing to track. Usage across claude.ai, Claude Code, and Claude Desktop draws from a single shared pool. Anthropic states this directly, and it\u0026rsquo;s the fact that makes the whole problem real, because it means a scheduled background fire and an interactive session are actually competing resources, not two separate budgets I could reason about independently.\nOn top of that shared pool sit two rolling windows that reset on different clocks. A 5-hour session window resets from the timestamp of your first prompt, not wall-clock time, so two people starting a session an hour apart are on different reset schedules even on the same day. A weekly cap sits above that, and it isn\u0026rsquo;t one number either: there\u0026rsquo;s an all-models weekly sub-cap and a separate, narrower Sonnet-only weekly sub-cap layered inside it. A governor that only watches the 5-hour window will run headlong into the weekly Sonnet cap with no warning, because that cap can bind long before the session window ever does.\nThe acceleration limit rules out a hard stop-start throttle # The design constraint that changed my approach most came from a practitioner writeup on Claude Code rate limits, not from Anthropic\u0026rsquo;s own docs: Anthropic\u0026rsquo;s rate limiter applies something like an acceleration limit, where a sharp spike in request volume can trigger a 429 even with headroom remaining in the steady-state window. That rules out the simplest version of a governor: check remaining budget, run at full speed until the number hits zero, then hard-stop. A background job snapping from idle to full concurrency the instant a window opens looks exactly like the kind of spike that limiter is built to catch, quota headroom or not. The governor has to ramp cadence down and back up gradually, on both ends, not flip a binary switch.\nWhat I actually built # The governor is a budget-aware layer that sits on top of the timing logic I already had for scheduling background campaigns, rather than replacing it. It reads from a local SQLite corpus that already tails Claude Code\u0026rsquo;s own transcript files, and from that it computes two rolling figures continuously: weighted token consumption over the trailing 5 hours, and the same over the trailing 7 days, combining interactive and automated usage together since they draw from the same pool. As either figure approaches its ceiling, the governor ramps down the cadence of scheduled claude -p fires (not the fires I\u0026rsquo;m running interactively, only the automated ones), targeting no more than 98% utilization of whatever ceiling it\u0026rsquo;s currently tracking. That reserves roughly 2% of headroom specifically so an interactive session I start doesn\u0026rsquo;t land on an already-exhausted window. As usage clears on either rolling window, cadence ramps back up, on the same gradual curve rather than snapping back to full speed.\nHere\u0026rsquo;s the loop the governor actually runs:\nflowchart TD A[\"Claude usage: interactive + claude -p, shared pool\"] --\u003e B[Track 5hr rolling window] A --\u003e C[Track 7-day rolling window] B --\u003e D{Approaching ceiling?} C --\u003e D D --\u003e|Yes| E[\"Ramp down claude -p cadence,target 98% utilization\"] D --\u003e|No| F[Ramp cadence back up, gradually] G[429 response received] -.-\u003e|calibrates working ceiling| DThe \u0026ldquo;ceiling it\u0026rsquo;s currently tracking\u0026rdquo; part is the honest workaround for not having a real number. Since Anthropic doesn\u0026rsquo;t publish one, the governor treats its threshold as calibrated, not assumed: when a claude -p fire actually gets rate-limited, Claude\u0026rsquo;s own error response carries a reset timestamp, and the governor parses that as ground truth and adjusts its working ceiling estimate from it. Absent a fresh 429 to calibrate against, it falls back to a conservative default rather than guessing high. It\u0026rsquo;s closer to an adaptive controller reacting to real signals than a static budget checked against a spec sheet, because there is no spec sheet.\nI wired the throttle into the two places that actually spend tokens unattended: a scheduled research campaign that fires on a timer, and a document-ingestion pipeline where the lever isn\u0026rsquo;t fire frequency but concurrency: how many ingestion workers run in parallel against the shared quota. Those are shaped differently enough that the governor treats them as separate levers under the same budget rather than one input. I also checked a third scheduled job that looked like a candidate and found it makes no LLM calls at all. It\u0026rsquo;s a deterministic RSS collector, so it was never competing for the quota in the first place, and I excluded it rather than throttling something that didn\u0026rsquo;t need throttling.\nWhere I think this could be wrong # The strongest argument against building any of this is that I might have solved a problem that a much dumber approach handles just as well. A purely reactive design (let jobs run at full speed, catch the 429 when it happens, back off with exponential jitter, retry) needs a fraction of the code and doesn\u0026rsquo;t require guessing at a ceiling that keeps moving anyway. I built the proactive version because I wanted to protect interactive sessions from ever seeing a 429 in the first place, not just recover gracefully after one, but I can\u0026rsquo;t prove that protection is worth the complexity it costs. It\u0026rsquo;s possible the reactive fallback alone would have covered 90% of the actual harm.\nThe number I\u0026rsquo;m least confident in is the 2% headroom target itself. I picked it because it felt like enough margin without leaving obvious quota on the table, not because I derived it from anything. Since Anthropic doesn\u0026rsquo;t publish the real ceiling, I have no way to check that 2% against ground truth. I can only watch whether interactive sessions still hit limits in practice and adjust after the fact. That\u0026rsquo;s the same calibration-from-observed-429s approach the governor itself uses internally, which means the whole system, including the part of it that\u0026rsquo;s supposed to be doing the calibrating, is ultimately tuned against my own incomplete observations rather than a documented spec. I\u0026rsquo;m comfortable shipping that. I\u0026rsquo;m not comfortable calling it settled.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/self-throttling-claude-max-without-a-published-ceiling/","section":"Blog Posts","summary":"Anthropic will not tell you how many tokens or messages Claude Max 20x actually gives you, and I had to build a throttle for it anyway. I run several personal research projects on background schedules through claude -p, unattended fires that call Claude Code from cron and systemd timers while I’m not watching. Those fires draw from the exact same quota as the interactive Claude Code sessions I use to do actual work. If a background job burns the pool at 2pm, my 2:15pm session pays for it. I wanted those jobs to back off automatically as usage climbed, and hand back the room the moment I sat down to work. The problem is that Anthropic gives you nothing to calibrate that against.\n","title":"Building a Self-Throttling Governor for Claude Max With No Published Ceiling","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/clamav/","section":"Tags","summary":"","title":"ClamAV","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/claude-code/","section":"Tags","summary":"","title":"Claude Code","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/code-quality/","section":"Tags","summary":"","title":"Code Quality","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/code-review/","section":"Tags","summary":"","title":"Code Review","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/cost-engineering/","section":"Tags","summary":"","title":"Cost Engineering","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/debugging/","section":"Tags","summary":"","title":"Debugging","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/developer-workflow/","section":"Tags","summary":"","title":"Developer Workflow","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/devops/","section":"Categories","summary":"","title":"DevOps","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/docker/","section":"Tags","summary":"","title":"Docker","type":"tags"},{"content":"Fifteen of eighteen root causes I proposed for four firing alerts turned out to be wrong. Four alerts were going off across my home infrastructure at once: a stuck download post-processing backlog, and three separate automation alerts tied to a video-discovery pipeline. My first instinct on each one was the usual instinct: form a theory fast, patch it, watch the alert clear. Instead I forced myself to generate every plausible root cause I could find, then attacked each one before touching anything. Eighteen candidates went in. Three survived. That refutation rate is the actual finding here, more than any single bug I fixed.\nThe method was adversarial verification, not more logging # Adversarial verification means you treat your own hypothesis as something to disprove, not something to confirm. For each candidate root cause, I ran three independent checks against different failure modes: is the claim actually correct, is there a more likely alternative explanation for the same symptom, and would acting on this fix cause harm even if the diagnosis were right. If two of the three checks came back negative, the finding was refuted and I moved on without touching code. I used parallel background agents to run these checks concurrently, one per lens, working off the same evidence but arguing independently. The mechanism doesn\u0026rsquo;t matter much. You could do this with three colleagues, or with yourself on three separate days. What matters is that confirmation and refutation are different jobs, and doing them with the same brain in the same sitting is how bad root causes survive into production.\nHere\u0026rsquo;s how the 18 candidates actually funneled down:\nflowchart TD A[18 candidate root causes] --\u003e B[3 independent adversarial checks per candidate] B --\u003e C{2 of 3 checks negative?} C --\u003e|Yes, 15 candidates| D[Refuted - no action taken] C --\u003e|No majority reached, 1 candidate| E[Left open - reviewers split, no coin flip] C --\u003e|No, holds up, 2 candidates| F[Confirmed - acted on]The whole investigation stayed read-only until every surviving finding cleared verification. No config edits, no restarts, no \u0026ldquo;let me just try this\u0026rdquo; during the diagnostic pass. That discipline is what made the refuted list trustworthy. I never contaminated a measurement by fixing something mid-investigation.\nA refuted finding: zero didn\u0026rsquo;t mean what I thought it meant # Earlier in this same session, before I tightened up the process, I had already reported that a download client\u0026rsquo;s bandwidth was pinned at 0 B/s and blamed an empty configuration value colliding with a governor script that writes percentage-based limits. That looked like an obvious bug. It would have been an easy one-line fix: set the missing value.\nIt was wrong, and setting that value would have made things actively worse. I traced the actual code path and found that in this particular download client\u0026rsquo;s percentage-limit branch, a zero limit means unlimited, not stopped. The log line that reads like a stall is literally the client\u0026rsquo;s own phrasing for \u0026ldquo;no cap applied.\u0026rdquo; I confirmed this three separate ways, including running the branch logic directly inside the container and cross-checking it against a measured throughput number that only made sense if the download was, in fact, running at full speed. Setting the value I\u0026rsquo;d flagged would have flipped the client into a different code branch entirely, one that computes a mismatched percentage on every release cycle and throws a runtime error every time. I would have taken a healthy, fast-running download client and broken it on my own advice, in service of \u0026ldquo;fixing\u0026rdquo; something that was never broken. That\u0026rsquo;s the kind of mistake adversarial verification exists to catch, including from yourself an hour earlier.\nA confirmed finding: the alert metric was lying about its own units # One of the four original alerts was measuring how long the oldest item had been stuck in the post-processing queue. The number it reported never looked right. It read low even when I could see items sitting untouched for days. The bug turned out to be in how the metric collector seeded its internal clock: it stamped each item\u0026rsquo;s \u0026ldquo;first seen\u0026rdquo; time from the moment the collector itself first observed it, not from when the item actually entered the queue. Every entry read back the exact same duration, no matter how long it had really been waiting, because the whole gauge was secretly measuring collector uptime.\nThat one survived all three checks cleanly. The alternative-cause reviewer couldn\u0026rsquo;t find a queue-processing explanation that fit the flat, identical readings across separate instances. The fix-safety reviewer confirmed the correct source of truth was already present in the underlying data and just needed to be read instead of guessed. After I re-seeded the clock from the real timestamp, the two queue instances immediately started reporting different, correct numbers: one nearly four days old, the other over a day and a half. The alert had been reporting a real problem\u0026rsquo;s existence without ever reporting its true severity, for as long as it had been deployed.\nA second confirmed finding was worse than the alert it was supposed to fix # A budget governor script was supposed to reduce how often a discovery pipeline fired, to stay under a resource cap. Its \u0026ldquo;reduced\u0026rdquo; setting was implemented as a scheduling override applied on top of the baseline schedule. The override mechanism in the underlying scheduler doesn\u0026rsquo;t replace an existing schedule when you add to it that way. It appends. So the \u0026ldquo;reduced\u0026rdquo; tier was adding a second firing schedule on top of the first one instead of replacing it, and the lever meant to cut cadence was quietly increasing it. Separately, a blank scheduling directive left in one code path caused the whole timer unit to fail to load at all, silently, with no warning that it had been disabled rather than paused. Both bugs shipped together and had been live long enough that nobody would have found either by reading the code once and moving on.\nWhat this cost, and what it still couldn\u0026rsquo;t tell me # Running eighteen hypotheses through three-lens verification is not fast. It took a long investigation session, and most of the eighteen candidates burned real analysis time before getting refuted. That\u0026rsquo;s the tax you pay for not shipping a plausible-sounding fix on the first guess. I think it was worth it here, mostly because two of the three survivors were actively harmful if left alone, and the one I would have shipped from my earlier, faster pass would have made a healthy system fail on the next release cycle.\nThe process also has a real blind spot. One finding, a file-permission mismatch behind a wave of import errors, split my reviewers. One found evidence the bad files existed for hours before the failures started; another found the same failures beginning within minutes of a container restart despite those files already being in place. Majority-refutation needs an actual majority, and a genuine split doesn\u0026rsquo;t produce one. I left that finding open rather than act on a coin flip, which is the right call, but it means adversarial verification didn\u0026rsquo;t resolve it. It just kept me from pretending it had. The backlog itself is also still draining slower than it should, and I haven\u0026rsquo;t traced a single item through the pipeline start to finish to prove why. Eighteen hypotheses in, some things are still genuinely unknown, and the honest move is to say so instead of closing the ticket.\nThe point of this exercise was never about the agents. It was about building a process where a plausible root cause has to survive someone actively trying to kill it before I\u0026rsquo;m allowed to act on it. Fifteen didn\u0026rsquo;t survive. I\u0026rsquo;m glad I found out before I touched anything.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/adversarial-verification-home-lab-alerts/","section":"Blog Posts","summary":"Fifteen of eighteen root causes I proposed for four firing alerts turned out to be wrong. Four alerts were going off across my home infrastructure at once: a stuck download post-processing backlog, and three separate automation alerts tied to a video-discovery pipeline. My first instinct on each one was the usual instinct: form a theory fast, patch it, watch the alert clear. Instead I forced myself to generate every plausible root cause I could find, then attacked each one before touching anything. Eighteen candidates went in. Three survived. That refutation rate is the actual finding here, more than any single bug I fixed.\n","title":"Fifteen of Eighteen Root Causes I Was Sure About Were Wrong","type":"blog"},{"content":"Idle power draw, not the price tag on a mini PC, is what actually decides whether a dedicated low-power compute box saves you money over running everything on a gaming desktop. A gaming desktop idles somewhere around 80-200W depending on the board, PSU, and how many drives are spinning. A purpose-built low-power box, the N100-class mini PCs and similar, idles at 10-15W. That gap is real and it\u0026rsquo;s large. Whether it means anything for your electricity bill depends entirely on one question: does buying the mini PC let the desktop actually power off or sleep when you\u0026rsquo;re not gaming? If the answer is no, the math falls apart, and I found that out the hard way while pricing hardware for my own setup.\nThe wattage gap turns into real money over a year # Run the numbers and the case looks obvious. A desktop idling at 80-200W, left on continuously, costs roughly $200-460 a year in electricity depending on local rates. A mini PC idling at 10-15W costs roughly $20-43 a year for the same always-on duty. That\u0026rsquo;s a savings of $150-400 a year, enough to pay back a $300-500 mid-tier mini PC in 18-24 months, or a $90-110 used enterprise small-form-factor desktop in well under a year. On paper this is a fast, boring, obviously-correct upgrade.\nThe number only works, though, if the desktop\u0026rsquo;s power draw during \u0026ldquo;off\u0026rdquo; hours is actually the low idle number and not the number it draws while doing something. A desktop that\u0026rsquo;s rendering, transcoding, or serving requests around the clock isn\u0026rsquo;t idling at 80-200W, it\u0026rsquo;s running at whatever load those tasks add on top of that baseline. The savings calculation compares two idle states. If one of your machines never reaches an idle state, you\u0026rsquo;re not comparing what you think you\u0026rsquo;re comparing.\nBuying a mini PC doesn\u0026rsquo;t save power if the desktop stays on anyway # Here\u0026rsquo;s where my own case broke the clean version of this argument. My desktop wasn\u0026rsquo;t just gaming hardware sitting idle between sessions. It was already running 24/7 to serve a stack of self-hosted services: a media-automation pipeline, a personal trading-research pipeline, and a local broker that arbitrates GPU access for LLM inference. None of that stops when I\u0026rsquo;m not gaming. The desktop was never going to drop to a true idle state, let alone power off, regardless of what other hardware I bought.\nThat fact kills the power-savings case outright. Adding a 10-15W mini PC next to a desktop that keeps running at its existing load doesn\u0026rsquo;t subtract 80-200W from my bill, it adds 10-15W on top of what I was already paying. Total household power draw goes up, not down. Anyone pricing this decision purely on wattage needs to check their own uptime pattern first, because the entire payback calculation assumes the expensive box gets to power down once the cheap box exists. Mine didn\u0026rsquo;t, so I didn\u0026rsquo;t get that check to cash.\nThe whole decision comes down to one branch:\nflowchart TD A[Considering a low-power mini PC] --\u003e B{\"Does the desktop actuallyidle down or sleep today?\"} B --\u003e|Yes, it goes idle| C[\"Mini PC saves ~150-400 dollars/yearreal payback in 12-24 months\"] B --\u003e|No, runs 24/7 for other services| D[\"Mini PC adds 10-15W on toptotal household draw goes UP\"] D --\u003e E[\"Buy it anyway? Only for isolation/reliability,not for watts\"] The case for a dedicated box shifts to reliability once power savings are off the table # So why did I build one anyway? Once electricity cost stopped being the argument, the case for splitting workloads off the desktop moved to reliability, and that argument turned out to be stronger than I expected. Every driver update, every Windows patch, every game that wants a reboot to apply a change takes every hosted service down with it. A media pipeline and a trading-research pipeline don\u0026rsquo;t care about my GPU driver version, but they go offline anyway every time I reboot for one. Decoupling the services from the gaming machine means a driver crash or a game install no longer doubles as a service outage.\nSplitting the workloads also removes a category of risk that has nothing to do with watts: a misbehaving game, a bad driver, or a resource-hungry mod shouldn\u0026rsquo;t be able to starve a database import or a scheduled job of CPU and memory it needs. Contention on a shared machine is invisible until it isn\u0026rsquo;t, and I\u0026rsquo;d rather not find out about it during something time-sensitive. That\u0026rsquo;s a maintenance and stability argument, not a power argument, and it\u0026rsquo;s the one that actually justified the purchase in my case.\nGPU-bound work stayed on the desktop, and that\u0026rsquo;s a separate decision # I did not move everything off the desktop. Local LLM inference stayed exactly where it was, running through the existing GPU-arbitration broker, and that was a deliberate choice, not an oversight. VRAM, not CPU or system RAM, is the binding constraint for local LLM workloads, and VRAM contention with a running game is the one real risk in sharing a GPU between gaming and inference. Video transcoding and CUDA inference use physically separate silicon on the same card, so they mostly coexist fine; a game competing for the same VRAM pool is the actual failure mode to watch for.\nMoving LLM inference to its own hardware is a real option, but it\u0026rsquo;s a much bigger and separate spend. A dedicated inference-capable box, something like a Mac Mini M4 Pro with 48GB of unified memory or an AMD Ryzen AI Max+ box with 128GB, runs $600-2000 and only earns its keep under heavy or continuous inference load. Bundling that decision in with \u0026ldquo;buy a $300 mini PC for CPU-only services\u0026rdquo; muddies two questions that have different price floors and different payback conditions. I split them on purpose.\nWhat I\u0026rsquo;d actually check before buying # Check your desktop\u0026rsquo;s real uptime pattern before you check mini PC prices. If it\u0026rsquo;s already running 24/7 for reasons unrelated to gaming, buying a low-power box will not lower your electricity bill, and anyone telling you otherwise hasn\u0026rsquo;t looked at your actual load. The purchase can still be worth it, but the reason changes: you\u0026rsquo;re buying isolation and uptime, not watts. I ended up repurposing an old laptop I already owned as the dedicated box, running Proxmox, rather than buying new hardware, since the reliability case didn\u0026rsquo;t require the cheapest possible idle wattage, just a second machine that wasn\u0026rsquo;t also my gaming rig. If your desktop genuinely goes idle for long stretches, the wattage math is worth taking seriously, the payback period is short and the number is real. Just do the arithmetic on your own machine\u0026rsquo;s actual behavior, not on the average box in a benchmark.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/gaming-desktop-vs-dedicated-compute-box-idle-power/","section":"Blog Posts","summary":"Idle power draw, not the price tag on a mini PC, is what actually decides whether a dedicated low-power compute box saves you money over running everything on a gaming desktop. A gaming desktop idles somewhere around 80-200W depending on the board, PSU, and how many drives are spinning. A purpose-built low-power box, the N100-class mini PCs and similar, idles at 10-15W. That gap is real and it’s large. Whether it means anything for your electricity bill depends entirely on one question: does buying the mini PC let the desktop actually power off or sleep when you’re not gaming? If the answer is no, the math falls apart, and I found that out the hard way while pricing hardware for my own setup.\n","title":"Gaming Desktop or Dedicated Compute Box: Idle Power Decides, Not Sticker Price","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/git/","section":"Tags","summary":"","title":"Git","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/go/","section":"Tags","summary":"","title":"Go","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/gpu/","section":"Tags","summary":"","title":"GPU","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/grafana/","section":"Tags","summary":"","title":"Grafana","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/hardware/","section":"Categories","summary":"","title":"Hardware","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/hardware/","section":"Tags","summary":"","title":"Hardware","type":"tags"},{"content":"I\u0026rsquo;m a full stack developer based in Atlanta with over 10 years of experience. My specialty is building and optimizing web apps, leading projects that enhance user experience and performance.\nI focus on solving challenges and creating impactful solutions. I am dedicated to sharing my knowledge and helping other developers grow into leaders.\nConnect with me on LinkedIn or check out my projects on Github. Take a look at my blog for articles on technology and development.\n","date":"10 August 2026","externalUrl":null,"permalink":"/","section":"Home","summary":"I’m a full stack developer based in Atlanta with over 10 years of experience. My specialty is building and optimizing web apps, leading projects that enhance user experience and performance.\n","title":"Home","type":"page"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/home-lab/","section":"Categories","summary":"","title":"Home Lab","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/home-lab/","section":"Tags","summary":"","title":"Home Lab","type":"tags"},{"content":"A bulk-reprocess job against one of my LightRAG instances crashed three times in one afternoon, and I shipped eight legitimate fixes before I found the actual cause. That same afternoon I\u0026rsquo;d also fixed a false-positive bug in the GPU broker that arbitrates my home GPU between gaming and local inference. The two bugs had nothing to do with each other. They just happened to land on the same day, which made it tempting at first to blame one on the other. I want to walk through the LightRAG crash specifically, because the honest version of this story is that most of my fixes were correct and still didn\u0026rsquo;t solve it.\nThe crash looked like a concurrency problem, and the first fix was one # LightRAG is a knowledge-graph pipeline I run against a local Ollama embedding backend for a personal research project. I\u0026rsquo;d triggered its reprocess_failed endpoint against an 800-document backlog, and it kept dying with the same signature: an httpx.ReadError, then IndexFlushError, then Pipeline halted, cascading the entire in-flight batch to failed. A stray backup file on disk showed that an earlier session had quietly raised MAX_ASYNC and MAX_PARALLEL_INSERT from 1 to 4, chasing throughput without realizing it would destabilize a local embedding backend. Community guidance backs this up directly: parallel-insert should stay well under async concurrency, not equal to it, and that gap matters more against a local model than a cloud API. I reverted both to 1. It was a real bug that had probably been causing failures for a while. It was not the crash.\nReverting concurrency didn\u0026rsquo;t stop the crash, so I chased connections next # The next run survived sixteen minutes instead of failing instantly, then died with a different-looking error: a stale connection reused after going dead. I added explicit idle timeouts on both sides of my broker\u0026rsquo;s HTTP handling. Along the way I found a second real bug: Ollama\u0026rsquo;s embedding model was cold-starting every seven to twelve minutes because idle gaps between embedding bursts routinely exceeded its five-minute keep-alive default, and every one of those reloads was hitting a missing ROCm library file on my GPU. I set a sixty-minute keep-alive to stop the reload cycling entirely. Both fixes were correct diagnoses of real problems. The crash came back anyway, at almost the same elapsed time, on a different document.\nThree more fixes addressed real mechanisms and still didn\u0026rsquo;t touch the cause # I kept narrowing. A retry layer for connection-level failures on the broker\u0026rsquo;s outbound leg was real hardening, but the retries never fired, meaning the failure wasn\u0026rsquo;t happening on that leg at all. Removing an inbound idle timeout I\u0026rsquo;d added earlier, once I realized it was closing connections during LightRAG\u0026rsquo;s own multi-minute merge phases rather than protecting against staleness, was a legitimate correction that stayed reverted. Disabling connection reuse entirely on the broker\u0026rsquo;s batch server, so every request got a fresh TCP connection, was also real and also didn\u0026rsquo;t change the outcome. By fix eight I\u0026rsquo;d addressed concurrency, idle timeouts, a GPU driver bug, retry logic, and connection reuse, and the job still died in a seventeen-to-thirty-seven-minute window every time. That consistency, regardless of which mechanism I\u0026rsquo;d just changed, was the actual clue. Something systemic was setting the clock, not the code I kept adjusting.\nHere\u0026rsquo;s the shape of the whole afternoon, eight real fixes deep before the actual cause showed up:\nflowchart TD A[Bulk reprocess job crashes] --\u003e B[Fix 1: revert concurrency 4 to 1] B --\u003e C[Crash persists, 16 min instead of instant] C --\u003e D[Fixes 2-3: idle timeouts, 60min keep-alive] D --\u003e E[Crash persists, same 17-37min window] E --\u003e F[\"Fixes 4-8: retry logic, timeout removal,connection-reuse disabled\"] F --\u003e G[Crash STILL persists, same window every time] G --\u003e H[\"Checked the host directly:NAS at \u0026lt;500MB free, 5GB+ in swap\"] H --\u003e I[\"Real cause: host OOM stalling networkunder memory pressure, not the app\"] I --\u003e J[Real fix: moved the workloadto a host with headroom] The real cause was the host running out of memory, not the application # Checking the NAS\u0026rsquo;s own resource state directly settled it. The box had 7.7GB of RAM, roughly 38 Docker containers running on it, and under 500MB genuinely free during a live run, with over 5GB in swap and the kernel\u0026rsquo;s swap-reclaim daemon burning real CPU. LightRAG\u0026rsquo;s own footprint was tiny, under 1.5GB, but it didn\u0026rsquo;t need to be large to get caught in the crossfire. Under that kind of sustained memory pressure, the kernel can stall a process\u0026rsquo;s network handling unpredictably, and from either endpoint\u0026rsquo;s perspective that looks exactly like the other side vanished mid-response. No exception in my code, no crash log on Ollama\u0026rsquo;s side, nothing to grep for. Every timing and connection fix I\u0026rsquo;d shipped was chasing a symptom that could surface anywhere the OS decided to stall.\nThe fix was moving the workload, not patching around the host # I migrated the LightRAG instance off the NAS onto a desktop machine with far more headroom, keeping every earlier hardening change in place since none of them were wrong, just insufficient alone. I hit one more mistake during the move.\nWarning Don\u0026rsquo;t point a migrated container at a loopback address, even when co-locating services on the same host. A container has its own network namespace, so 127.0.0.1 inside it isn\u0026rsquo;t the host\u0026rsquo;s loopback — it won\u0026rsquo;t reach a service the host itself is running. Use the host\u0026rsquo;s real local-network address instead.\nI\u0026rsquo;d reasoned that co-locating services meant loopback would work. It doesn\u0026rsquo;t, for the reason above. Switching to the machine\u0026rsquo;s real local-network address fixed the connection immediately. The reprocess job then ran clean for fifty-two minutes, well past the worst crash point of thirty-seven, with steady progress and zero halts.\nI also owe a correction to my own process here. Partway through this, I declared an earlier fix verified after watching a run for thirty clean minutes, then stopped monitoring it to go write notes. The job crashed seven minutes later. Thirty minutes of no errors isn\u0026rsquo;t proof of anything if you stop watching before the job finishes. I don\u0026rsquo;t think that mistake changes the eventual diagnosis, but it added a full extra round of debugging that a longer, unattended check would have skipped.\nThe GPU broker bug was a genuinely different problem, same day # The other bug that afternoon lived in a completely separate piece of code: the broker that decides when my shared GPU should yield away from local inference toward gaming or Plex. It was yielding every ten to twenty minutes around the clock, including at 1am, because its detector matched on a process name that Plex also runs for background maintenance work like intro-skip detection, not just during actual playback. The fix was to stop pattern-matching on a process name and start asking Plex\u0026rsquo;s own session API whether anything is actually playing. That\u0026rsquo;s a clean, self-contained fix with no connection to memory pressure, embedding batches, or anything else in the LightRAG saga. I mention it here only because \u0026ldquo;one bad day\u0026rdquo; is the accurate frame: two real, unrelated bugs, fixed hours apart, that happened to share an afternoon.\nWhat I\u0026rsquo;m not sure about # I\u0026rsquo;ll admit the two bugs aren\u0026rsquo;t fully unrelated in one respect: both started from trusting a single signal without corroborating it, a log line in one case, a process-name match in the other. That\u0026rsquo;s a real pattern in how I was debugging that day, even though the bugs live in different systems. I\u0026rsquo;m also not confident I\u0026rsquo;ve found the true floor on the embedding-batch size that caused an earlier, secondary instability risk; I tested ten against two and picked the smaller number, without ever bisecting where the actual safe threshold sits. If that pipeline ever needs more throughput, I\u0026rsquo;ll have to test that properly instead of assuming two is magic. What I am confident about is the general lesson: when a fix addresses a real, verified mechanism and the crash still recurs on the same clock, stop tuning that mechanism and check what the host itself is doing.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/nine-fixes-lightrag-embedding-crash-one-afternoon/","section":"Blog Posts","summary":"A bulk-reprocess job against one of my LightRAG instances crashed three times in one afternoon, and I shipped eight legitimate fixes before I found the actual cause. That same afternoon I’d also fixed a false-positive bug in the GPU broker that arbitrates my home GPU between gaming and local inference. The two bugs had nothing to do with each other. They just happened to land on the same day, which made it tempting at first to blame one on the other. I want to walk through the LightRAG crash specifically, because the honest version of this story is that most of my fixes were correct and still didn’t solve it.\n","title":"It Took Nine Fixes to Stop a LightRAG Crash. The First Eight Were All Real Bugs","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/lightrag/","section":"Tags","summary":"","title":"LightRAG","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/linux/","section":"Tags","summary":"","title":"Linux","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/llm-infrastructure/","section":"Tags","summary":"","title":"LLM Infrastructure","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/machine-learning/","section":"Categories","summary":"","title":"Machine Learning","type":"categories"},{"content":"The choice that matters most in this build is form factor, and the trendy answer is wrong for the job. Every \u0026ldquo;quiet home-lab PC\u0026rdquo; guide points at mini-ITX: small, tucked in a corner, low power draw. I already own an RTX 3060 and want a box around it that stays quiet, stays cool, and lets me swap the CPU, RAM, storage, and eventually the GPU without replacing the motherboard underneath them. Mini-ITX fails on all three requirements at once, and it took reading real bench data and practitioner threads, not case marketing copy, to see why.\nMini-ITX trades away the two things this build needs # Mini-ITX cases force two acoustic penalties that stay hidden until you look at the actual hardware inside them. A small case only fits small, high-RPM fans, and small fans have to spin faster than large fans to move the same volume of air. Faster fans are louder fans, full stop. ITX also all but requires an SFX power supply instead of a full ATX unit, and SFX units run louder at idle because their tiny fans work harder inside a smaller housing. Practitioner testing backs this up directly: builders chasing a genuinely silent PC report mATX and ATX cases as consistently quieter than ITX equivalents at equivalent airflow, precisely because of the fan-size and PSU-size penalty ITX imposes.\nMini-ITX also caps upgrade room in ways a spec sheet hides until you\u0026rsquo;re staring at four screw holes. Most ITX boards ship with two RAM slots and one M.2 slot, sometimes two. That\u0026rsquo;s fine on day one and a wall on day four hundred, when I want a second GPU for a small inference cluster, more NVMe for a growing model cache, or just more RAM without pulling both sticks to replace them. A build I\u0026rsquo;m calling upgradable at every part can\u0026rsquo;t start on a board with no more slots to fill.\nmATX gets the noise win ITX promises but can\u0026rsquo;t deliver # mATX solves the acoustic problem ITX claims to own, without the expansion penalty. A mATX case is roomy enough for a full ATX power supply and full-size 120mm or 140mm fans, and larger fans move the same air at lower RPM, which is the actual mechanism behind a quiet PC, not the size badge on the case. mATX boards typically carry four RAM slots and two or three M.2 slots, plus a full-length PCIe slot for the GPU and often room for a second card down the road. I stop fighting the case for room to grow.\nThe tradeoff I\u0026rsquo;m accepting here is real: footprint, not sound. A mATX build sits noticeably larger on a desk or shelf than a genuinely compact ITX box like the Fractal Design Ridge, which measures around 32dB idle by itself — real engineering, in a real quiet ITX case. mATX doesn\u0026rsquo;t beat that on size. It wins on the constraint I actually have, which is upgrade room and noise together, not either one alone. If someone only cares about quiet in the smallest possible box, ITX with a case like the Ridge is still the right call. That isn\u0026rsquo;t my constraint set.\nSocket choice decides how long the board lasts # AM5 is the safer bet for a board I don\u0026rsquo;t want to replace in two years. AMD extended AM5 platform support through 2029, up from an earlier 2027 commitment, with Zen 6 and likely Zen 7 landing on the same socket. Intel\u0026rsquo;s next socket, LGA1954, has only a VP\u0026rsquo;s public statement pointing toward similar multi-generation support, not a locked commitment the way AMD\u0026rsquo;s is. A CPU swap two or three years out should mean unscrewing four cooler mounts, not buying a new motherboard and new RAM and reinstalling the OS.\nChipset tier drives idle power more than the CPU spec sheet # Chipset tier changes idle power draw on AM5 boards more than most builders expect. Measured bench data on a single-chip B650E board showed roughly 71W idle, tying the dual-chip X670E flagship board tested alongside it. The second chip on X670 and X670E boards buys nothing here and just adds another die pulling power around the clock. I\u0026rsquo;m buying a single-chip B650 or B650E board and skipping X670E outright, since this machine runs continuously as an inference host, and idle draw compounds over a year in a way a gaming rig\u0026rsquo;s idle time never does.\nThe CPU\u0026rsquo;s job is sitting at 20W, not winning benchmarks # The GPU carries the AI workload here, so the CPU\u0026rsquo;s real job is staying quiet at idle. A Ryzen 5 7600, non-X and without 3D V-Cache, measured around 20W idle in independent testing, and that figure held across two separate sources. Picking the 3D-cache or X variant would buy gaming frame rates this box has no use for, on a machine whose actual work happens on the GPU sitting next to it.\nSize the power supply to the real load, not to imagined headroom # An oversized power supply runs less efficiently on this build than a right-sized one. The RTX 3060 carries a hard 170W power limit set by Nvidia across every partner card, and a Ryzen 5 7600 idles around 20W and stays well under 100W under load. Measured efficiency curves on 600-650W ATX units peak near 91% at 50% load and dip at both the 10% and 100% ends. A Corsair RM650e held 90.9% efficiency at 50% load with average noise measured at only 12.6 dBA. An 850W-plus unit bought for headroom would run this system under 20% load most of the time, off its efficiency peak, for no real benefit. 550 to 650W, full-size ATX, is the right target.\nThe board and case pick still isn\u0026rsquo;t verified # One piece of this build isn\u0026rsquo;t locked yet. I haven\u0026rsquo;t picked a specific mATX board or case, and I don\u0026rsquo;t want to dress up a guess as a confirmed pick the way the rest of this list is confirmed. Candidates worth pricing out are an ASRock B650M Pro RS or MSI B650M Mortar for the board, and a Fractal Design Pop Air or Meshify 2 Compact for the case, but none of those came from measured bench data the way the CPU, chipset tier, and PSU sizing did. Reddit\u0026rsquo;s homelab and SFF communities would probably settle this faster than another round of vendor listicles, but that search hit a wall this round and I\u0026rsquo;m not filling the gap with a guess dressed as a finding.\nThe build that comes out of all this is AM5, a single-chip B650 or B650E board, mATX case, a non-X Ryzen 5 7600, and the RTX 3060 I already own, on a 550-650W full-size ATX PSU sized to the real load instead of an imagined one. None of the individual parts are exotic or expensive. The only decision that took real digging was form factor, and the small-box answer everyone defaults to turned out to work against what I actually needed: room to add parts later, without losing the quiet.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/mini-itx-is-the-wrong-form-factor-for-a-quiet-ai-homelab-pc/","section":"Blog Posts","summary":"The choice that matters most in this build is form factor, and the trendy answer is wrong for the job. Every “quiet home-lab PC” guide points at mini-ITX: small, tucked in a corner, low power draw. I already own an RTX 3060 and want a box around it that stays quiet, stays cool, and lets me swap the CPU, RAM, storage, and eventually the GPU without replacing the motherboard underneath them. Mini-ITX fails on all three requirements at once, and it took reading real bench data and practitioner threads, not case marketing copy, to see why.\n","title":"Mini-ITX Is the Wrong Form Factor for a Quiet AI Home-Lab PC","type":"blog"},{"content":"A Go service I run at home kept canceling in-flight LLM inference jobs because it thought a game had launched, and most of the time nothing had launched at all. The service is a broker that arbitrates my desktop\u0026rsquo;s single GPU between gaming, Plex transcoding, and local inference through Ollama. When it detects gaming or Plex activity, it force-cancels whatever inference request is running and unloads the model from VRAM, no exceptions, because in my house whoever is playing a game or watching something wins that argument. That priority order is correct. The detector deciding when to enforce it was not.\nI found the bug while chasing a different crash. A bulk ingestion job that leans on the broker for embeddings kept dying partway through with a read error on the Ollama calls, which cascaded into a full pipeline halt. Nothing in the job\u0026rsquo;s own code looked wrong. Checking the broker\u0026rsquo;s logs during the failure windows turned up the real problem: it was flipping into a \u0026ldquo;yielding\u0026rdquo; state roughly every 10 to 20 minutes, around the clock, including the 1am to 6am stretch when nobody in this house is playing anything. ps aux during one of those windows showed only Steam\u0026rsquo;s idle background client. No game process, no active transcode, nothing.\nA single matching process was enough to cancel a running job # The detector scans /proc every three seconds for command-line substrings: Plex Transcoder, Steam\u0026rsquo;s launch marker, Heroic\u0026rsquo;s and Lutris\u0026rsquo;s runner patterns, a bare wine .exe. The moment any one poll matched, the controller flipped to yielding and canceled whatever inference was in flight. There was no debounce, the industry term for waiting out a signal before trusting it, and no second signal to corroborate the first. One sample counted as ground truth. That design wasn\u0026rsquo;t an oversight so much as an unexamined assumption: I\u0026rsquo;d built the hard-cancel policy deliberately, then never asked whether the thing triggering it deserved that much trust.\nPlex\u0026rsquo;s own maintenance jobs look identical to real playback # Plex\u0026rsquo;s own support documentation confirms that Skip Intro and Credits detection, along with chapter-thumbnail generation, run through the same Plex Transcoder binary that handles real playback, on a server-scheduled cadence that has nothing to do with anyone pressing play. My detector grepped for that process name, so a 3am maintenance pass looked exactly like me starting a movie. No amount of debounce timing fixes this, because the false match isn\u0026rsquo;t a brief blip, it can run for several minutes at a stretch. Tautulli, a widely used third-party Plex monitoring tool, sidesteps the whole problem by reading Plex\u0026rsquo;s /status/sessions API instead of the process table, since that endpoint only reports sessions that are actually \u0026ldquo;now playing.\u0026rdquo; That\u0026rsquo;s the real fix for the Plex side: stop grepping for the binary and ask Plex what\u0026rsquo;s actually playing.\nNo game launcher exposes a real \u0026ldquo;foreground game\u0026rdquo; signal # The gaming side is a different problem, and I can\u0026rsquo;t fix it by finding a better API, because none exists. Steam\u0026rsquo;s overlay APIs report whether the overlay is active, not whether a game is running in the foreground. Heroic and Lutris expose no equivalent signal at all. Process-name matching is the only practical option left for gaming detection, and the logs showed those false matches clustering as three-to-six-second blips rather than Plex\u0026rsquo;s multi-minute stretches. Different noise shape, different fix.\nConfirmation gates the cancel, not the recovery # Here\u0026rsquo;s the actual change, before and after:\nflowchart LR subgraph Before[\"Before: single-poll trigger\"] A1[Poll /proc every 3s] --\u003e A2{Any match?} A2 --\u003e|1 match| A3[Cancel inference immediately] end subgraph After[\"After: debounced trigger\"] B1[Poll /proc every 3s] --\u003e B2{Match?} B2 --\u003e|1st match| B3[Wait for confirmation] B3 --\u003e B4{2-3 consecutive matches?} B4 --\u003e|Yes| B5[Cancel inference] B4 --\u003e|No, false blip| B6[Ignore, keep running] endFor the gaming side, the fix is the debounce pattern I should have had from the start: require several consecutive positive polls, not one, before flipping to yielding. I set the default at two or three consecutive matches. Recovery, the transition back out of yielding, stays instant and undebounced, because delaying it only costs a few extra seconds of inference downtime and never risks letting inference run over an actual game. Requiring confirmation before the cancel and skipping it before the recovery isn\u0026rsquo;t symmetric, and it doesn\u0026rsquo;t need to be, the two directions have different failure costs. On a genuine game launch this adds a few seconds of latency before the GPU actually frees up, which is a small price against jobs dying for no reason.\nI\u0026rsquo;ve only shipped half of this. The poll-confirmation gate is small, self-contained, and went in first. The Plex session-API swap hasn\u0026rsquo;t happened yet, because it needs a token Plex issues locally, and I haven\u0026rsquo;t wired that up. Until I do, a multi-minute Plex maintenance run will still trip the broker no matter how high I set the confirm-poll count, since debounce only filters single-sample noise and does nothing against a signal that stays true for five straight minutes. I\u0026rsquo;m also not confident two or three polls is the right number for every workload this machine runs. I picked it from a general flapping-detection convention, not from measurement on my own logs, and I won\u0026rsquo;t know if it\u0026rsquo;s wrong until the false positives either stop or don\u0026rsquo;t.\nHard-canceling instead of throttling is a defensible but costly choice # There\u0026rsquo;s a case against the whole design that the debounce fix doesn\u0026rsquo;t touch. My broker treats every real contention event as a hard stop: cancel the inference request, unload the model, hand the GPU over completely. A tool called Process Lasso does something closer to priority scheduling instead, deprioritizing background GPU compute rather than killing it outright when a game starts. That approach would have made this entire bug far less painful, a false positive would have meant a slower inference request instead of a canceled one. I built it as a hard cutover on purpose, because I wanted a guarantee that the GPU comes back completely clean the moment someone in this house wants to play, and priority-based throttling can\u0026rsquo;t promise that as cleanly. I still think that tradeoff was right for a shared family machine. But it\u0026rsquo;s the reason a detection bug that would have been a minor inconvenience under a softer policy turned into a pipeline outage under mine.\nThe debounce fix is live, the Plex fix isn\u0026rsquo;t, and I\u0026rsquo;ll find out whether either was tuned correctly the next time this job runs unattended overnight and either survives or doesn\u0026rsquo;t.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/debugging-false-positive-gpu-contention-detection/","section":"Blog Posts","summary":"A Go service I run at home kept canceling in-flight LLM inference jobs because it thought a game had launched, and most of the time nothing had launched at all. The service is a broker that arbitrates my desktop’s single GPU between gaming, Plex transcoding, and local inference through Ollama. When it detects gaming or Plex activity, it force-cancels whatever inference request is running and unloads the model from VRAM, no exceptions, because in my house whoever is playing a game or watching something wins that argument. That priority order is correct. The detector deciding when to enforce it was not.\n","title":"My GPU Broker Kept Killing Inference Jobs for Games That Weren't Running","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/networking/","section":"Categories","summary":"","title":"Networking","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/networking/","section":"Tags","summary":"","title":"Networking","type":"tags"},{"content":"Every embedding server I tested handles a vanished GPU the same way: queue requests until a buffer fills, then reject them. Ollama does this. Text Embeddings Inference (TEI) does this. Infinity and llama.cpp do it too, with different buffer sizes and different error codes but the same shape. None of them pause a request and wait out a short GPU outage. They drop it, either immediately or once a queue limit is hit. I run one GPU at home across gaming, media transcoding, and every local model behind my personal tools, and a broker process decides who gets the card and when. That gap between reject-fast and wait-it-out is what forced me to build the missing layer myself.\nThe shared GPU has to change hands, and that\u0026rsquo;s the actual problem # My home GPU serves three tiers of work: interactive chat that needs a response in seconds, batch jobs like embeddings that can tolerate some delay, and long-running jobs that can wait minutes. A broker I run arbitrates access between them. When gaming or a higher-priority job needs the card, the broker yields it away from whatever lower-priority work was using it. That yield might last a few seconds or a couple of minutes. Nothing about the GPU itself failed. It\u0026rsquo;s just occupied elsewhere for a bounded window, and any request caught mid-flight needs to survive that window instead of dying because of it.\nNo shipping server treats a busy GPU as temporary # I went looking for prior art before writing anything, and the pattern held across every tool I checked. Ollama\u0026rsquo;s queue (OLLAMA_MAX_QUEUE, default 512) holds requests FIFO and returns a 503 once the queue is full. TEI\u0026rsquo;s --max-concurrent-requests flag is explicit reject-fast backpressure by design, not an accident of implementation. Infinity and llama.cpp follow the same logic with their own limits. All of them treat a full queue as a hard stop, not something to wait out. That\u0026rsquo;s a reasonable default for a public-facing inference server fielding requests from strangers. It\u0026rsquo;s the wrong default for a private broker that knows exactly why the GPU is unavailable and roughly how long the wait will be.\nLightRAG has no protection of its own, so it has to come from below # I use LightRAG for a knowledge-graph project, and it calls an embedding backend directly with no retry or backpressure logic of its own. Its maintainers\u0026rsquo; fix for slow embed calls is to set TIMEOUT=None and disable the timeout entirely, not to add retries. Three separate open issues track embed failures during batch ingest across different backends, and one of them traces directly to an embed call timing out mid-ingest. None of that gets fixed inside LightRAG. Whatever protection exists has to sit underneath it, in whatever actually talks to the GPU. That\u0026rsquo;s the argument for putting this logic in the broker instead of waiting for any upstream project to add it.\nlitellm gets close but solves a different problem # The nearest thing to a real solution I found was litellm\u0026rsquo;s Router, which supports fallback, cooldown, and timeout configuration for embedding calls. It\u0026rsquo;s a genuinely useful primitive, and I\u0026rsquo;d reach for it if I ever wanted a second embedding backend to fail over to. But its timeout wraps the entire call including retries, not each individual attempt, so it\u0026rsquo;s built for choosing between backends, not for waiting on one backend to come back. I also checked two open-source Ollama proxies, Olla (roughly 260 stars, actively maintained) and ollamaMQ (roughly 114 stars, a fair-share queue proxy written in Rust). Both are solid queueing and failover tools. Neither parks a request through an outage and replays it once the outage ends, which is the specific behavior I needed.\nWhat I built: park the request instead of rejecting it # The fix lives in the fronting proxy inside my broker, one layer above Ollama. When a yield starts, batch-class synchronous requests, which in practice means embeddings, get parked instead of bounced. The hold has a bound: 600 seconds by default, comfortably under LightRAG\u0026rsquo;s own 1200-second embedding timeout, so a parked request never expires on the caller\u0026rsquo;s side while it\u0026rsquo;s still waiting on mine. There\u0026rsquo;s also a hard ceiling on how many requests can be parked at once. Past that ceiling, the broker returns a fast 503, the same reject-fast principle TEI already applies, just moved up a layer instead of invented from scratch. When the yield ends, parked requests replay in FIFO order with a cap on how many go out at once, so the queue doesn\u0026rsquo;t dump a burst back onto Ollama the instant the GPU returns and cause a second failure right after fixing the first one. I also added Prometheus gauges for parked depth, time spent parked, and replay outcomes, plus an alert rule, because TEI already treats queue depth as something worth exposing as a metric and I didn\u0026rsquo;t see a reason to do less.\nHere\u0026rsquo;s the path a request actually takes:\nflowchart LR A[Embedding request arrives] --\u003e B{GPU yielded tohigher-priority work?} B --\u003e|No| C[Serve immediately] B --\u003e|Yes| D{Parked queue below cap?} D --\u003e|No| E[Fast 503, reject] D --\u003e|Yes| F[Park request, up to 600s] F --\u003e G[Yield ends] G --\u003e H[\"Replay parked requests FIFO,capped rate\"]Whether 600 seconds is the right number, I\u0026rsquo;m not fully sure. It\u0026rsquo;s a good margin under LightRAG\u0026rsquo;s timeout today, but it\u0026rsquo;s tuned to my current yield patterns, and if a yield ever runs long for a reason the broker doesn\u0026rsquo;t already know about, that bound will need to move.\nThe CPU fallback I built but haven\u0026rsquo;t turned on # There\u0026rsquo;s an obvious alternative to parking: fall back to a CPU-based embedding model during a yield instead of making anything wait. I have that path built. I\u0026rsquo;m leaving it off by default. I\u0026rsquo;ve seen it misbehave before, unpredictably enough that I don\u0026rsquo;t trust it as a silent fallback, and a LightRAG issue reports CPU-only embedding backends behaving badly specifically inside LightRAG\u0026rsquo;s pipeline, not just running slow. Before I flip that flag on, I want to smoke-test it through LightRAG\u0026rsquo;s actual embedding function, not a standalone request that only proves the model responds to a prompt. A silent, unverified fallback is worse than an honest wait.\nWhat I still haven\u0026rsquo;t proven # The parking logic passes against requests I send it directly, one at a time. What it hasn\u0026rsquo;t seen yet is a forced yield in the middle of a real embed burst, the exact failure mode this whole thing exists to survive. That test is next: trigger a yield artificially while LightRAG is mid-ingest and confirm zero failures on the caller\u0026rsquo;s side, then fold that scenario into the broker\u0026rsquo;s regular test suite so a future change can\u0026rsquo;t quietly break it. Until that runs, this is a design I believe in, not one I\u0026rsquo;ve fully verified under load.\nIf you\u0026rsquo;re running an embedding server behind a shared GPU at home, this gap is worth checking for directly. Query your server\u0026rsquo;s own queue limit, and ask what happens to a request sitting in that queue when the GPU it\u0026rsquo;s waiting on disappears for reasons the server itself doesn\u0026rsquo;t control. In every server I checked, the answer was the same: it dies. Mine doesn\u0026rsquo;t anymore, but only because I stopped assuming someone else had already solved it.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/surviving-a-gpu-yield-window-embedding-servers/","section":"Blog Posts","summary":"Every embedding server I tested handles a vanished GPU the same way: queue requests until a buffer fills, then reject them. Ollama does this. Text Embeddings Inference (TEI) does this. Infinity and llama.cpp do it too, with different buffer sizes and different error codes but the same shape. None of them pause a request and wait out a short GPU outage. They drop it, either immediately or once a queue limit is hit. I run one GPU at home across gaming, media transcoding, and every local model behind my personal tools, and a broker process decides who gets the card and when. That gap between reject-fast and wait-it-out is what forced me to build the missing layer myself.\n","title":"No Embedding Server Survives a GPU Yield Gracefully. I Had to Build That Layer Myself","type":"blog"},{"content":"Family-facing and storage-coupled services stay on the NAS. Compute-heavy personal projects move to a separate host with real memory to spare. That\u0026rsquo;s the whole framework, and I only arrived at it after a Synology DS1522+ with 8GB of RAM spent months running roughly 35 Docker containers and periodically falling over under memory pressure. ContainerManager doesn\u0026rsquo;t fail loudly when it runs out of headroom. It stalls, swaps, and eventually kills something, and figuring out which container mattered enough to protect took longer than it should have.\nStorage coupling decides placement, not how important a service feels # A service that\u0026rsquo;s coupled to storage or answers requests from other people in real time belongs on the NAS regardless of how heavy it is. A photo backup tool needs to sit next to the disks it writes to and needs to respond whenever someone in the house opens the app, so it stays put. A knowledge-graph pipeline or a data-ingestion job runs on my own schedule, tolerates a restart without anyone noticing, and doesn\u0026rsquo;t need to answer anything at 11pm on a Tuesday. That kind of workload moved to my desktop, which has far more RAM than the NAS and isn\u0026rsquo;t a fragile appliance I need to baby. The shift buys headroom on the box that actually has to stay predictable.\nThe placement call itself is a simple branch:\nflowchart TD A[New self-hosted service] --\u003e B{\"Storage-coupled, or answersreal-time requests from people?\"} B --\u003e|Yes| C[Stays on the NAS] B --\u003e|No — tolerates a restart,runs on its own schedule| D[\"Moves to desktop(more RAM headroom)\"] Immich\u0026rsquo;s remote machine-learning support is meant to run alongside the local container, not replace it # Immich, the self-hosted photo app I use for family photo backup, officially supports running its machine-learning container on a separate host from the main server, through the IMMICH_MACHINE_LEARNING_URL setting. That\u0026rsquo;s documented, production-used behavior. The trap is treating it as a full swap: point Immich only at the desktop\u0026rsquo;s ML container, and Smart Search and Face Detection break outright the moment the desktop is off, because my desktop isn\u0026rsquo;t an always-on box the way the NAS is. Immich\u0026rsquo;s own docs are explicit about the right pattern. Keep the local ML container running as a fallback and add the remote URL alongside it, so jobs degrade to local processing instead of failing outright. Facial recognition itself talks to the database directly and doesn\u0026rsquo;t care where the ML container lives, so the underlying Postgres database can stay NAS-side no matter what. One more detail worth flagging: the ML container ships with no authentication at all, so it only ever gets exposed on the local network, never forwarded anywhere.\nSQLite-backed services migrate cheaply; Postgres-backed services need a logical dump # Migrating a stateful service safely comes down to what\u0026rsquo;s storing its state. Anything backed by SQLite in a config directory, which covers most media-automation tools in the *arr family, migrates with a stop-the-container, sync-the-volume, start-on-the-new-host sequence. That\u0026rsquo;s close to zero-risk, because the database is just a file sitting still while you copy it. Anything backed by Postgres is a different problem: copying a live data directory risks corruption, so the safe path is a logical dump while the source stays running, a transfer of that dump, then a restore on the destination with a row-count check before you touch the original. I moved a Postgres-backed data pipeline this way and it went cleanly, though I\u0026rsquo;d braced for it to be worse. I\u0026rsquo;d read enough migration horror stories going in that I probably over-prepared for a problem that never showed up.\nA media library mounted at different paths on two hosts needs a one-time remap # One gotcha cost me more time than the actual migration. Media-automation tools store absolute library paths inside their own database, and if the new host mounts the same share at a different path than the old one did, every stored path is now wrong. Nothing crashes when this happens. Shows just stop being tracked as monitored, and the failure mode looks like a metadata bug instead of a path problem. The fix is a one-time script against the SQLite database that rewrites the stored root-folder paths to match the new mount layout. It\u0026rsquo;s a five-minute job once you know it\u0026rsquo;s coming and an afternoon of confused debugging if you don\u0026rsquo;t.\nMonitoring belongs on the host that isn\u0026rsquo;t under memory pressure # A watchdog that lives on the same box it\u0026rsquo;s protecting adds to the exact pressure it\u0026rsquo;s supposed to catch. I run a lightweight watchdog on the NAS itself, a cron job paired with an ntfy push notification, because that footprint is small enough to not matter. Anything heavier, like Uptime Kuma, I\u0026rsquo;d rather run on the desktop watching the NAS remotely than install directly on the NAS. Putting your monitoring right next to the thing it\u0026rsquo;s watching feels natural. On a RAM-constrained box, it\u0026rsquo;s backwards.\nA RAM upgrade is a hedge, not a proven fix # I haven\u0026rsquo;t upgraded the NAS\u0026rsquo;s memory, and I genuinely don\u0026rsquo;t know if it would solve the problem I moved workloads to avoid. Third-party memory is a real risk on this model specifically. At least one report describes a 16GB module in a DS1522+ registering as only 8GB, so the upgrade can fail silently instead of throwing an obvious error. Even with compatible memory, I couldn\u0026rsquo;t find a solid first-hand account confirming that more RAM actually stops the crash pattern rather than just raising the ceiling before it comes back at a higher container count. So the upgrade sits on my list as a possible complement to the migration, not a substitute for it. If I do it eventually, it\u0026rsquo;s insurance layered on a split that\u0026rsquo;s already working, not a fix I\u0026rsquo;m betting the outcome on.\nThe framework holds up months in, but I don\u0026rsquo;t think the split is finished. Every time a new self-hosted idea shows up, the first question is still which side of this line it belongs on, and I\u0026rsquo;ve gotten that call wrong at least once. A stack I placed on the desktop early has since moved a second time, to a third box entirely, because \u0026ldquo;more RAM than the NAS\u0026rdquo; turned out not to be the same thing as \u0026ldquo;the right home for this workload.\u0026rdquo; The framework tells you which way to lean. It doesn\u0026rsquo;t promise you\u0026rsquo;ll land a given workload in the right spot on the first try.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/not-every-docker-container-belongs-on-the-nas/","section":"Blog Posts","summary":"Family-facing and storage-coupled services stay on the NAS. Compute-heavy personal projects move to a separate host with real memory to spare. That’s the whole framework, and I only arrived at it after a Synology DS1522+ with 8GB of RAM spent months running roughly 35 Docker containers and periodically falling over under memory pressure. ContainerManager doesn’t fail loudly when it runs out of headroom. It stalls, swaps, and eventually kills something, and figuring out which container mattered enough to protect took longer than it should have.\n","title":"Not Every Docker Container Belongs on the NAS","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/observability/","section":"Tags","summary":"","title":"Observability","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/ollama/","section":"Tags","summary":"","title":"Ollama","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/pc-build/","section":"Tags","summary":"","title":"PC Build","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/pi-hole/","section":"Tags","summary":"","title":"Pi-Hole","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/power-efficiency/","section":"Tags","summary":"","title":"Power Efficiency","type":"tags"},{"content":"Preston Bernstein is a versatile full-stack developer with experience in both front-end and back-end technologies. He is based in Atlanta, GA.\n","date":"10 August 2026","externalUrl":null,"permalink":"/authors/preston-bernstein/","section":"Authors","summary":"Preston Bernstein is a versatile full-stack developer with experience in both front-end and back-end technologies. He is based in Atlanta, GA.\n","title":"Preston Bernstein","type":"authors"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/prometheus/","section":"Tags","summary":"","title":"Prometheus","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/proxmox/","section":"Tags","summary":"","title":"Proxmox","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/rag/","section":"Tags","summary":"","title":"RAG","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/rate-limiting/","section":"Tags","summary":"","title":"Rate Limiting","type":"tags"},{"content":"I rebuilt my home network from the ISP modem outward instead of swapping in a new router and hoping the rest sorted itself out. The order was: modem into passthrough (a mode where the ISP box stops doing routing and just hands its public IP straight through), then a UniFi gateway and switch as the core, then Pi-hole DNS filtering running on a Raspberry Pi controller, then every downstream device reconnected one at a time. Bottom-up, slowest layer first, nothing skipped ahead of something it depended on.\nStarting at the modem forces every later phase to be honest # Most rebuild guides start at the router, because the router is the interesting box. I started at the AT\u0026amp;T modem because it was the thing everything else would eventually depend on, and because getting it wrong meant redoing every phase after it. A gateway sitting behind a modem that\u0026rsquo;s still doing its own routing and NAT gets a private IP instead of the real one, and half its features either misbehave or silently don\u0026rsquo;t work. Fixing that after the fact means re-wiring a spine you already built. Fixing it first means every phase after is building on a foundation that\u0026rsquo;s actually solid.\nThe Pi controller has to prove itself before touching hardware # Before I unplugged a single cable, I checked that the Raspberry Pi meant to run both the UniFi controller software and Pi-hole was actually in working order. That\u0026rsquo;s a controller and a DNS filter sharing one small board, so if the board is flaky, both systems inherit the problem. I SSH into the Pi directly, not through any intermediate device, and check that the UniFi controller process is running, that Pi-hole\u0026rsquo;s FTL service is active, and that Pi-hole\u0026rsquo;s local API responds. If any of those fail, I fix them before phase one starts, because a rebuild where the controller itself is unreliable just produces new mystery failures later that look like network problems and aren\u0026rsquo;t.\nPhysical inspection beats trusting old notes # The next step was confirming what the UniFi switch actually was: model, MAC address, firmware version. I had this written down from an earlier setup, but hardware gets swapped and notes go stale, so I checked the label on the unit itself rather than trust a document from months ago. It\u0026rsquo;s a small step and it\u0026rsquo;s easy to skip. Skipping it is also how you end up troubleshooting a switch that isn\u0026rsquo;t the switch you think it is.\nFactory reset comes before adoption, not after # I factory reset the UniFi gateway before letting the controller adopt it, instead of adopting whatever configuration state it happened to be in. Holding the reset button through a full LED flash cycle wipes prior config and puts the device back to a known default, which matters because adopting a gateway with leftover settings from a previous topology is how you get rules that contradict what you\u0026rsquo;re about to build. Once it settles, the gateway is reachable at its default local address over a direct wired connection, and that\u0026rsquo;s the state I want walking into adoption.\nAdoption is where the controller and the gateway agree to work together # Adoption is UniFi\u0026rsquo;s term for a device formally joining a controller: the controller pushes its configuration down, the device reboots into it, and from then on the controller manages it. I connect a laptop directly to the gateway\u0026rsquo;s LAN port, open the controller\u0026rsquo;s web dashboard from the Pi, and adopt the gateway once it shows up as pending. Most of the time this works from the UI in a few minutes. When it doesn\u0026rsquo;t, there\u0026rsquo;s a command-line fallback that points the device at the controller\u0026rsquo;s inform address directly, run over a direct SSH session into the gateway itself, and then the UI adoption is retried. I didn\u0026rsquo;t need the fallback this time, but I wrote it into the plan anyway, because the one time you skip documenting the fallback is the one time you need it at 11pm.\nWiring the spine follows a strict power-on order # Physical wiring came after every device was individually verified, not before. The modem\u0026rsquo;s LAN port feeds the gateway\u0026rsquo;s WAN port. The gateway\u0026rsquo;s LAN port feeds the UniFi switch, which acts as the spine, the central point everything downstream connects through. The switch feeds the Pi controller on one port and the rest of the existing switch gear on another. Power-on order matters here: switch first, then gateway, so it has something to talk to on boot, then the Pi last. Skipping that order doesn\u0026rsquo;t necessarily break anything, but it\u0026rsquo;s one more variable I didn\u0026rsquo;t need while troubleshooting a fresh spine.\nHere\u0026rsquo;s the spine those wiring steps actually build, in the order signal flows through it:\nflowchart LR A[\"ISP modem(passthrough mode)\"] --\u003e B[UniFi gateway] B --\u003e C[UniFi switch — the spine] C --\u003e D[\"Pi controller(UniFi + Pi-hole)\"] C --\u003e E[Rest of existing switch gear]Power-on order runs switch first, then gateway, then the Pi last, so the gateway always has something to talk to the moment it boots.\nPassthrough is a modem setting, not a gateway setting # Passthrough gets configured on the ISP modem, not on the UniFi side, which is a detail that trips people up. It lives in the modem\u0026rsquo;s own admin firewall settings, tied to the gateway\u0026rsquo;s MAC address so the modem knows which downstream device gets the real public IP. After enabling it and letting the modem reboot, I confirm the gateway\u0026rsquo;s WAN interface picked up a public IP instead of a private one handed out by the modem\u0026rsquo;s own NAT, and I check that the controller\u0026rsquo;s dashboard shows the same address. If those two don\u0026rsquo;t match, passthrough isn\u0026rsquo;t actually active yet, no matter what the modem\u0026rsquo;s settings page claims.\nPi-hole runs on the same board as the controller, which is a real tradeoff # Pi-hole filters DNS requests before they leave the network, blocking ads and unwanted domains at the resolver instead of per-device. Running it on the same Raspberry Pi as the UniFi controller keeps the hardware footprint small, and for a home network that\u0026rsquo;s a fine tradeoff. It also means a single board failure takes out both the DNS filter and the controller UI at once, which is the kind of shortcut worth being honest about instead of glossing over.\nWhere downstream devices land was a decision I hadn\u0026rsquo;t made yet # Here\u0026rsquo;s the part of the plan I can\u0026rsquo;t write up as finished, because it wasn\u0026rsquo;t. Before the rebuild, the NAS, desktop, laptop, and a couple of media devices connected straight into modem ports, flat, no managed switch in the path. Once the modem is just a passthrough bridge and the UniFi gateway is the real router, those devices need a new home: stay on the old flat ports and lose DHCP consistency with everything else, or get rewired into the managed spine and gain it. I listed four options in my planning notes and didn\u0026rsquo;t pick one, because it touches a NAS with a bonded network connection I didn\u0026rsquo;t want to reroute on a guess, and a couple of devices whose physical cable runs I hadn\u0026rsquo;t confirmed. That\u0026rsquo;s an honest gap. I\u0026rsquo;d rather admit the plan stalled on a real unknown than pretend I closed it out.\nThe plan mattered more than the finish line # What I actually got out of this wasn\u0026rsquo;t a finished network. It was a sequence I trust: verify the controller, confirm hardware, reset before adopting, wire in a fixed order, flip passthrough, filter DNS, and only then touch the devices that depend on all of it. Each phase has a clear pass or fail condition, which means when something breaks later, I know roughly which layer to check first instead of guessing across the whole stack. The device-landing question is still sitting there unresolved, and I\u0026rsquo;d rather leave it open in writing than pretend the rebuild wrapped up neatly. It didn\u0026rsquo;t, not yet.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/rebuilding-home-network-from-the-modem-up/","section":"Blog Posts","summary":"I rebuilt my home network from the ISP modem outward instead of swapping in a new router and hoping the rest sorted itself out. The order was: modem into passthrough (a mode where the ISP box stops doing routing and just hands its public IP straight through), then a UniFi gateway and switch as the core, then Pi-hole DNS filtering running on a Raspberry Pi controller, then every downstream device reconnected one at a time. Bottom-up, slowest layer first, nothing skipped ahead of something it depended on.\n","title":"Rebuilding a Home Network from the Modem Up, One Phase at a Time","type":"blog"},{"content":"Run one shared Grafana and Prometheus stack for your whole home lab, not one per repo. I have around 30 GitHub repos and 15-20 always-on self-hosted services running mostly on one desktop plus a NAS, and I recently found two separate Grafana containers on that desktop, each spun up by a different project\u0026rsquo;s docker-compose file, each with its own dashboards nobody was cross-referencing. That\u0026rsquo;s the anti-pattern this post argues against, and it happened because \u0026ldquo;just add a Grafana container to the compose file\u0026rdquo; felt like the path of least resistance at the time.\nThe isolation argument doesn\u0026rsquo;t apply to a personal setup # Per-repo or per-tenant observability stacks exist to solve one problem: hard isolation between parties who must never see each other\u0026rsquo;s data. Grafana Labs\u0026rsquo; own guidance treats a single shared stack as the default, and reserves multi-tenant splits for cases like separate customers or separate teams inside a company, where combining dashboards would be a compliance or trust violation. None of that applies when every service on your network is yours. There\u0026rsquo;s no tenant boundary to protect, so there\u0026rsquo;s no isolation benefit to buy with the extra containers.\nThe resource argument doesn\u0026rsquo;t hold either # A full Prometheus, Grafana, and Loki stack runs comfortably in 500MB to 2GB of RAM on a single host, even in a single-binary \u0026ldquo;everything in one process\u0026rdquo; configuration. That number doesn\u0026rsquo;t change much whether it\u0026rsquo;s watching 3 services or 20. Fragmenting into two or three separate stacks doesn\u0026rsquo;t save meaningful memory, because most of that footprint is fixed cost (the databases, the web UI, the query engine) rather than something that scales down with fewer targets. Multiplying that fixed cost across five projects instead of paying it once is pure waste. On my desktop the two duplicate Grafana instances were quietly holding memory that a single shared one wouldn\u0026rsquo;t have needed twice.\nHub-and-spoke is the actual pattern people run at this scale # Homelab operators running desktop-plus-NAS setups converge on the same shape: one central Prometheus/Grafana/Loki stack, and a lightweight collection agent on every monitored host. The current standard agent is Grafana Alloy, an OpenTelemetry-based collector that replaced the older Grafana Agent (which is now deprecated). Alloy ships metrics, logs, and traces from each host back to the one shared backend using a single config file per host. You install one small agent per machine, not one full stack per project. That\u0026rsquo;s the part I got backwards when I let each project\u0026rsquo;s compose file bring its own Grafana along for the ride.\nKeeping the central stack on a separate machine from the workloads it watches also matters. If your monitoring stack lives on the same box as the service it\u0026rsquo;s alerting on, a crash on that box takes out your visibility into the crash at the exact moment you need it. Splitting stack and workload physically, not just logically, is what turns \u0026ldquo;monitoring\u0026rdquo; into something you can actually trust during an incident.\nHere\u0026rsquo;s the shape of the migration, anti-pattern on the left, target on the right:\nflowchart TD subgraph Before[\"Before: one stack per repo\"] A1[Repo A] --\u003e G1[Grafana + Prometheus A] A2[Repo B] --\u003e G2[Grafana + Prometheus B] A3[Repo C] --\u003e G3[Grafana + Prometheus C] end subgraph After[\"After: hub-and-spoke\"] H[One central Prometheus/Grafana/Loki stack] S1[Alloy agent, host 1] --\u003e H S2[Alloy agent, host 2] --\u003e H S3[Alloy agent, host 3] --\u003e H end This is the same shared-infrastructure pattern I already use # I already draw a line between shared infrastructure and project-specific code: things like networking and VPN routing live in one dedicated infra repo, and shared libraries get imported by whichever project needs them instead of being copy-pasted. Observability is the same category of thing as networking or shared libraries. It\u0026rsquo;s plumbing every project needs, not something any one project owns. Treating it as project-specific and letting each repo bootstrap its own copy is the same mistake as vendoring a shared library into five places and letting the copies drift.\nThe real downside: cross-project noise and a bigger blast radius # The honest cost of consolidating is that one shared stack means one shared failure domain and one shared signal-to-noise problem. A misbehaving data-ingestion service can spam the same Grafana instance that\u0026rsquo;s supposed to be giving you a calm read on a media pipeline\u0026rsquo;s health, and if you don\u0026rsquo;t tag and label rigorously, alerts start blurring together across projects that have nothing to do with each other. A stack outage now takes down visibility into everything at once, instead of just one project. And I\u0026rsquo;ll admit dashboard sprawl is a real risk once ten or fifteen projects are all reporting into the same Grafana instance — without folders and consistent naming, the dashboard list turns into its own mess. None of that is imaginary, and the fix is discipline (consistent labels, per-project dashboard folders, and alert routing that filters by service) rather than pretending the problem doesn\u0026rsquo;t exist because you gave up and split the stacks anyway.\nWhat I\u0026rsquo;m actually doing about it # I\u0026rsquo;m standing up a single Prometheus, Grafana, and Loki stack in my shared infrastructure repo, with Alloy as the collector on every host instead of the deprecated Agent. Each service exposes a metrics endpoint where it has one, and node and container-level metrics get scraped centrally rather than per-project. The two duplicate Grafana instances get their dashboards migrated over and then get decommissioned, one at a time and carefully, since one of those projects touches live financial data and I\u0026rsquo;d rather not break its alerting mid-migration. The remaining always-on services that currently have zero monitoring, which is most of them, get wired into the shared stack as I go instead of getting their own bespoke setup.\nNone of this required new hardware or a new product. It required admitting that \u0026ldquo;quick, add Grafana to this compose file\u0026rdquo; was a decision I kept making locally that never added up to a coherent system, and that the fix was to stop treating observability as part of each project and start treating it as part of the network.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/one-observability-stack-not-one-per-repo/","section":"Blog Posts","summary":"Run one shared Grafana and Prometheus stack for your whole home lab, not one per repo. I have around 30 GitHub repos and 15-20 always-on self-hosted services running mostly on one desktop plus a NAS, and I recently found two separate Grafana containers on that desktop, each spun up by a different project’s docker-compose file, each with its own dashboards nobody was cross-referencing. That’s the anti-pattern this post argues against, and it happened because “just add a Grafana container to the compose file” felt like the path of least resistance at the time.\n","title":"Run One Observability Stack, Not One Per Repo","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/runpod/","section":"Tags","summary":"","title":"RunPod","type":"tags"},{"content":"I run vision-model inference for a personal image pipeline on RunPod GPUs instead of calling Google\u0026rsquo;s Gemini API, and the decision has nothing to do with accuracy. Gemini wins on accuracy. RunPod wins on cost, as long as I babysit it correctly, and the babysitting turned out to be the actual engineering problem. This is a companion piece to the estate-sale scanner series on this blog, one narrow decision about where the vision step runs, not a rewrite of that whole pipeline.\nGemini scores higher on accuracy, but my pipeline doesn\u0026rsquo;t need every field to be right # Gemini 2.5 Flash tops a structured-extraction benchmark for vision-language models (VLMs, models that take an image and text prompt together and return structured output) at 0.75 mAP, the highest score of any model tested, self-hosted or managed. A self-hosted model like Qwen2.5-VL trails that number on raw accuracy. What it doesn\u0026rsquo;t have is a marginal cost per call. Every image I send to Gemini costs money regardless of volume; every image I send to a GPU I already control costs whatever fraction of an hour that request occupies the card.\nThat gap only matters if the pipeline can tolerate the accuracy Qwen actually delivers, and mine can. Every field my pipeline extracts carries a confidence tag, and low-confidence lines get flagged for a human glance instead of trusted outright. A task that needs every field right on the first pass shouldn\u0026rsquo;t make this trade. Mine doesn\u0026rsquo;t need that, so the cost side of the ledger got to decide.\nServerless pricing looked like the whole answer until I read the sizing requirements # RunPod\u0026rsquo;s serverless tier scales to zero between requests, so idle time costs nothing, which is the actual reason serverless is attractive for a personal project with bursty traffic. But Qwen2.5-VL isn\u0026rsquo;t a drop-in fit on a serverless worker. Community deployment threads put it on 48GB-class cards, L40, L40S, or RTX 6000 Ada, with GPU memory utilization tuned to 0.90 and prefix caching turned on just to fit the model weights alongside the KV cache the image tokens generate. vLLM\u0026rsquo;s own multimodal serving docs require setting --limit-mm-per-prompt explicitly, for example image=1 for a pipeline that sends one photo per request, because the default silently drops image inputs instead of accepting them.\nNone of that is disqualifying, but it isn\u0026rsquo;t free either, and the same vLLM community thread that gave me the sizing numbers also flags multi-image batching efficiency as an open problem with no confirmed fix. I don\u0026rsquo;t send multiple images per request today, so that gap doesn\u0026rsquo;t block me, but it\u0026rsquo;s a sign the serverless-vision path is younger than the serverless-text path I\u0026rsquo;ve used elsewhere. I\u0026rsquo;m not treating serverless as a settled choice for this workload yet.\nDedicated pods are cheaper per hour, and that\u0026rsquo;s exactly what makes them dangerous # A dedicated RunPod GPU, an A40 with 48GB running a vLLM template, prices out around $0.44 an hour, a small fraction of what a larger card costs me for other GPU work I run at home. At that rate, a dedicated pod running vision inference all day still costs less than a handful of Gemini calls at any real volume. The catch is that a dedicated pod bills for every minute it\u0026rsquo;s running, whether or not anything is calling it.\nServerless pods scale to zero automatically. Dedicated pods don\u0026rsquo;t, and I went looking in RunPod\u0026rsquo;s own docs assuming I\u0026rsquo;d just missed a toggle. There isn\u0026rsquo;t one. RunPod\u0026rsquo;s GraphQL API documents a podStop mutation, podStop(input: {podId: \u0026quot;ID\u0026quot;}) { id desiredStatus }, which stops a pod and preserves its volume data, but there\u0026rsquo;s no podTerminate mutation and no built-in idle timeout anywhere in the dedicated-pod management docs. Idle-auto-stop is a serverless feature. A dedicated pod left running after the last request just keeps billing by the minute until something outside RunPod tells it to stop.\nI built a watchdog because nothing else was going to call podStop for me # Once I confirmed the gap was real and not a documentation oversight, the fix was straightforward: an external watchdog that checks how long the pod has been idle and calls podStop once that idle window passes a threshold I set. This wasn\u0026rsquo;t a workaround I invented out of necessity. RunPod\u0026rsquo;s own cost-control guidance recommends exactly this shape: treat the GPU as fully ephemeral, let an external scheduler launch the pod, and have either the job itself or the scheduler call stop once the work is done. Pods bill minute by minute while running, so the whole cost argument for choosing a dedicated pod over Gemini falls apart if nothing is watching the clock. I\u0026rsquo;d already written a version of this watchdog for a different self-hosted GPU job, so this was mostly reusing a pattern rather than inventing one from scratch.\nHere\u0026rsquo;s what the watchdog actually does, on a loop:\nflowchart LR A[Watchdog checks pod idle time] --\u003e B{Idle threshold exceeded?} B --\u003e|No| A B --\u003e|Yes| C[Call podStop via RunPod GraphQL API] C --\u003e D[Pod stopped, billing stops,volume data preserved] What I still haven\u0026rsquo;t proven # I\u0026rsquo;ve committed to dedicated-pod-plus-watchdog for now, but I haven\u0026rsquo;t run a real head-to-head between serverless and dedicated at my actual production volume yet. The sizing and batching caveats from the vLLM community are enough to make me wary of trusting serverless vision inference on faith, so a dedicated pod with a watchdog is the safer default while that\u0026rsquo;s unverified. I could end up moving to serverless once I actually benchmark cold-start latency and per-image cost against what the watchdog setup gives me today. For now, the dedicated pod is cheaper, the watchdog keeps it honest, and I\u0026rsquo;d rather admit that\u0026rsquo;s a decision I haven\u0026rsquo;t fully stress-tested than pretend the comparison is closed.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/runpod-vs-gemini-vlm-inference-idle-auto-stop-gap/","section":"Blog Posts","summary":"I run vision-model inference for a personal image pipeline on RunPod GPUs instead of calling Google’s Gemini API, and the decision has nothing to do with accuracy. Gemini wins on accuracy. RunPod wins on cost, as long as I babysit it correctly, and the babysitting turned out to be the actual engineering problem. This is a companion piece to the estate-sale scanner series on this blog, one narrow decision about where the vision step runs, not a rewrite of that whole pipeline.\n","title":"RunPod Beats Gemini on Cost for My Vision Pipeline, and the Idle-Stop Feature It's Missing","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/security/","section":"Categories","summary":"","title":"Security","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/security/","section":"Tags","summary":"","title":"Security","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/self-hosting/","section":"Tags","summary":"","title":"Self-Hosting","type":"tags"},{"content":"Shipping fast is not the same as being done, and I had to learn that the expensive way with a CLI tool an agent pipeline built for me in one afternoon. The pipeline is one I built myself: I give it a one-line description of what I want, it writes a spec, runs that spec through seven parallel agents whose only job is to attack it from different angles, spins up parallel build agents against the hardened spec, runs a full code review pass, then smoke-tests the real thing before calling it done. For a small outreach-automation CLI (local SQLite state, a human approval gate before anything goes out, a GitHub-facing sourcing loop), that pipeline produced working software in an afternoon. It ran. It did the job I asked for. It was also not something I trusted enough to extend to a second platform without checking it first.\nThe pipeline optimizes for the spec, not for what the spec left out # Every phase in that pipeline checks the code against what I asked for. The adversarial challenge attacks the spec itself, the build agents implement against the hardened version, the review pass checks the diff for bugs and style, and the verify step proves the CLI actually runs end to end. None of that touches questions I never thought to ask in the spec. I hadn\u0026rsquo;t written \u0026ldquo;honor GitHub\u0026rsquo;s rate-limit contract\u0026rdquo; or \u0026ldquo;make sure the SQLite backup survives a write in progress\u0026rdquo; anywhere, so nothing in the pipeline went looking for those gaps. A spec-driven pipeline is only as complete as the spec, and mine had holes I couldn\u0026rsquo;t see until something outside the pipeline pointed a light at them.\nHere\u0026rsquo;s the shape of both passes side by side: the build pipeline that shipped the CLI, and the separate audit pass that checked its work.\nflowchart LR A[One-line description] --\u003e B[Spec written] B --\u003e C[7 parallel adversarial challenge agents] C --\u003e D[Hardened spec] D --\u003e E[Parallel build agents] E --\u003e F[Code review pass] F --\u003e G[Smoke test / verify] G --\u003e H[Working CLI, shipped in an afternoon] H -.-\u003e|separate pass, run on purpose| I[External research audit] I --\u003e J[\"4 concrete bugs found:rate limits, WAL backup,approval-log gap, thin lead signal\"] I --\u003e K[\"1 strategic decision:second platform goes draft-only,no automation\"]That something was a separate research pass I ran on purpose, specifically to find holes before adding a second platform. It pulled in GitHub\u0026rsquo;s own API documentation, SQLite backup literature, comparable open-source tools, and, because the second platform I wanted to add was one with a strict terms-of-service posture around automation, that platform\u0026rsquo;s actual user agreement. Four concrete problems came out of it, plus one decision that changed my plan for the second platform entirely.\nThe GitHub loops never met GitHub\u0026rsquo;s own rate-limit contract # My tool has three loops that poll and post against GitHub (sourcing, checking, and queue-draining), and none of them honored the limits GitHub documents for its own API. GitHub publishes real numbers: a cap on concurrent requests, a points-per-minute budget on REST calls, a separate and much stricter cap on content-creating requests per minute and per hour. GitHub\u0026rsquo;s docs are also explicit that repeatedly ignoring rate-limit errors can get an integration banned outright, not just throttled. My loops were calling the API and hoping, with no code anywhere that read a Retry-After header or backed off on a 403. The fix was mechanical once I knew what to build: honor Retry-After and the rate-limit-reset header first, switch polling loops to conditional requests so unchanged data comes back as a cheap 304 instead of spending budget, and space out anything that creates content by at least a second. None of that is clever. All of it was missing.\nA raw file copy could have quietly corrupted the backup # The tool\u0026rsquo;s entire state (accounts, drafts, leads) lives in one SQLite file, and the backup routine copied that file directly on a schedule. SQLite in its default mode buffers recent writes in a separate write-ahead log file, and a plain file copy of the main database while that log holds unflushed writes can capture a database that looks intact and isn\u0026rsquo;t. This is the kind of bug that never shows up in testing, because testing doesn\u0026rsquo;t usually catch a backup mid-write, and it only bites the one time you actually need the backup to be good. The fix is a single command swap, from a raw copy to a WAL-safe backup call that captures a consistent snapshot regardless of what\u0026rsquo;s mid-flight. Small fix, but it was sitting on exactly the failure mode I\u0026rsquo;d never notice until it was too late to matter.\nThe approval gate had no memory of its own decisions # Nothing goes out of this tool without a human approving it first, and that gate is tied to a hash of the exact content being approved, so any edit after approval voids it automatically. That part of the design held up fine under review. What was missing was history: no log of who approved what, when, or what got rejected and why. If I wanted to know later why a specific piece of content went out, or audit a month of decisions, there was nothing to check against but my own memory of pressing a key. Commercial approval-workflow tools keep exactly this kind of log by default. Mine didn\u0026rsquo;t, and it\u0026rsquo;s the kind of gap that\u0026rsquo;s invisible right up until you need it for a reason you didn\u0026rsquo;t plan for.\nLead-sourcing ran on one thin signal # The tool finds candidates to reach out to using keyword matching against a configured niche list, and that\u0026rsquo;s the whole signal. Comparable tools in this space enrich candidates with graph signals (repository stars, forks, contributor overlap) that catch relevance keyword matching alone misses. I haven\u0026rsquo;t fixed this one yet. It\u0026rsquo;s queued, not resolved, and I\u0026rsquo;m listing it here instead of pretending it\u0026rsquo;s closed because the rest of this post is about being honest about what \u0026ldquo;done\u0026rdquo; actually took.\nExtending to a second platform meant deciding not to automate it # The most consequential finding wasn\u0026rsquo;t a bug at all. Before writing a single line for the second platform, I checked its user agreement, and it explicitly bans the exact category of automation my GitHub loops already do: auto-connecting, auto-posting, auto-commenting, and scraping via any bot or script. Real ban-rate data on comparable automation tools for that platform backs the terms up. Even the more cautious, cloud-hosted versions of that kind of automation carry meaningful suspension risk. And I found at least one legitimate, adopted product in that space that does exactly what I was already leaning toward: format drafts for a human to review and post manually, no session automation at all. That\u0026rsquo;s proof draft-only is a real category, not a compromise I was talking myself into. So the second-platform build changed shape entirely: instead of extending the same auto-post pattern, I\u0026rsquo;m formatting approved drafts with suggested timing for that platform\u0026rsquo;s own native scheduler and stopping there.\nWhat I\u0026rsquo;m still not sure about # I don\u0026rsquo;t know yet whether a dedicated audit pass like this needs to happen after every run of my build pipeline, or whether this project just happened to be unusual enough, real external APIs, real state that has to survive a backup, a second platform with real legal terms, to need one. Running an audit like this on every small tool I build would be pure overhead for most of them. I lean toward doing it whenever a tool talks to another service\u0026rsquo;s API or holds state I\u0026rsquo;d actually miss if it corrupted, and skipping it otherwise, but I\u0026rsquo;ve only tested that rule on one project so far. The build pipeline did exactly what I asked it to do, fast and correctly. It just turned out that \u0026ldquo;what I asked for\u0026rdquo; and \u0026ldquo;what I actually needed before trusting this thing\u0026rdquo; were two different lists, and finding the second list took a separate pass I almost skipped.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/auditing-what-an-agent-pipeline-shipped-in-an-afternoon/","section":"Blog Posts","summary":"Shipping fast is not the same as being done, and I had to learn that the expensive way with a CLI tool an agent pipeline built for me in one afternoon. The pipeline is one I built myself: I give it a one-line description of what I want, it writes a spec, runs that spec through seven parallel agents whose only job is to attack it from different angles, spins up parallel build agents against the hardened spec, runs a full code review pass, then smoke-tests the real thing before calling it done. For a small outreach-automation CLI (local SQLite state, a human approval gate before anything goes out, a GitHub-facing sourcing loop), that pipeline produced working software in an afternoon. It ran. It did the job I asked for. It was also not something I trusted enough to extend to a second platform without checking it first.\n","title":"Shipping Fast Isn't the Same as Being Done: Auditing a CLI My Agent Pipeline Built in an Afternoon","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/software-architecture/","section":"Categories","summary":"","title":"Software Architecture","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/synology-nas/","section":"Tags","summary":"","title":"Synology NAS","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"Running four or five Claude Code agents at once across separate repos felt like one problem: things drifting out of state while I wasn\u0026rsquo;t watching. It turned out to be three separate failure modes wearing one name, and each one needed a different fix. Worktree sprawl (leftover git checkouts an agent session opened and nobody closed) was mostly a feature I hadn\u0026rsquo;t turned on, not a missing tool. Deploy drift (a running service that no longer matches what an agent thought it built) was a problem I\u0026rsquo;d already solved once, for one project, and just needed to generalize. Wasted compute across my machines is the one piece nothing solved, and it\u0026rsquo;s still open. Treating all three as one complaint is exactly what kept me from seeing that only one of them was actually about drift.\nHere\u0026rsquo;s where each of the three actually landed:\nflowchart TD A[\"One complaint: 'state drift'\"] --\u003e B[Worktree sprawl] A --\u003e C[Deploy drift] A --\u003e D[Wasted compute] B --\u003e B1[\"RESOLVED — a feature alreadyshipped, just needed reading the docs\"] C --\u003e C1[\"RESOLVED — pattern I'd alreadybuilt once, generalized to every repo\"] D --\u003e D1[\"OPEN — no real fix found,nothing scheduled to build it yet\"] Worktree sprawl turned out to be a feature nobody had switched on # I run Claude Code as several parallel agents, each working a different repo or branch, and each one needs its own working directory so two agents don\u0026rsquo;t stomp on the same uncommitted edits. Git\u0026rsquo;s answer to that is a worktree: a second working directory attached to the same repository, checked out on its own branch, addable and removable independently of the main clone. The complaint that started this whole investigation was plain. I kept finding worktrees on disk that some agent session had opened and nobody, including me, had closed.\nThe research sweep turned up something I hadn\u0026rsquo;t clocked: Claude Code already ships a lifecycle for this, and most of it just needs turning on rather than replacing. It auto-sweeps worktrees it created for subagents and background sessions once they clear a configurable age, but only if they\u0026rsquo;re clean, with no uncommitted changes and no unpushed commits. Anything opened with an explicit --worktree flag, or created mid-session with the EnterWorktree tool, is permanently exempt from that sweep. The documentation says so directly: it never removes a worktree you create that way. That distinction explains most of what I\u0026rsquo;d been seeing. My deliberate multi-agent sessions, the ones I open on purpose rather than the throwaway subagent kind, were never going to get swept, because the sweep was never built to touch them.\nDeploy drift is a different bug wearing the same complaint # Deploy drift means a running service no longer matches what the agent that built it believes is deployed: config edited by hand after the fact, a container that never picked up the latest image, a service pointed at a stale checkout. This isn\u0026rsquo;t a worktree problem at all. It\u0026rsquo;s a gap between git state and live state, and no worktree cleanup script reaches it. I\u0026rsquo;d already closed that gap once, for one home-lab service, with a script that checks the deploy target after every push and diffs what\u0026rsquo;s actually running against what git says should be running, backed by a written rule that every service needs the same coverage. The pattern the wider search turned up, a scheduled check that shells out over SSH to compare live state against the repo, was structurally the same thing I\u0026rsquo;d already built. The gap wasn\u0026rsquo;t a missing tool. It was that the pattern only ran against one project instead of every project with something deployed.\nHeavier options exist, like a continuous-reconciliation controller that diffs live cluster state against a git manifest on every change. That shape is built for orchestrating containers across a cluster, and my footprint is a handful of systemd services and Docker Compose stacks on two machines. Adopting that would mean running infrastructure to manage infrastructure I don\u0026rsquo;t have. The actual fix is unglamorous: copy the pattern I already trust to the rest of the repos that deploy something.\nWasted compute is the one nobody has actually solved # Compute utilization across my machines is where the search came back empty-handed. Agents sit idle on one box while the other has spare capacity, and nothing I found actually schedules work across that gap the way a real fleet scheduler would. The closest candidate was a small, early open-source CLI aimed at exactly this, routing work and judging reliability across agent runtimes, but it\u0026rsquo;s unverified: too new and too thin on real adoption to trust with anything that matters. I\u0026rsquo;m calling that an honest gap, not a wait-and-see item with nothing to do. If I want it solved, I have to build a thin version myself, and I haven\u0026rsquo;t started.\nA follow-up audit checked whether the fix actually held # Two weeks after landing on that plan, I went back and audited every repo on both machines against the documented worktree lifecycle instead of taking the research sweep\u0026rsquo;s conclusion on faith. The native EnterWorktree/ExitWorktree lifecycle, Claude Code\u0026rsquo;s own tools for opening and closing a worktree mid-session, works correctly in exactly the one workflow I built for it, and doesn\u0026rsquo;t exist anywhere else yet. That workflow opens a worktree at the start of a run and closes it right after a successful merge. Every other repo on the Mac (roughly two dozen of them) and every repo on the desktop, which is a deploy target rather than somewhere agents run, has never had a worktree at all. There\u0026rsquo;s no adoption gap to close in those repos, because there\u0026rsquo;s no worktree activity in them to sweep in the first place.\nTotal inventory across both machines came to four worktrees. One was a live session, locked and actively in use, correctly left alone. Two belonged to a separate build-cache tool used by another skill in my pipeline, not Claude Code\u0026rsquo;s own lifecycle, which is a different kind of accumulation than agent sprawl. One was a genuine dead worktree: a merged, clean, three-day-old checkout that should have been removed and wasn\u0026rsquo;t.\nThat fourth one is the interesting case, because it wasn\u0026rsquo;t a bug. My own workflow documents a fallback rule for exactly this situation: if a run fails partway through after the merge already succeeded, leave the worktree on disk and say so in the final report instead of silently deleting work mid-failure. The dead worktree on the Mac is that rule firing exactly as designed, on a run where some later phase (hardening, review, deploy, or verification) stopped short after the merge had already landed. The lifecycle didn\u0026rsquo;t fail. It did exactly what I told it to do when something breaks downstream, which is preserve state over convenience. I understand that failure now; I haven\u0026rsquo;t fixed the part where nothing reminds me to go check for it after a run stops early. That\u0026rsquo;s still a manual habit, not automation.\nOf the three failure modes, one turned out to already be handled by a feature I hadn\u0026rsquo;t read the docs on. One is a pattern I\u0026rsquo;d already proved, just not everywhere it needs to run. The third has no real answer yet, and I\u0026rsquo;m not going to dress up a small unverified GitHub repo as a fix. Three problems, three different states of done. Calling all of it \u0026ldquo;state drift\u0026rdquo; on day one is exactly what stopped me from seeing that only one of the three was actually a drift problem at all.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/three-failure-modes-one-name-concurrent-claude-code-agents/","section":"Blog Posts","summary":"Running four or five Claude Code agents at once across separate repos felt like one problem: things drifting out of state while I wasn’t watching. It turned out to be three separate failure modes wearing one name, and each one needed a different fix. Worktree sprawl (leftover git checkouts an agent session opened and nobody closed) was mostly a feature I hadn’t turned on, not a missing tool. Deploy drift (a running service that no longer matches what an agent thought it built) was a problem I’d already solved once, for one project, and just needed to generalize. Wasted compute across my machines is the one piece nothing solved, and it’s still open. Treating all three as one complaint is exactly what kept me from seeing that only one of them was actually about drift.\n","title":"Three Failure Modes Wearing One Name: Running Concurrent Claude Code Agents Without State Drift","type":"blog"},{"content":"Feeding a few hundred books into LightRAG through Gemini taught me that concurrency tuning is the wrong first lever, and the rate-limit table you\u0026rsquo;d normally tune it against doesn\u0026rsquo;t exist anymore anyway. I run a personal knowledge-graph project that ingests close to a thousand book-length documents through LightRAG (HKUDS), using Gemini for entity extraction and embeddings behind a LiteLLM proxy. The corpus is entity-dense enough that the LLM merge phase dominates ingestion time, and early runs kept marking documents FAILED with no obvious cause in the LightRAG logs. This post covers what I found chasing that down: the actual concurrency knobs, why Gemini\u0026rsquo;s rate limits are a moving target now, and the one setting that mattered more than any of it.\nA 429 in LightRAG doesn\u0026rsquo;t retry, it fails the document # The failure mode is quiet and that\u0026rsquo;s what makes it dangerous. When a Gemini call returns HTTP 429, LightRAG doesn\u0026rsquo;t queue the document and try again later. It marks the document FAILED and moves on. Nothing crashes, nothing pages you, and unless you\u0026rsquo;re watching the per-document status table you won\u0026rsquo;t notice until the corpus finishes and a chunk of it is just missing from the graph. On my first real run against this corpus, that\u0026rsquo;s exactly what happened: documents disappeared from the pipeline in a way that looked like success from a distance.\nLightRAG\u0026rsquo;s own tuning knobs assume a ratio, not a rate # LightRAG exposes four environment variables that govern ingestion concurrency, and reading the source is more reliable than trusting the docs prose. MAX_ASYNC_LLM sets the number of concurrent LLM calls (extraction, merge, keyword generation, answer synthesis) and defaults to 4. MAX_PARALLEL_INSERT sets how many documents get processed in parallel, defaults to 3, and LightRAG\u0026rsquo;s own env.example recommends keeping it near MAX_ASYNC_LLM / 3. EMBEDDING_FUNC_MAX_ASYNC governs concurrent embedding calls on a separate pool from the LLM pool, default 8. EMBEDDING_BATCH_NUM sets how many chunks get bundled into one embedding request, default 10.\nThe project\u0026rsquo;s documented high-throughput profile is MAX_ASYNC_LLM=8, MAX_PARALLEL_INSERT=3, EMBEDDING_FUNC_MAX_ASYNC=16, EMBEDDING_BATCH_NUM=32, and a real-world test on GitHub (issue #2264) using a similar profile took ingestion from 7 hours 8 minutes down to 1 hour 45 minutes on the same corpus. That\u0026rsquo;s a legitimate 4x. But that ratio describes how LightRAG should divide work internally. It says nothing about how much total work your Gemini project is allowed to accept per minute, and that ceiling is the one that actually throws the 429s.\nGemini\u0026rsquo;s rate limit isn\u0026rsquo;t a table you can hardcode anymore # Google stopped publishing a static per-model rate-limit table as of July 2026. The docs now say limits depend on your project\u0026rsquo;s usage tier and are \u0026ldquo;not guaranteed,\u0026rdquo; which in practice means you read the live number out of AI Studio for your specific project before you tune anything. That was a real adjustment for me: I\u0026rsquo;d been treating rate limits like a spec you design against once, and they\u0026rsquo;re now closer to a runtime condition you have to check. Free and early-tier flash access is often in the 10-15 RPM range, which makes MAX_ASYNC_LLM=8 from the \u0026ldquo;official\u0026rdquo; profile actively dangerous rather than aspirational. There\u0026rsquo;s also a second, independent limiter on paid tiers: a spend-based burst cap over a rolling 10-minute window, separate from the RPM/TPM ceiling. You can be well under your requests-per-minute limit and still get 429\u0026rsquo;d by the burst cap.\nThe derivation that actually holds up: set MAX_ASYNC_LLM to roughly your live RPM times average call latency in seconds, divided by 60. Flash\u0026rsquo;s latency runs 1-3 seconds per call, so a 10 RPM tier caps you at 2-4 concurrent calls, while a paid tier with thousands of RPM lets you approach the documented profile. Everything else (insert parallelism, embedding pool size) derives from that number, not the other way around. Tune to the ratio first and you\u0026rsquo;re tuning against a number that doesn\u0026rsquo;t reflect your actual ceiling.\nHere\u0026rsquo;s the derivation chain end to end, tuning knobs plus the absorb layer:\nflowchart TD A[\"Check live RPM from AI Studio,not a hardcoded table\"] --\u003e B[\"MAX_ASYNC_LLM = live RPM x latency(s) / 60\"] B --\u003e C[MAX_PARALLEL_INSERT derives from ratio] B --\u003e D[EMBEDDING_FUNC_MAX_ASYNC derives from ratio] E[EMBEDDING_BATCH_NUM: fix leftover local-GPU value] --\u003e D B --\u003e F[\"LiteLLM router: rpm/tpm caps + RateLimitErrorRetries\"] F --\u003e G[\"429 becomes a delayed retry,not a FAILED document\"] The single highest-leverage fix wasn\u0026rsquo;t concurrency at all # My container had EMBEDDING_BATCH_NUM set to 2, a leftover from an earlier era when embeddings ran on a local GPU model instead of Gemini\u0026rsquo;s hosted embedding API. Against a local model, batch size barely matters. You\u0026rsquo;re not paying per request. Against a rate-limited cloud API, a batch size of 2 versus the recommended 32 means sixteen times more embedding requests for the exact same corpus, which is sixteen times more pressure on the embedding RPM ceiling for zero benefit. Fixing that one line did more for my 429 rate than any concurrency change did, and it carried no downside: same total work, dramatically fewer requests. If you\u0026rsquo;re moving a LightRAG setup from a local embedder to a cloud one, check this value before touching anything else.\nGitHub issue #1648 is a useful reality check here too: someone running a 50,000-document ingest with a conservative embedding concurrency of 5 still hit 429s on the embedding service. Low concurrency reduces the odds of hitting a ceiling, but it doesn\u0026rsquo;t eliminate them, because a single misconfigured batch size can undo the benefit of a conservative concurrency setting entirely.\nThe proxy layer should absorb overshoot, not let it fail documents # Concurrency limits are a best-effort guess at the ceiling, and best-effort guesses are sometimes wrong. The fix isn\u0026rsquo;t guessing more precisely, it\u0026rsquo;s making the failure mode survivable when you guess wrong. LiteLLM\u0026rsquo;s router supports rpm and tpm caps per model in its model_list, and if you don\u0026rsquo;t set max_parallel_requests explicitly it derives concurrency from those numbers automatically. It also supports a retry_policy with a dedicated RateLimitErrorRetries count, separate from timeout or server-error retries, which is the setting that actually matters here: a 429 that hits LiteLLM with that policy configured gets retried with backoff instead of surfacing as an error LightRAG has to interpret. Set those caps to your project\u0026rsquo;s real live limits, add the retry policy, and a burst that exceeds your ceiling turns into a delayed request instead of a failed document. Without that layer, every concurrency tweak is a bet that you never overshoot, and eventually you will.\nOne caveat if you run LiteLLM with multiple worker processes: rpm/tpm counters need to be backed by Redis to be shared across workers, or each worker enforces the cap independently and your real aggregate concurrency against Gemini is a multiple of what you configured. I haven\u0026rsquo;t needed multi-worker LiteLLM for this corpus size, so I can\u0026rsquo;t speak to how much that matters in practice, but it\u0026rsquo;s a documented gap worth knowing about before you scale up.\nWhat I\u0026rsquo;d push back on in my own conclusion # The uncomfortable part of this whole exercise is that the \u0026ldquo;optimal ratio\u0026rdquo; LightRAG documents is close to useless without knowing your live rate limit first, which makes it feel like the wrong thing to have started with. I could argue I wasted time reading env.example in detail when the real fix was one line in a docker-compose file. I don\u0026rsquo;t think that\u0026rsquo;s quite right, though. The ratio still matters once you know your ceiling, because it tells you how to divide a fixed budget of concurrent calls between insertion and embedding rather than just picking a number. What I\u0026rsquo;m genuinely unsure about is whether the entity-merge phase\u0026rsquo;s partial serialization (the same GitHub issue that got the 4x speedup also noted the GPU sat underutilized because merge logic doesn\u0026rsquo;t fully parallelize) is a bigger long-term bottleneck than rate limits for a corpus this size. I haven\u0026rsquo;t run the numbers on a from-scratch full reingest with the fixed batch size and proxy guardrails in place, so that\u0026rsquo;s a real open question, not a settled one.\nIf you\u0026rsquo;re running LightRAG against any rate-limited cloud LLM, check three things before you touch a single concurrency variable: your embedding batch size, your provider\u0026rsquo;s live rate limit for your actual tier, and whether your proxy retries 429s or just lets them through. Concurrency tuning is the part that feels like engineering. Getting those three right is the part that actually stops documents from silently failing.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/tuning-lightrag-ingestion-concurrency-against-gemini-rate-limits/","section":"Blog Posts","summary":"Feeding a few hundred books into LightRAG through Gemini taught me that concurrency tuning is the wrong first lever, and the rate-limit table you’d normally tune it against doesn’t exist anymore anyway. I run a personal knowledge-graph project that ingests close to a thousand book-length documents through LightRAG (HKUDS), using Gemini for entity extraction and embeddings behind a LiteLLM proxy. The corpus is entity-dense enough that the LLM merge phase dominates ingestion time, and early runs kept marking documents FAILED with no obvious cause in the LightRAG logs. This post covers what I found chasing that down: the actual concurrency knobs, why Gemini’s rate limits are a moving target now, and the one setting that mattered more than any of it.\n","title":"Tuning LightRAG Ingestion Concurrency Against a Rate-Limited Gemini API","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/unifi/","section":"Tags","summary":"","title":"UniFi","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/vision-models/","section":"Tags","summary":"","title":"Vision Models","type":"tags"},{"content":"A single Claude Code session in my home lab cost $364. I found it on my own /usage report, not from a billing alert, and it was enough to make me stop and read the full week behind it instead of writing it off as one bad run.\nThe breakdown behind that number told a clean story. Every dollar of that week\u0026rsquo;s spend, 100 percent of it, came from sessions that had spawned subagents, meaning the main session had delegated work to separate Claude instances running in parallel rather than doing everything itself. Ninety-nine percent came from sessions that ran longer than eight hours straight. Ninety percent of spend happened while the session\u0026rsquo;s context window, the running token budget that holds the full conversation history, sat above 150,000 tokens. And by the middle of that week, continuous claude -p jobs, Claude Code\u0026rsquo;s non-interactive mode for scripted and scheduled work, firing unattended on my desktop had already burned 62 percent of my weekly usage cap. Four numbers, one shape: long, subagent-heavy, unattended sessions running with no lifecycle boundary at all.\nHere\u0026rsquo;s how the four numbers converged into one diagnosis, and the fixes that came out of it:\nflowchart TD A[\"$364 session\"] --\u003e B[\"100% of spend: sessions with subagent fan-out\"] A --\u003e C[\"99% of spend: sessions open 8+ hours\"] A --\u003e D[\"90% of spend: context above 150k tokens\"] A --\u003e E[\"62% of weekly cap: unattended claude -p jobs\"] B \u0026 C \u0026 D \u0026 E --\u003e F[\"One shape: long, subagent-heavy,unattended sessions, no lifecycle boundary\"] F --\u003e G[Fix: CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=60] F --\u003e H[Fix: SessionStart recovery hook] F --\u003e I[Fix: --max-turns hard stop] Long sessions cost more than caching can offset # Claude Code resends the full conversation history with every turn. Message 201 in an eight-hour session costs as much input processing as messages 1 through 200 combined, before any caching discount applies. Prompt caching cuts the price of resending unchanged prefix content, but it discounts the resend, it doesn\u0026rsquo;t remove it. A session that stays open for eight or more hours keeps paying that growing tax on every single turn, and my own numbers show it: 99 percent of the week\u0026rsquo;s spend sat in sessions that never closed. The fix Anthropic documents for this is blunt and manual, run /compact proactively around 60 percent context fill instead of waiting until the window is nearly full, and run /clear between unrelated phases of work rather than letting one session drift across a full day. Manual is fine for a person sitting at a terminal. It does nothing for a job that fires at 3am with nobody watching.\nSubagents multiply spend before anyone notices # Fanning work out to subagents is supposed to save tokens, since each subagent\u0026rsquo;s verbose output stays in its own context and only a summary comes back to the parent. In practice, a pipeline that spawns seven or more subagents per run, planning coordination between them, runs at roughly seven times the cost of a single standard session. That number matched what I saw: sessions with any subagent fan-out accounted for the entire week\u0026rsquo;s spend. The part I\u0026rsquo;d missed is that subagents inherit the parent session\u0026rsquo;s model by default. A subagent running a mechanical task, checking test coverage, scaffolding a file, grepping logs for a pattern, gets billed at the same rate as one doing real design judgment, unless something explicitly tells it not to. Claude Code exposes that override three ways: a model: field in the subagent\u0026rsquo;s own frontmatter, an invocation parameter, or a CLAUDE_CODE_SUBAGENT_MODEL environment variable that downgrades every subagent in a session at once. None of my heavier pipelines were using any of the three.\nA billing change turned unattended jobs into first-class spenders # On 2026-06-15, Anthropic changed how programmatic usage counts: headless claude -p calls and Agent SDK calls now draw from the same weekly subscription cap as interactive sessions. Before that change, a scheduled job firing every thirty minutes around the clock was close to a free lever, since it ran against a separate allowance. After it, every one of those fires competes directly with my own interactive coding time for the same weekly ceiling. That\u0026rsquo;s the direct explanation for 62 percent of the cap gone by midweek. A continuous loop with no turn limit and no session boundary just kept spending against a budget it used to sit outside of, and nothing in the loop itself knew the rules had changed.\nHeadless jobs already get half the fix for free # Once the diagnosis was clear, the question became how to enforce session hygiene on jobs that run with no human present to type /clear or /compact. The useful discovery here is that claude -p is stateless per invocation unless you explicitly pass --continue or --resume. Every scheduled fire already starts with a clean context window by default, which is the programmatic equivalent of running /clear before every cycle. The pattern that makes this work is phase-per-process: one claude -p call per unit of work, with state persisted to files or a database on disk rather than kept in conversation memory. My own automation pipelines already write their working state that way, mostly because on-disk state is easier to debug than a live session, and it turns out that habit already satisfies the fresh-context half of the fix. I built it for a different reason and got session hygiene as a side effect.\nTwo small mechanisms close the rest of the gap # The remaining gaps needed deliberate, not incidental, fixes.\nThe first is compaction. CLAUDE_AUTOCOMPACT_PCT_OVERRIDE is an environment variable, taking a value from 1 to 100, that sets the context-fill percentage at which auto-compaction fires. Setting it to 60 turns the \u0026ldquo;compact at 60 percent, not 95 percent\u0026rdquo; guidance from a habit a person has to remember into a rule the process enforces on itself, whether or not anyone is watching that particular run.\nThe second is recovery. A session that gets cleared or auto-compacted loses whatever informal context it was tracking, and a scripted job has no one to re-explain the situation to it afterward. Claude Code\u0026rsquo;s SessionStart hook fires with a source field that reports whether the session is starting fresh, resuming, or recovering from a clear or compact event, and anything the hook\u0026rsquo;s command prints to stdout gets injected directly into the new context. A short hook that matches on clear or compact and echoes a pointer back at the run\u0026rsquo;s on-disk checkpoint state makes every compaction self-healing: the session picks itself back up without a person there to remind it where it left off.\nTwo smaller rails round this out. --resume, using a session ID captured from a prior run\u0026rsquo;s --output-format json output, is the reliable way to chain phases that genuinely need continuity; --continue is documented as unreliable inside scripted loops and can silently start a new session instead of resuming the old one. And --max-turns on every headless invocation is the hard stop that keeps a misbehaving loop from running past its budget even if the compaction and recovery hooks are working exactly as intended.\nWhat I still don\u0026rsquo;t know # I set CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=60 and wrote the SessionStart re-seed hook the same week I found the $364 session, and I added --max-turns to the campaign fires that were running unbounded. What I don\u0026rsquo;t have yet is a second week of /usage data that proves any of it actually moved the number down. The four-number diagnosis was solid, it came straight off measured usage, but the fix is still an inference from Claude Code\u0026rsquo;s documented mechanics, not a before-and-after I\u0026rsquo;ve verified with my own eyes. It\u0026rsquo;s entirely possible the compaction override behaves differently than the docs describe, or that the hook injects less useful context than I think it does, and I won\u0026rsquo;t know until a comparably heavy week passes and I pull /usage again. Until then, this is the right fix on paper, applied, and unconfirmed.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/what-a-364-dollar-claude-code-session-taught-me-about-agent-hygiene/","section":"Blog Posts","summary":"A single Claude Code session in my home lab cost $364. I found it on my own /usage report, not from a billing alert, and it was enough to make me stop and read the full week behind it instead of writing it off as one bad run.\n","title":"What a $364 Claude Code Session Taught Me About Running Agents Unattended","type":"blog"},{"content":"The idea is this: run two coding-agent orchestration suites that share none of the same prompts, config, or instruction derivation, and make them review each other\u0026rsquo;s pull requests the way two engineers with different judgment catch each other\u0026rsquo;s mistakes. Suite A opens a PR. Then suite B, built from scratch with no visibility into how A was built, reviews it cold. Suite A reads the review, decides what\u0026rsquo;s real, and applies fixes. The loop can repeat from there. I thought of this on my own, then went and checked whether it already existed somewhere, because that\u0026rsquo;s usually the fastest way to find out if an idea is obvious or overlooked.\nA single agent reviewing its own PR doesn\u0026rsquo;t catch much # A single agent reviewing its own pull request doesn\u0026rsquo;t catch much, and there\u0026rsquo;s a number behind that claim now. CodeRabbit, a production code-review tool, published a self-correction failure rate of 64.5 percent for models asked to review their own output, and named the pattern the \u0026ldquo;Homogenization Trap\u0026rdquo;: models trained on overlapping data share the same blind spots, so asking one model to grade its own work just replays the assumptions that produced the bug in the first place. That\u0026rsquo;s the whole justification for splitting author and reviewer into separate agents. It\u0026rsquo;s also why splitting them into two copies of the same model barely helps.\nThe design: independent suites, not two calls to the same agent # The design only works if the two suites are actually independent, not just two separate agent invocations. Suite A and suite B need different base models, or at minimum instruction sets and personas derived without either side looking at the other\u0026rsquo;s files, the same way two engineers who never compared notes would naturally write different code for the same ticket. When B reviews A\u0026rsquo;s PR, it should start from a fresh session rather than carry context across review rounds, because letting a reviewer agent hold onto its own earlier verdict is a known way for it to anchor on that verdict instead of looking again. And the loop needs a hard round limit, somewhere around three to five exchanges, so the respond-and-re-review cycle can\u0026rsquo;t spin forever on a disagreement neither side will drop.\nHere\u0026rsquo;s the loop itself:\nflowchart LR A[Suite A opens PR] --\u003e B[\"Suite B reviews cold(fresh session, no shared config)\"] B --\u003e C[Suite A decides what's real, applies fixes] C --\u003e D{\"Round limit reached?(3-5 exchanges)\"} D --\u003e|No| B D --\u003e|Yes| E[Loop ends] Nobody ships this as a preset, but the pieces exist # Nobody ships this exact pattern as a ready preset, but the pieces are scattered across current tools and papers. Academic research on adversarial debate between large language models already studies quality gains when review peers are genuinely different rather than cooperative copies of each other, and at least one recent paper formalizes almost the same author-reviewer-critic loop I sketched, adding a third agent that audits the reviewer\u0026rsquo;s own review. Qodo\u0026rsquo;s second-generation review tool runs several specialized agents in parallel against one PR and posted the best F1 score, a standard accuracy measure combining precision and recall, of eight review tools tested, though that\u0026rsquo;s parallel specialist review rather than an adversarial author-versus-reviewer duel. Mainstream orchestration frameworks ship a generic writer/reviewer role you can wire up yourself, but none of them package \u0026ldquo;two independently-derived agent suites duel it out\u0026rdquo; as something you install and configure. That gap is the actual whitespace here. The concept is already well studied; what\u0026rsquo;s missing is a packaged version of it.\nThe market\u0026rsquo;s clearest independent reviewer just lost its independence # The strongest counter-signal I found points the other way. Cursor, one of the more popular AI coding tools, acquired Graphite in December 2025 and folded its Diamond reviewer into Cursor\u0026rsquo;s own Bugbot, so the most notable separate-company code reviewer on the market is now owned by the same vendor that ships the authoring agent. If the industry keeps consolidating that way, buying genuine cross-vendor independence gets harder every year rather than easier, and a dueling-suite design that leans on \u0026ldquo;different vendor, different training run\u0026rdquo; as its independence guarantee is betting against that trend.\nThis is a design sketch, not something running # I want to be honest about where this stands. This is a design sketch pulled together from research, not a system I\u0026rsquo;ve built or run. I don\u0026rsquo;t have a working prototype, I don\u0026rsquo;t have latency or cost numbers from my own attempts, and everything above about round limits and fresh sessions is a plan rather than a measurement. One source did report real numbers from someone else\u0026rsquo;s cross-model adversarial review setup. Each review pass took 30 to 90 seconds. A full exchange ran three to five debate rounds with two separate models in play the entire time. Multiply that across a normal-sized PR and the wait before a suite even finishes disagreeing with itself starts to look expensive for something that might just find the same handful of issues a single well-configured reviewer agent would have caught in one pass.\nThe real question is whether disagreement finds bugs or just makes noise # The real question I can\u0026rsquo;t answer yet is whether independent agent suites disagreeing actually surfaces real bugs, or just generates plausible-sounding noise that a human still has to sort through. Two of my sources flagged \u0026ldquo;negation-blindness\u0026rdquo; as a structural weakness independent of which model you pick, meaning a reviewer agent can miss that a fix does the opposite of what\u0026rsquo;s needed regardless of how independently it was built. If that failure mode shows up in both suites, I end up with two agents that agree with each other and still miss the same bug. I don\u0026rsquo;t know yet whether that happens rarely enough to be worth the extra compute and wall-clock time, or often enough that this is a more expensive way to get the same review quality I\u0026rsquo;d get from one well-configured agent and a human final pass. Building a small version of this against a real repo, probably an old orchestration prototype I already have sitting around, is the only way I\u0026rsquo;ll find out.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/dueling-agent-orchestration-suites/","section":"Blog Posts","summary":"The idea is this: run two coding-agent orchestration suites that share none of the same prompts, config, or instruction derivation, and make them review each other’s pull requests the way two engineers with different judgment catch each other’s mistakes. Suite A opens a PR. Then suite B, built from scratch with no visibility into how A was built, reviews it cold. Suite A reads the review, decides what’s real, and applies fixes. The loop can repeat from there. I thought of this on my own, then went and checked whether it already existed somewhere, because that’s usually the fastest way to find out if an idea is obvious or overlooked.\n","title":"What If Two Independently-Built Agent Suites Reviewed Each Other's Code?","type":"blog"},{"content":"Proxmox VE won the decision, and the reason is simple: five separate workloads sharing one box is exactly the situation where isolation stops being a nice-to-have. I had an old Dell XPS 17 sitting around and a growing list of services that needed a new home. Instead of buying dedicated hardware, I closed the lid, racked the laptop, and pointed five different stacks at it. Picking the OS underneath that decision took longer than I expected, because the obvious answer (plain Ubuntu Server plus Docker Compose, matching every other machine I run) turned out to be the wrong one for this specific job.\nFive workloads on one box changes the math # A single laptop was about to run a media-automation stack, a financial data-ingestion pipeline, a research-automation pipeline, a knowledge-graph service built on LightRAG, and a Prometheus/Grafana monitoring stack. Each of those is its own multi-container application with its own dependencies, its own restart policy, and its own blast radius if something goes wrong. Running all five as Docker Compose stacks directly on one Ubuntu install works fine until one of them needs a kernel module the others don\u0026rsquo;t, or a bad docker compose down -v on one project takes out a volume mount that another project happened to share. Isolation is the whole argument. When you\u0026rsquo;re running one app, bare metal plus Docker is simpler and I\u0026rsquo;d pick it again without hesitation. When you\u0026rsquo;re running five unrelated apps that used to live on five different sets of assumptions, a hypervisor layer that can wall each one off starts paying for itself.\nProxmox VE\u0026rsquo;s per-workload containers made the isolation case concrete # Proxmox VE is a free, Debian-based hypervisor that runs VMs and LXCs (Linux containers that isolate at the kernel-namespace level, lighter than a full VM but heavier than a Docker container) side by side, managed through one web UI and API. The plan that came out of the research was Proxmox on bare metal, with one LXC per workload, each running its own Ubuntu or Debian userland and its own Docker daemon inside. That gives every stack its own filesystem snapshot and its own rollback point. If the knowledge-graph service breaks something in an upgrade, I can snapshot before, wreck the container trying to fix it, and roll back in under a minute, without touching the other four workloads. A flat Docker host can\u0026rsquo;t give me that; a bad apt upgrade or a stray volume prune affects everything on the box at once. As of Proxmox VE 9.2 in mid-2026, it\u0026rsquo;s built on Debian 13, which means the underlying package base is the same stable Debian everyone already trusts, just with a newer kernel and better hardware support layered on top.\nHere\u0026rsquo;s the shape of the box either way — the rejected flat host on top, the isolation Proxmox actually buys below it:\nflowchart TD subgraph Flat[\"Flat Docker host (rejected)\"] H1[One Ubuntu host] --\u003e D1[\"5 Docker Compose stacks,shared kernel, shared blast radius\"] end subgraph Proxmox[\"Proxmox VE (chosen)\"] H2[Proxmox bare metal] --\u003e L1[LXC: media automation] H2 --\u003e L2[LXC: financial data pipeline] H2 --\u003e L3[LXC: research automation] H2 --\u003e L4[LXC: LightRAG knowledge graph] H2 --\u003e L5[LXC: Prometheus/Grafana] end Fedora Server lost on two separate grounds # Fedora Server was in the running early and got cut for two reasons, not one. First, Fedora Server defaults to Podman instead of Docker, and every workload I was moving over already had working docker-compose.yml files. Moving to Fedora would have meant either bolting Docker back on top of a distro that doesn\u0026rsquo;t want it there, or rewriting five stacks\u0026rsquo; worth of compose files against Podman\u0026rsquo;s command differences. Second, Fedora ships on roughly a 13-month release cadence, and this box is meant to be racked and left alone. A \u0026ldquo;rack it and forget it\u0026rdquo; machine and a distro that forces a major-version upgrade about once a year are a bad match. Either problem alone might have been worth working around. Together they weren\u0026rsquo;t worth it.\nThe real downside: this isn\u0026rsquo;t a flat SSH target anymore # I run two other machines on this network the same way: SSH in, you\u0026rsquo;re on the box, you run Docker Compose, done. Proxmox breaks that pattern. Now I SSH into the hypervisor host first, then hop into whichever LXC I actually need to touch. That\u0026rsquo;s a second layer of indirection every single time I want to check a log or restart a container, and it\u0026rsquo;s a real cost, not a hypothetical one. I went into this decision aware of it and still made the tradeoff, because five isolated workloads beat one flat access pattern, but anyone copying this setup should know that convenience is what you\u0026rsquo;re giving up. I don\u0026rsquo;t love it. Some days I still type the wrong SSH target out of habit and have to back out and hop again.\nThe laptop-specific gotchas turned out to matter more than the distro choice itself. Built-in Wi-Fi cannot bridge to Proxmox VMs or LXCs, full stop. The wireless card only associates with the access point directly, so any container trying to reach the network through a bridged Wi-Fi interface gets its frames silently dropped at the AP. That\u0026rsquo;s not a driver bug you can patch around; it\u0026rsquo;s how 802.11 associations work. The fix is wired Ethernet only, which meant confirming the XPS 17 actually had a working port before I bothered racking it. The other gotcha is that laptops suspend when you close the lid, and a suspended hypervisor is a hypervisor that just stopped running your services. The fix lives in two config files: HandleLidSwitch=ignore in /etc/systemd/logind.conf (and the matching line in sleep.conf) so closing the lid doesn\u0026rsquo;t trigger suspend, plus consoleblank=300 on the kernel boot line so the display blanks instead of the system sleeping. Both fixes are well-documented and have shown up independently across guides going back to 2022, so this isn\u0026rsquo;t a fragile hack, it\u0026rsquo;s the standard answer to \u0026ldquo;how do I run a laptop headless.\u0026rdquo;\nThe box is live, and the isolation argument held up # I took the Proxmox route, and the box has been running since. Media automation and a newer monitoring workload moved over cleanly, each in its own container, and I\u0026rsquo;ve already used a snapshot rollback once when an upgrade inside one LXC went sideways, without any of the other four workloads noticing. The one thing still unresolved is a research pipeline that ended up duplicated across two machines during the migration, an unfinished cleanup rather than a design flaw. If I were doing this again for a single app, I\u0026rsquo;d skip the hypervisor and just run Docker on bare metal. The extra SSH hop is a real tax I pay every day. But for a laptop absorbing five workloads that used to trust five different sets of assumptions about the box under them, the isolation is worth the tax.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/proxmox-for-the-xps-17-offload-box/","section":"Blog Posts","summary":"Proxmox VE won the decision, and the reason is simple: five separate workloads sharing one box is exactly the situation where isolation stops being a nice-to-have. I had an old Dell XPS 17 sitting around and a growing list of services that needed a new home. Instead of buying dedicated hardware, I closed the lid, racked the laptop, and pointed five different stacks at it. Picking the OS underneath that decision took longer than I expected, because the obvious answer (plain Ubuntu Server plus Docker Compose, matching every other machine I run) turned out to be the wrong one for this specific job.\n","title":"Why the XPS 17 Offload Box Runs Proxmox, Not Plain Ubuntu","type":"blog"},{"content":"This site had no image pipeline until this week. Every image in every post loaded at its original file size and format, usually a multi-megabyte PNG screenshot, with no responsive sizing and no loading placeholder. Diagrams worked through exactly one path: a Blowfish theme shortcode you had to remember to wrap your diagram in by hand, with no way to drop a diagram into a plain fenced code block the way you would in a GitHub README or almost anywhere else that renders Markdown. I fixed both problems in the same pass, because they share a mechanism: Hugo render hooks, which let a site override how the built-in Markdown renderer turns one specific element — an image, a code block — into HTML.\nRender hooks, not a CDN # The obvious alternative to fixing this in Hugo would have been an image CDN — a hosted service like Cloudinary or imgix that resizes and reformats images on request. I didn\u0026rsquo;t want a third-party dependency for something Hugo already does natively at build time. Every image on this blog is a file checked into the repo. Hugo\u0026rsquo;s resources.Get and the image processing methods it exposes (.Resize, format conversion, quality settings) run once, during hugo build, and the output is a static file next to everything else this site already serves. No runtime cost, no external service, no new failure mode when that service has an outage.\nA render hook is Hugo\u0026rsquo;s supported way to intercept one piece of that build. Drop a template at layouts/_default/_markup/render-image.html and every Markdown image reference in every post routes through it instead of Hugo\u0026rsquo;s default renderer. Same idea for code blocks: layouts/_default/_markup/render-codeblock-mermaid.html intercepts only the fenced blocks tagged with the language name mermaid, leaving every other code block (Python, Bash, YAML, whatever) untouched.\nWebP conversion, responsive srcset, and a blur-up placeholder # The image hook does three things to every local raster image (a PNG or JPEG that isn\u0026rsquo;t an SVG and isn\u0026rsquo;t loaded from a remote URL):\nConverts it to WebP at two widths, 800px and 1280px, quality 75. WebP is a modern image format that produces meaningfully smaller files than PNG or JPEG at the same visual quality — the actual win here, since the original screenshots on this blog were often 1–3MB PNGs. Builds a srcset so the browser picks whichever of the two sizes fits the reader\u0026rsquo;s screen, instead of always downloading the largest version. Generates a low-quality placeholder. LQIP stands for low-quality image placeholder: a tiny, heavily compressed preview — 24px wide, WebP quality 40 — encoded directly into the HTML as a base64 data URI and shown as a blurred background while the real image loads, then swapped out once it finishes (onload, checking the image actually has real pixels rather than firing on a broken image). Neither the resizing nor the srcset width, both capped at the source image\u0026rsquo;s own width so a small source image never gets upscaled past its native resolution.\nTwo cases skip all of this on purpose. Remote images (anything with an http://, https://, or data: URL) pass straight through, since Hugo can\u0026rsquo;t resize a file it doesn\u0026rsquo;t have locally. SVGs also pass through unmodified — SVG is already a compact vector format, and converting one to a raster WebP would only make it bigger and blurrier. There\u0026rsquo;s also a site-wide escape hatch, a disableImageOptimizationMD parameter that reverts every image on the site to the original, unconverted file, for the rare case where exact pixel fidelity matters more than page weight.\nHere\u0026rsquo;s the decision flow the hook actually runs, from a Markdown image reference to the final rendered figure:\nflowchart TD A[Markdown image reference] --\u003e B[render-image.html hook fires] B --\u003e C{Remote URL, or local resource not found?} C --\u003e|Yes| D[Plain img tag, no conversion] C --\u003e|No, local resource found| E{SVG, or optimization disabled by site param?} E --\u003e|Yes| D E --\u003e|No| F[Responsive path] F --\u003e G[Resize to 800w and 1280w WebP, quality 75, capped at source width] F --\u003e H[Resize to 24px WebP, quality 40, base64-encode] G --\u003e I[img src + srcset + sizes] H --\u003e J[Inline background-image data URI, cleared once the real image loads] I --\u003e K[Rendered figure: responsive WebP with blur-up placeholder] J --\u003e KHere\u0026rsquo;s a real image going through that exact path, reused from an earlier post on this blog rather than a synthetic test image, so the pipeline is doing real work instead of rendering a stock placeholder:\nThe Docker Compose + VPN topology from an earlier post on this blog, now served as WebP with a blur-up placeholder The bug that would have shipped: images under 800px skipped WebP # Blowfish, the theme this site runs on, already had an image render hook, and the one I built started as a fork of it rather than something written from scratch. Its responsive-image logic resized to WebP only inside a conditional gated on the source image\u0026rsquo;s width, and that conditional was written so images narrower than 800px fell through without ever hitting the .Resize call. A screenshot that happened to be, say, 600px wide would render as a plain, unconverted PNG — no WebP, no srcset, no LQIP, and no error to indicate anything had gone wrong.\nI caught this during spec review, before it shipped, by deliberately testing against a narrow image instead of only the wide screenshot used elsewhere in this post. The fix was to make the WebP conversion unconditional: every local raster image gets resized to WebP now, with each target width capped at math.Min(originalWidth, 800) (or 1280 for the larger variant) so a small source image gets downsized cleanly and never upscaled. A conditional that silently skips work instead of erroring is invisible until someone tests the exact input it was written to exclude — this one only surfaced because the test plan called for a narrow image specifically instead of reusing the same wide screenshot every other check already covered.\nDiagrams from a plain code fence, not just a custom shortcode # Before this, the only way to add a diagram to a post was Blowfish\u0026rsquo;s mermaid shortcode — Hugo\u0026rsquo;s mechanism for calling a custom template from inside Markdown by name, wrapped around the content it applies to. It works, but it\u0026rsquo;s specific to this theme. Paste the same Markdown into GitHub, or into any other Hugo site without that exact shortcode installed, and it renders as literal, broken-looking text instead of a diagram.\nMermaid — the diagramming library, not the theme feature — has a real, portable convention for this: a fenced code block tagged with the word mermaid as its language name, the same triple-backtick-plus-language convention you\u0026rsquo;d use for any other code block, just with mermaid in place of python or bash. GitHub, GitLab, and most Markdown renderers already recognize that convention natively. Hugo\u0026rsquo;s code-block render hook lets this site recognize it too: render-codeblock-mermaid.html intercepts any fenced block tagged that way and wraps its raw content in a \u0026lt;pre class=\u0026quot;mermaid\u0026quot;\u0026gt; element — the exact markup the existing shortcode already produced, so the same CSS and the same Mermaid JavaScript runtime pick it up identically no matter which syntax wrote it. The diagram earlier in this post, the one showing the image hook\u0026rsquo;s decision flow, is a real instance of that fenced block, not a mockup.\nLoading the Mermaid bundle exactly once, from either entry point # Mermaid\u0026rsquo;s JavaScript runtime is a real cost — tens of kilobytes a reader\u0026rsquo;s browser has to fetch and execute — so it should only load on pages that actually use it, and it should never load twice on the same page. Blowfish\u0026rsquo;s theme already handled the first half of that for the shortcode: a partial checks .Page.HasShortcode \u0026quot;mermaid\u0026quot; and only then fetches, minifies, concatenates, and fingerprints the Mermaid library and its config into one bundle.\nRather than fork that theme file too, which would mean re-syncing it by hand on every future Blowfish update, I added a second, narrower check in a site-level partial, extend-head-uncached.html. It loads the same bundle only when the page\u0026rsquo;s raw source contains a fenced block tagged mermaid and the shortcode is absent. That \u0026ldquo;and shortcode is absent\u0026rdquo; clause is the double-load guard: on a page that uses only the shortcode, the theme\u0026rsquo;s own check already loads the bundle, so this second check backs off. On a page that uses only the fenced-block syntax, the theme\u0026rsquo;s check is false (no shortcode), so this one fires instead. On a page using both — like this one — the theme\u0026rsquo;s check fires and loads it, and this one backs off for the same reason as the shortcode-only case. One script tag, regardless of which syntax, or both, a given post uses.\nThe old shortcode syntax still renders on the same page # The diagram below is a regression check more than content in its own right: it\u0026rsquo;s written with the original mermaid shortcode syntax, sitting on the same page as the fenced diagram above, to confirm both entry points coexist without loading the Mermaid bundle twice or conflicting with each other.\nflowchart LR A[Shortcode entry point] --\u003e B[Blowfish's original loader: HasShortcode check] B --\u003e C[Same Mermaid runtime bundle] C --\u003e D[Renders next to the fenced-block diagram above] Where this stands # hugo build runs clean across every existing post plus this one, and the build output confirms both diagrams render and the image above comes out as WebP with a srcset and a blur-up placeholder rather than a flat PNG. I also wrote real browser tests, not just a clean build, to catch a regression here automatically: they confirm the bundle loads exactly once on this page (both syntaxes present), stays absent on pages with neither, that the diagram actually renders as an SVG rather than sitting as unrendered text, that its colors really change between light and dark mode after clicking the appearance switcher, and that the LQIP placeholder clears once the real image loads rather than just being present in the markup. One gap I\u0026rsquo;m not pretending isn\u0026rsquo;t there: there\u0026rsquo;s still no isolated fixture anywhere on this site for \u0026ldquo;fenced block only, no shortcode\u0026rdquo; or \u0026ldquo;shortcode only, no fenced block\u0026rdquo; in separate pages — this post exercises both at once, which proves the double-load guard but not each syntax fully alone. That guard\u0026rsquo;s logic is simple enough to have checked by reading the template directly, so I\u0026rsquo;m treating it as covered, not untested.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/native-hugo-image-pipeline-webp-lqip-and-mermaid/","section":"Blog Posts","summary":"This site had no image pipeline until this week. Every image in every post loaded at its original file size and format, usually a multi-megabyte PNG screenshot, with no responsive sizing and no loading placeholder. Diagrams worked through exactly one path: a Blowfish theme shortcode you had to remember to wrap your diagram in by hand, with no way to drop a diagram into a plain fenced code block the way you would in a GitHub README or almost anywhere else that renders Markdown. I fixed both problems in the same pass, because they share a mechanism: Hugo render hooks, which let a site override how the built-in Markdown renderer turns one specific element — an image, a code block — into HTML.\n","title":"A Native Hugo Image Pipeline: WebP, LQIP Blur-Up, and Mermaid Diagrams","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/hugo/","section":"Tags","summary":"","title":"Hugo","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/image-optimization/","section":"Tags","summary":"","title":"Image Optimization","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/mermaid/","section":"Tags","summary":"","title":"Mermaid","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/performance/","section":"Tags","summary":"","title":"Performance","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/static-site/","section":"Tags","summary":"","title":"Static Site","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/web-development/","section":"Categories","summary":"","title":"Web Development","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/webp/","section":"Tags","summary":"","title":"WebP","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/categories/case-study/","section":"Categories","summary":"","title":"Case Study","type":"categories"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/computer-vision/","section":"Tags","summary":"","title":"Computer Vision","type":"tags"},{"content":"My resale-clothing monitor\u0026rsquo;s hardest problem isn\u0026rsquo;t finding new listings. It\u0026rsquo;s deciding which ones match my taste well enough to interrupt me about, and the design leans hard toward false positives over false negatives in a way I can defend today but haven\u0026rsquo;t stress-tested. This is part 3 of a series; part 1 covers the shared architecture, and part 2 covers a sibling project, an estate-sale scanner, that runs on the same foundation.\nA free rules layer rejects most listings before any model sees them # The monitor watches several secondhand clothing marketplaces. Its pre-filter is a set of deterministic rules that runs before any model call: a fast-fashion brand blocklist, a per-brand minimum plausible price (a \u0026ldquo;designer\u0026rdquo; item priced far below that floor is usually a knockoff), and a price ceiling by category. Together those three rules eliminate roughly 40 to 60 percent of raw listings for free.\nSize is never a hard-reject rule, and that\u0026rsquo;s deliberate. Sizing across resale platforms is too unreliable to gate on mechanically. Brands run differently, cuts vary, sellers mislabel. Instead of a brittle \u0026ldquo;reject anything not size L\u0026rdquo; rule, the raw size text and any stated measurements get passed to the model as a soft signal, with instructions that measurements in the description always override the label.\nScoring is two passes, and only the ambiguous cases get the expensive one # Every new listing (title, brand, a truncated description, price, condition, size) goes through a local model first, batched 15 to 20 listings per call. That range matters: fewer wastes the fixed cost of the system prompt, and past roughly 30 the model\u0026rsquo;s attention starts to degrade. The model returns a verdict, YES, MAYBE, or NO, broken into three independent dimensions (quality, value, aesthetic) plus a separate size read.\nOnly listings that come back MAYBE and have a usable image go to a second pass with a vision-capable model. A MAYBE with no resolvable image just stays a MAYBE and still gets surfaced, at lower confidence, rather than getting silently dropped. A parse error or malformed model output defaults the same way.\nThe provider for each pass, local, cloud, or a hybrid, sits behind one interface, so which backend actually runs a given scoring pass is a config change, not a code change. Once a listing has a real score, it\u0026rsquo;s never re-scored. That alone is the single biggest cost reduction in the pipeline, ahead of anything model-related.\nHere\u0026rsquo;s the scoring pipeline a listing actually moves through:\nflowchart TD A[New listing] --\u003e B{\"Rules pre-filter:brand blocklist, price floor/ceiling\"} B --\u003e|Rejected, 40-60%| C[Discarded, free] B --\u003e|Passed| D[Local model, batched 15-20/call] D --\u003e E{Verdict per dimension} E --\u003e|NO| C E --\u003e|YES| F[Surfaced as alert] E --\u003e|MAYBE + usable image| G[Vision model, second pass] E --\u003e|MAYBE, no image| H[Surfaced at lower confidence] G --\u003e F The bias toward false positives has no counterweight yet # Missing a genuinely good item is worse than one extra alert I dismiss in two seconds, and for a system with one user and nothing riding on a bad alert, I still think that\u0026rsquo;s the right call. What it doesn\u0026rsquo;t do is push back against alert volume creeping up as more edge cases land in MAYBE instead of NO over time. Nothing in the current design notices that drift or corrects for it. If this ever had to serve more than one household, that gap would be the first thing I\u0026rsquo;d have to actually solve instead of shrug at.\nFeedback splits into two tiers with different lifespans # Every run, the system prompt gets appended with a rotating set of my most recent thumbs-up and thumbs-down reactions to past alerts. The effect is staged: under 10 feedback events, nothing measurable; 10 to 25, a noticeable improvement; 25 to 50, strong calibration. Past 50, the oldest examples age out in favor of recent ones.\nSeparately, I can hand-write known-good and known-bad example items directly into config. Those never rotate out, and exist so the system has some ground truth before a single real alert has fired.\nOne migration broke three things at once, silently # The one incident I\u0026rsquo;d flag hardest here broke silently across three places at once instead of one. The monitor\u0026rsquo;s alerts and feedback originally rode the same channel: a chat bot where a thumbs-up or thumbs-down tap wrote straight back to the feedback table, no extra infrastructure needed. When the design changed to stop depending on that bot\u0026rsquo;s webhook, the alert transport got swapped to a self-hosted push service in one large change. The feedback-ingestion path got gutted down to a disabled stub with no replacement wired up yet, while the docs of record still described the old bot. For a stretch, the code, the deployment config, and the docs each told a different story about how alerts and feedback worked, and the system\u0026rsquo;s only learning mechanism sat fully severed with no error to say so. The fix restored feedback through a proper API endpoint on the dashboard instead of a chat bot\u0026rsquo;s callback.\nWhere both projects\u0026rsquo; open questions actually meet # Both this project\u0026rsquo;s MAYBE-drift and the estate scanner\u0026rsquo;s cascade-complexity doubt (in part 2) share a shape I didn\u0026rsquo;t notice until writing all three of these posts back to back. Every incident across both systems announced itself eventually, through a dashboard that looked stale or a log that looked suspiciously clean. Neither open question has that kind of tell. Alert volume creeping up over months as more edge cases land in MAYBE instead of NO, or a feedback loop gradually reinforcing a preference I don\u0026rsquo;t actually hold anymore, wouldn\u0026rsquo;t announce itself at all. Nothing in either system watches for drift like that. I\u0026rsquo;d have to notice it myself, on some Saturday, looking at a list that feels a little worse than it used to for reasons I can\u0026rsquo;t immediately name. I haven\u0026rsquo;t built anything that would catch it sooner than that, and I don\u0026rsquo;t have a good reason why not beyond not having hit it yet.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/deciding-what-fits-resale-clothing-monitor/","section":"Blog Posts","summary":"My resale-clothing monitor’s hardest problem isn’t finding new listings. It’s deciding which ones match my taste well enough to interrupt me about, and the design leans hard toward false positives over false negatives in a way I can defend today but haven’t stress-tested. This is part 3 of a series; part 1 covers the shared architecture, and part 2 covers a sibling project, an estate-sale scanner, that runs on the same foundation.\n","title":"Deciding What Fits: Inside My Resale-Clothing Monitor","type":"blog"},{"content":"My estate-sale scanner\u0026rsquo;s real job is deciding which of a week\u0026rsquo;s new listings are worth a Saturday drive, and the interesting part isn\u0026rsquo;t the scraping. It\u0026rsquo;s how the system scores unlabeled photos, and an asymmetric feedback loop that treats a bad sale and a good sale as very different kinds of evidence. This is part 2 of a series; part 1 covers the shared architecture and GPU constraints behind this and a second project.\nEvery photo runs through four cheap gates before any paid model call # The scanner pulls new listings from a regional aggregator once a week, then runs each photo through a pipeline in order:\nPerceptual-hash dedup. Catches the same photo re-uploaded across listings. A quality gate. Brightness and blur checks, cheap and CPU-only, drop photos too dark or blurry to read. A free local pre-filter. A small local Ollama call answers PASS or SKIP on things like empty rooms, driveways, or cardboard boxes, before any money gets spent on a stronger model. It\u0026rsquo;s fail-open: if the model call errors, the photo passes through anyway. An outage never suppresses a real find. It just costs more that week. Full vision analysis. Either a local Ollama model or, for volume, a hosted GPU endpoint running a larger vision-language model. Here\u0026rsquo;s that pipeline as an actual flow:\nflowchart TD A[New listing photo] --\u003e B[Perceptual-hash dedup] B --\u003e C[Quality gate: brightness/blur] C --\u003e D[\"Free local pre-filter (fail-open)\"] D --\u003e|PASS or error| E[Full vision analysis] D --\u003e|SKIP| F[Discarded] E --\u003e G[Item list: maker, era, materials, condition, confidence]The model gets told what I collect: quality furniture and antiques, kitsch and camp collectibles, vintage electronics. It lists each item with a maker guess, era, materials, condition, and a confidence tag:\nDanish teak side table, likely 1960s, veneer chip on one corner [high] Chalkware TV lamp, mid-century, black light wear on base [medium] NOTHING Plain text, not JSON. An internal comparison found the plain-text format caught meaningfully more real items than forcing the same model into strict JSON output. Worth knowing before you assume structured output is free.\nTwo separate scores exist because they answer two separate questions # One score decides whether the rest of a sale\u0026rsquo;s photos are worth analyzing at all. Process the first quarter of a sale\u0026rsquo;s photos; if they come back strong, run the rest; if they come back empty, spot-check a handful from later in the listing before giving up on the sale entirely. That\u0026rsquo;s pure cost control. It decides how many model calls a sale gets, not whether any single item is good.\nA second, separate score, built from a curated brand list, era keywords, and the model\u0026rsquo;s own confidence tag, is what the dashboard actually sorts by. The budget heuristic optimizes for not wasting calls on a dead sale. The display score optimizes for what to look at first. Conflating the two would have made cheap sales look worse than they are.\nA \u0026ldquo;waste\u0026rdquo; outcome teaches the system more than a \u0026ldquo;good\u0026rdquo; one does # After visiting a sale, I log an outcome: good, meh, or waste. That single decision, what to do with each label, is the anti-overfit design in this whole project, and it\u0026rsquo;s deliberately lopsided.\nA \u0026ldquo;waste\u0026rdquo; outcome propagates in bulk. Every photo from that sale becomes a confirmed-negative training example, because \u0026ldquo;the whole sale was junk\u0026rdquo; is a clean, complete signal. A \u0026ldquo;good\u0026rdquo; outcome does not auto-label every photo in the sale as good. It only proves something there was worth it, not which item. Auto-labeling the whole sale would teach a future ranker that the box of tube socks next to the good chair was also desirable. Positive labels only get created when I tap the specific item that earned the trip. It\u0026rsquo;s slower to build a clean positive set this way, but a small, correct one beats a large, contradictory one.\nThere\u0026rsquo;s also a real ground-truth run behind the scenes. Occasionally I run every surviving photo from a batch of sales through the strongest model available, no sampling, no budget limit, and treat that as the reference answer. Comparing a cheaper run\u0026rsquo;s recall against that reference is how I picked a monthly spend target instead of guessing at one.\nThe tiered cascade\u0026rsquo;s complexity is the part I\u0026rsquo;m least sure about # The reference-pass math tells me the cheap tiers catch most of what the expensive tier would have found, which is the number I actually wanted. It doesn\u0026rsquo;t tell me whether a dumber two-tier version, a quality gate plus one model call, would have caught nearly as much for a lot less engineering. I never built that version to find out. The tiered design looks rigorous because I can point at a recall number that justifies it, and I\u0026rsquo;d be lying if I said that number wasn\u0026rsquo;t also the thing that let me stop second-guessing myself and ship it.\nThree failures that never threw an error # Every incident here shares one shape: nothing crashed, nothing logged an error, and the system kept looking healthy from the outside while quietly doing the wrong thing.\nThe free pre-filter asks a model to answer with exactly one word, PASS or SKIP, within a small token budget. The model in use is reasoning-tuned. It spends part of that budget thinking before it answers, and at the original budget it never got past its own reasoning. Every single call came back with an empty response. Because the fail-open logic treats anything that isn\u0026rsquo;t literally SKIP as a pass, the gate silently passed everything, every time, for an unknown stretch. Fixed by raising the token budget and telling the model explicitly to skip its reasoning step. It\u0026rsquo;s now the first thing checked whenever this project swaps in a new model.\nWorse, a run could fail completely and still report success. Per-image failures were counted internally but never surfaced anywhere or reflected in the run\u0026rsquo;s exit status. A week where every single paid vision call failed still logged \u0026ldquo;scan complete, 0 findings\u0026rdquo; and exited clean, indistinguishable from a genuinely quiet week, despite real money spent on every failed call. The fix split \u0026ldquo;found nothing\u0026rdquo; into three honest, differently-alarmed outcomes: genuinely nothing found, the source site\u0026rsquo;s page structure likely changed, or the vision backend failed enough calls that the count can\u0026rsquo;t be trusted.\nFor a period, the scan ran on one machine and served the dashboard from a different one, each with its own separate copy of the same SQLite file. The dashboard was quietly showing stale results relative to what the last real scan had actually found, with nothing anywhere to flag that the two had diverged. Fixed by consolidating both onto a single always-on host. The kind of bug that\u0026rsquo;s obvious in hindsight and invisible while it\u0026rsquo;s happening.\nI don\u0026rsquo;t have a general fix for this class of bug beyond looking harder at exactly the places I\u0026rsquo;m most tempted to assume are fine, and I\u0026rsquo;m not confident I\u0026rsquo;ve caught the last one.\nPart 3 covers the resale-clothing monitor, its own scoring problem, and where the two projects\u0026rsquo; open questions actually converge.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/deciding-whats-worth-a-saturday-estate-sale-scanner/","section":"Blog Posts","summary":"My estate-sale scanner’s real job is deciding which of a week’s new listings are worth a Saturday drive, and the interesting part isn’t the scraping. It’s how the system scores unlabeled photos, and an asymmetric feedback loop that treats a bad sale and a good sale as very different kinds of evidence. This is part 2 of a series; part 1 covers the shared architecture and GPU constraints behind this and a second project.\n","title":"Deciding What's Worth a Saturday: Inside My Estate-Sale Scanner","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/local-llm/","section":"Tags","summary":"","title":"Local LLM","type":"tags"},{"content":"Two personal tools I\u0026rsquo;ve built, an estate-sale scanner and a resale-clothing monitor, run on the exact same architecture: scrape listings, score every photo with a local vision model, surface only the ones worth my attention. This post covers that shared foundation, one SQLite-only pipeline and one shared GPU, and the two decisions in it I\u0026rsquo;m still not fully sure were right. Two follow-ups go deep on how each project decides what actually counts as a match: Deciding what\u0026rsquo;s worth a Saturday for the estate-sale scanner, and Deciding what fits for the resale monitor.\nOne pipeline, two projects, no message queue # Both projects are the same four stages, talking to each other through a single SQLite database instead of a queue:\nflowchart LR A[Listings site] --\u003e B[\"Scrape(new items)\"] B --\u003e C[\"Prefilter(free, no LLM cost)\"] C --\u003e D[\"LLM / Vision Score(Ollama + cloud escalationfor hard cases)\"] D --\u003e E[\"Alertdashboard / push\"] B -.-\u003e S[(\"SQLitesingle writer per stage\")] C -.-\u003e S D -.-\u003e S E -.-\u003e SA run is one process that walks through the stages in order and writes its results to disk as it goes, and the next stage reads whatever the last one left behind. I\u0026rsquo;d defend that against anyone who reflexively reaches for a queue on a hobby project this size; at dozens to low hundreds of listings per run, a queue buys nothing and costs a service to operate and monitor. I don\u0026rsquo;t think the choice is free, though. The first time I want two scrapers writing to the same SQLite file at once, or want one stage to retry independently of the one before it, this is the design that starts to hurt. I haven\u0026rsquo;t hit that yet. I expect I will.\nWhat differs between the two projects is entirely inside the middle two boxes, what gets filtered out before it costs anything, and what the model actually gets asked to judge. That\u0026rsquo;s what the next two posts cover.\nOne shared GPU forces the same cost tradeoff on both projects # Both pipelines lean on the same home-lab constraint: one GPU, shared with everything else that machine does, including gaming and media transcoding. That constraint shapes the architecture more than almost anything else.\nThe strongest available vision model, run on every image unthrottled, priced out at ten to twenty times a reasonable monthly budget. The obvious alternative, running everything locally on the home GPU, worked, but a full unbounded pass over a week\u0026rsquo;s photos took on the order of a day, on a machine other people in the house wanted to use in the meantime. Neither was acceptable, and that\u0026rsquo;s the actual reason both projects ended up with a tiered cascade instead of calling the best model on everything: cheap local checks first, a stronger model only on what survives, and an optional even-stronger model reserved for genuinely ambiguous cases.\nFitting a large vision model onto a consumer-class GPU has its own failure mode. The full-precision checkpoint of one candidate model didn\u0026rsquo;t fit and left the worker in a permanently unhealthy state, until I switched to an FP8-quantized build of the same model, which loaded cleanly. Serverless GPU workers that scale to zero when idle, the thing that keeps cost near zero between runs, carry a real cold-start cost too. One backend took roughly eight minutes to spin up from cold, against a hardcoded two-minute timeout on the client side. The result was a guaranteed failure on the first image of every single run, and it stayed that way until someone timed the cold start directly instead of assuming a fixed timeout was generous enough.\nNeither Ollama instance gets addressed directly by IP in either codebase anymore. Both pipelines read a plain environment variable for wherever inference happens to be running. That decision paid off the first time I moved the GPU host.\nWhat\u0026rsquo;s next # The scoring logic, the part that actually decides what\u0026rsquo;s worth flagging, is different enough between the two projects that it doesn\u0026rsquo;t fit here. Part 2 covers the estate-sale scanner\u0026rsquo;s asymmetric feedback loop, where a bad sale and a good sale teach the system very different things. Part 3 covers the resale monitor\u0026rsquo;s two-pass scoring and a false-positive bias I haven\u0026rsquo;t fully stress-tested.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/scrape-score-alert-resale-hunting-pipelines-local-vision-models/","section":"Blog Posts","summary":"Two personal tools I’ve built, an estate-sale scanner and a resale-clothing monitor, run on the exact same architecture: scrape listings, score every photo with a local vision model, surface only the ones worth my attention. This post covers that shared foundation, one SQLite-only pipeline and one shared GPU, and the two decisions in it I’m still not fully sure were right. Two follow-ups go deep on how each project decides what actually counts as a match: Deciding what’s worth a Saturday for the estate-sale scanner, and Deciding what fits for the resale monitor.\n","title":"Scrape, Score, Alert: The Pattern Behind Two Home-Lab Vision Pipelines","type":"blog"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/sqlite/","section":"Tags","summary":"","title":"SQLite","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/typescript/","section":"Tags","summary":"","title":"TypeScript","type":"tags"},{"content":"","date":"10 August 2026","externalUrl":null,"permalink":"/tags/github/","section":"Tags","summary":"","title":"GitHub","type":"tags"},{"content":"GitHub\u0026rsquo;s per-repo Agents tab is a mission-control surface, live since January 26, 2026, where GitHub-hosted coding agents pick up issues and turn them into PRs without anyone opening a terminal. Copilot\u0026rsquo;s own agent lives there by default, and Claude and Codex are selectable alongside it as \u0026ldquo;picked\u0026rdquo; third-party agents. The tab is part of Agent HQ, the umbrella GitHub announced on October 28, 2025, meant to give every agent vendor one shared surface inside Issues, PRs, and Actions instead of a pile of separate integrations. My read after digging into how it actually works: this is a real product, not a rebrand of anything Anthropic ships, and it solves a narrower slice of my workflow than local Claude Code already covers. Whether I keep reaching for it once the novelty wears off is the part I genuinely don\u0026rsquo;t know yet.\nThe Agents tab bills through Copilot, not through your Anthropic account # Running Claude or Codex inside GitHub\u0026rsquo;s Agents tab requires a paid Copilot plan: Pro at $10 a month with $15 of included AI credits, Pro+ at $39 with $70, Max at $100 with $200. Every session the tab runs draws down those credits, and GitHub moved the whole system to usage-based credit billing on June 1, 2026, so cost tracks how much work the agent actually does rather than a flat seat price. Anthropic\u0026rsquo;s own bridge into GitHub runs on a completely separate path: the claude-code-action GitHub App, which you install yourself by running /install-github-app from the Claude Code CLI, and which bills straight against an ANTHROPIC_API_KEY stored as a repo secret. Same Claude model either way, but two different accounts get charged, and two different places end up holding the session history. That\u0026rsquo;s worth deciding on purpose rather than defaulting into both.\nIt already reads the instructions file I wrote for a different reason # GitHub\u0026rsquo;s Copilot cloud agent reads whatever CLAUDE.md sits at a repo\u0026rsquo;s root, along with AGENTS.md and path-scoped .github/instructions/**/*.instructions.md files, with no extra setup on my end. Any repo I maintain that already keeps a CLAUDE.md as its canonical instructions file is handing that same document to GitHub\u0026rsquo;s agent the instant the Agents tab gets turned on for it. An excludeAgent property exists for scoping a file to specific agents, useful once Copilot needs house rules that shouldn\u0026rsquo;t also apply to Claude or Codex running in the same repo, though I haven\u0026rsquo;t hit a case where I\u0026rsquo;ve needed it. GitHub caps a single instructions file around 1,000 lines before response quality reportedly drops, a ceiling worth knowing before any CLAUDE.md grows past what an agent, local or cloud, can actually use.\nThe permission model is generic where mine is already specific # The cloud agent only touches the repo it\u0026rsquo;s assigned to, and any Actions workflow its PR triggers needs write-access approval before it runs. GitHub built that sandbox to hold for any repo any customer points it at, which makes it necessarily generic. My local Claude Code sessions already operate under a tighter and more specific version of the same idea: I decide per repo what a session is allowed to touch, and nothing runs unsupervised against something live without a change-control gate I wrote for that exact system. The two guardrails aren\u0026rsquo;t competing with each other. They sit at different points in the pipeline, and for anything touching a running service I still trust the gate I built over one designed to be safe for every customer\u0026rsquo;s repo at once.\nTask suitability draws the same line I already draw myself # GitHub is explicit about what belongs in the Agents tab: bug fixes, doc updates, dependency bumps, test coverage, accessibility fixes. It\u0026rsquo;s just as explicit about what doesn\u0026rsquo;t: complex cross-repo refactors, anything security-sensitive, anything with requirements that aren\u0026rsquo;t already nailed down. That boundary lands almost exactly where I already split unsupervised background work from the interactive sessions I sit and drive myself. GitHub\u0026rsquo;s own framing puts it plainly: local agents for interactive work that needs immediate feedback, cloud agents for tasks that can run all the way to a finished PR with nobody watching, and a /delegate command meant to hand a task from one mode to the other without losing context. That\u0026rsquo;s the model I was already running before this tab existed. What\u0026rsquo;s new is a GitHub-native trigger for the cloud half, reachable from the repo UI or a phone instead of only from my own machine.\nBenchmark rankings mean the tab is routing, not competing # Third-party benchmarks put Claude Opus ahead on SWE-bench Verified at 88.6 percent, Codex ahead on Terminal-Bench 2.0 at 77.3 percent, Cursor around 74 percent on SWE-bench, and Copilot\u0026rsquo;s own agent trailing around 54 percent. Picking Claude or Codex from inside the Agents tab, instead of defaulting to Copilot\u0026rsquo;s built-in agent, means picking the same models I already reach for locally. GitHub sits underneath that choice as a router and a billing layer, not a rival source of intelligence. If Copilot\u0026rsquo;s own agent were the only option in that tab, I\u0026rsquo;d have skipped this whole investigation. Because Claude sits there as a first-class pick, the real question the tab poses is whether I want GitHub\u0026rsquo;s UI and GitHub\u0026rsquo;s bill wrapped around Claude, or my own.\nWhether it earns a permanent spot comes down to one habit I haven\u0026rsquo;t built yet # The place I can actually see this earning a spot is triage: assigning a low-risk issue to the Agents tab from my phone the second I file it, instead of sitting on it until I\u0026rsquo;m back at a keyboard to spin up a local session. That\u0026rsquo;s a real gap in how I work today, since small fixes tend to wait for keyboard time regardless of how trivial they are. What I don\u0026rsquo;t know is whether I\u0026rsquo;ll build that habit once the first week of novelty wears off, or whether I\u0026rsquo;ll keep defaulting to my own Claude Code session because I already trust its logs, its worktree lifecycle, and my own change-control gate more than a run I can only inspect through GitHub\u0026rsquo;s diff view. I\u0026rsquo;m giving it a real trial on one low-stakes repo before I decide either way, and I\u0026rsquo;d rather report back after a month of actual use than guess now.\n","date":"10 August 2026","externalUrl":null,"permalink":"/blog/github-agents-tab-vs-claude-code/","section":"Blog Posts","summary":"GitHub’s per-repo Agents tab is a mission-control surface, live since January 26, 2026, where GitHub-hosted coding agents pick up issues and turn them into PRs without anyone opening a terminal. Copilot’s own agent lives there by default, and Claude and Codex are selectable alongside it as “picked” third-party agents. The tab is part of Agent HQ, the umbrella GitHub announced on October 28, 2025, meant to give every agent vendor one shared surface inside Issues, PRs, and Actions instead of a pile of separate integrations. My read after digging into how it actually works: this is a real product, not a rebrand of anything Anthropic ships, and it solves a narrower slice of my workflow than local Claude Code already covers. Whether I keep reaching for it once the novelty wears off is the part I genuinely don’t know yet.\n","title":"GitHub's Agents Tab Puts Claude and Codex in the Repo UI. It's a Separate Bill From Claude Code.","type":"blog"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/cloud-computing/","section":"Categories","summary":"","title":"Cloud Computing","type":"categories"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/containerization/","section":"Categories","summary":"","title":"Containerization","type":"categories"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/tags/containerization/","section":"Tags","summary":"","title":"Containerization","type":"tags"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/cybersecurity/","section":"Categories","summary":"","title":"Cybersecurity","type":"categories"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/tags/devops/","section":"Tags","summary":"","title":"DevOps","type":"tags"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/tags/docker-compose/","section":"Tags","summary":"","title":"Docker Compose","type":"tags"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/docker-tutorials/","section":"Categories","summary":"","title":"Docker Tutorials","type":"categories"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/it-infrastructure/","section":"Categories","summary":"","title":"IT Infrastructure","type":"categories"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/tags/openvpn/","section":"Tags","summary":"","title":"OpenVPN","type":"tags"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/software-development/","section":"Categories","summary":"","title":"Software Development","type":"categories"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/tags/step-by-step-guide/","section":"Tags","summary":"","title":"Step-by-Step Guide","type":"tags"},{"content":" Introduction # Docker containers share the host\u0026rsquo;s network stack by default. Any service you run is only as private as the connection it\u0026rsquo;s running on, unless you put something in front of it. Routing a container\u0026rsquo;s traffic through a VPN container fixes that: requests leave through the VPN, not your raw connection, and the container\u0026rsquo;s real IP stays hidden.\nThis guide builds a Docker Compose file that puts one or more services behind a VPN container using network_mode: service:vpn. It covers setting up the VPN container, wiring dependent services to route through it, and verifying the traffic actually goes through the VPN once everything\u0026rsquo;s running.\nBasic Docker familiarity helps but isn\u0026rsquo;t required — the official Docker documentation covers anything unfamiliar here.\nOverview of Docker Compose and VPNs # A diagram of docker compose with a vpn What is Docker Compose? # Docker Compose simplifies the deployment of multi-container Docker applications by allowing developers to define services, networks, and volumes in a single YAML file. This approach streamlines the management of containerized applications, enabling easy configuration and launching of complex applications with just a few commands.\nBenefits of Docker Compose # Simplifies multi-container deployments Ensures consistency across development, testing, and production environments Streamlines application scaling and maintenance Typical Use Cases # Microservices architecture Development environments Continuous integration and continuous deployment (CI/CD) pipelines Why Use a VPN with Docker Services? # A VPN encrypts a container\u0026rsquo;s outbound traffic and hides its real IP behind the VPN provider\u0026rsquo;s. That matters most for services that talk to external networks or handle data you don\u0026rsquo;t want tied back to your home connection.\nCommon Scenarios and Benefits: # Securing communications between distributed services Protecting data in transit from eavesdropping Ensuring privacy for services that need to access external resources Using a VPN allows for more secure communication across your Docker services. Understanding the Challenge # A container with no VPN in front of it sends traffic exactly the way the host would: same IP, same exposure to anything watching the host\u0026rsquo;s connection. Routing a service through a VPN container fixes this at the network layer instead of trusting each service to handle it individually.\nIssues with Networking and Container Isolation # Potential exposure of sensitive data Difficulty in managing network policies Ensuring consistent VPN connections for all services By understanding these challenges and implementing a VPN within your Docker Compose setup, you can create a more secure and reliable environment for your applications.\nSetting Up Docker Compose # Installing Docker and Docker Compose # Steps to Install Docker: # Update Your Package Database: # Ensure your system\u0026rsquo;s package database is up-to-date\nsudo apt update Install Prerequisite Packages # Install packages that allow apt to use repositories over HTTPS\nsudo apt install apt-transport-https ca-certificates curl software-properties-common Add Docker\u0026rsquo;s Official GPG Key: # Add Docker\u0026rsquo;s GPG key to verify the integrity of the software.\ncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - Add Docker Repository: # Add Docker\u0026rsquo;s official repository to your sources list.\nsudo add-apt-repository \u0026#34;deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable\u0026#34; Install Docker: # Update the package database again and install Docker.\nsudo apt update sudo apt install docker-ce Verify Docker Installation: # Confirm Docker is installed correctly by running:\nsudo docker --version Steps to Install Docker Compose # Download the Latest Version: # Download the Docker Compose from its official Github repository.\nsudo curl -L \u0026#34;https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)\u0026#34; -o /usr/local/bin/docker-compose Apply Executable Permissions: # Make the downloaded file executable.\nsudo chmod +x /usr/local/bin/docker-compose Verify Docker Compose Installation: # Check the version to ensure Docker Compose is installed.\ndocker-compose --version Creating a Docker Compose File # Basic Structure of a docker-compose.yml File: # A docker-compose.yml file defines the services, network, and volumes used in your application. Here is the basic structure:\nversion: \u0026#39;3.8\u0026#39; services: # Define your services here networks: # Define custom networks if needed volumes: # Define named volumes if needed Explanation of Key Directives: # version: Specifies the version of the Docker Compose file format.\nservices: Defines the containers to be run as the part of the application.\nimage: Specifies the Docker image to use. build: Allows specifying a build context and Dockerfile. ports: Maps container ports to host ports. volumes: Mounts host paths or named volumes. networks: Connects services to specific networks. networks: Customized networking configurations for services.\nvolumes: Manages data persistence using named volumes.\nExample: Basic Docker Compose File # Here\u0026rsquo;s a simple example with two services: a web server and a database.\nversion: \u0026#39;3.8\u0026#39; services: web: image: nginx:latest ports: - \u0026#34;80:80\u0026#34; networks: - webnet database: image: postgres:latest environment: POSTGRES_USER: exampleuser POSTGRES_PASSWORD: examplepass POSTGRES_DB: exampledb volumes: - db-data:/var/lib/postgresql/data networks: - webnet networks: webnet: volumes: db-data: By understanding the basic structure and following these steps, you can create a Docker Compose file that efficiently sets up and manages your services.\nConfiguring Each Service to Use the VPN # Choosing a VPN Provider # When selecting a VPN provider for your Docker setup, it\u0026rsquo;s essential to consider several key factors to ensure optimal performance and security:\nKey Factors to Consider: # Reliablity: Choose a provider with a reputation for uptime and reliability. Security Features: Ensure the provider offers strong encryption and no-log policies. Compatibility: Verify that the VPN service is compatible with Docker and can be used within containers. Performance: Consider the speed and latency, especially if your servicers require high bandwidth. Support: Look for providers that offer good customer support and detailed documentation. Example: Using OpenVPN or Another Common VPN Service: # OpenVPN is a popular choice due to its flexibility, strong security, and open-source nature. Another option is WireGuard, known for its simplicity and performance. Both can be used effectively with Docker.\nOpenVPN is a popular choice. Setting Up the VPN Container # Pulling a VPN Container Image (e.g., OpenVPN): # To set up a VPN container, you will first need to pull the appropriate image from Docker Hub. Here\u0026rsquo;s how you can do it using OpenVPN:\ndocker pull kylemanna/openvpn This command downloads the OpenVPN image, which you can then use to create and configure your VPN container.\nConfiguring the VPN Container: # Initialize the OpenVPN Configuration: Create a directory to store the OpenVPN configuration and initialize it: mkdir -p /path/to/your/config docker run -v /path/to/your/config:/etc/openvpn kylemanna/openvpn ovpn_genconfig -u udp://YOUR_VPN_SERVER This command sets up the necessary configuration for OpenVPN in the specified directory.\nGenerate the Certificates: Generate the necessary certificates and keys: docker run -v /path/to/your/config:/etc/openvpn -it kylemanna/openvpn ovpn_initpki This command initializes the Public Key Infrastructure (PKI), generating the certificates and keys required for OpenVPN.\nStart the OpenVPN Container: Start the container with the generated configuration: docker run -v /path/to/your/config:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN kylemanna/openvpn This command runs the OpenVPN container in detached mode, mapping the required port and granting the necessary network administration capabilities.\nModifying the Docker Compose File # Adding the VPN Container to the docker-compose.yml File: # To integrate the VPN container into your Docker Compose setup, modify your docker-compose.yml file to include the VPN container and configure your services to use it.\nConfiguring Services to Route Traffic Through the VPN: # Ensure your services are configured to route their traffic through the VPN container by setting the network mode of the service to the VPN container.\nExample: Updated Docker Compose File with VPN: # Here\u0026rsquo;s a step-by-step example:\nversion: \u0026#39;3.8\u0026#39; services: vpn: image: kylemanna/openvpn cap_add: - NET_ADMIN ports: - \u0026#34;1194:1194/udp\u0026#34; volumes: - /path/to/your/config:/etc/openvpn environment: - OPENVPN_PROVIDER=YourProvider - OPENVPN_CONFIG=YourConfig networks: - vpn_net web: image: nginx:latest depends_on: - vpn network_mode: service:vpn ports: - \u0026#34;80:80\u0026#34; volumes: - ./web:/usr/share/nginx/html environment: - VIRTUAL_HOST=yourdomain.com database: image: postgres:latest depends_on: - vpn network_mode: service:vpn environment: POSTGRES_USER: exampleuser POSTGRES_PASSWORD: examplepass POSTGRES_DB: exampledb volumes: - db-data:/var/lib/postgresql/data networks: vpn_net: volumes: db-data: In this example:\nThe vpn service sets up the VPN using the OpenVPN image. The web and database services are configured to use the VPN container\u0026rsquo;s network by setting network_mode to service:vpn. This configuration ensures that all traffic from the web and database services is routed through the VPN, providing an added layer of security. By following these steps and examples, you can successfully configure your Docker services to operate securely behind a VPN, enhancing both privacy and security for your applications.\nTesting and Troubleshooting # Testing the Setup # Verifying the VPN Connection: # To ensure that the VPN connection is functioning correctly, you can perform a few checks:\nCheck the VPN Container Logs: Inspect the logs of the VPN container to confirm it has started correctly and is connected.\ndocker logs \u0026lt;vpn-container-name\u0026gt; Test the VPN Connection: Use curl or wget from within a containe using the VPN to check the external IP address. The external IP address should be different than your local IP, and it should match the VPN server\u0026rsquo;s IP.\ndocker exec -it \u0026lt;container-name\u0026gt; curl ifconfig.me Ensuring Services are Behind the VPN: # To verify that the Dockers services you created route all their web traffic through the VPN, you can access the services and check their outgoing IP addresses.\nCheck Service IP: From within the service container, use the following command:\ndocker exec -it \u0026lt;service-container-name\u0026gt; curl ifconfig.me The output should match the VPN IP, indicating that the service routes traffic through the VPN.\nCommon Issues and Solutions # Network Connectivity Issues: # Issue: Services cannot connect to the internet. Solution: Doublecheck the VPN container configuration. Make sure that the network mode is correctly set in the docker.compose.yml file. VPN Container Fails to Start: # Issue: The VPN container doesn\u0026rsquo;t start / keeps restarting. Solution: Check the logs for any errors, and check that the configuration files and credentials you provided are correct. Make sure that the required ports are not bloced by a firewall. Services Not Routing Through the VPN: # Issue: Services bypass the VPN and use the host network. Solution: Verify the network_mode: service:vpn setting in the docker-compose.yml file. Verify that the dependent services start after the VPN container. Tips for Troubleshooting # Useful Commands and Logs to Check: # View Container Logs: Check the logs for both the VPN container, as well as the created services for any error messages.\ndocker logs \u0026lt;container-name\u0026gt; Inspect Network Settings: Verify that the network settings of your containers are properly configured.\ndocker network inspect \u0026lt;network-name\u0026gt; Check IP Routes: Verify the IP routing tables within the containers to ensure that traffic is being routed through the VPN.\ndocker exec -it \u0026lt;container-name\u0026gt; ip route Community and Support Resources: # Docker Documentation: The official Docker documentation is the defacto resource for troubleshooting and best practices when using Docker.\nOpenVPN Documentation: The OpenVPN documentation will help you in determining specific configurations and in general troubleshooting.\nCommunity Forums: Search your issue on community forums such as Stack Overflow, Docker Community Forums, and Reddit.\nConclusion # The network_mode: service:vpn pattern is the actual mechanism here — it\u0026rsquo;s what forces a dependent service to share the VPN container\u0026rsquo;s network namespace instead of the host\u0026rsquo;s. Everything else in this guide (provider choice, the OpenVPN setup, the verification commands) exists to get you to a Compose file where that one line does its job correctly. If curl ifconfig.me from inside a dependent container returns the VPN\u0026rsquo;s IP instead of your own, it\u0026rsquo;s working.\n","date":"1 July 2024","externalUrl":null,"permalink":"/blog/secure-services-docker-compose-and-nordvpn/","section":"Blog Posts","summary":"Introduction # Docker containers share the host’s network stack by default. Any service you run is only as private as the connection it’s running on, unless you put something in front of it. Routing a container’s traffic through a VPN container fixes that: requests leave through the VPN, not your raw connection, and the container’s real IP stays hidden.\n","title":"Step-by-Step Guide to Creating a Secure Docker Compose Script with VPN Integration","type":"blog"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/tech-how-tos/","section":"Categories","summary":"","title":"Tech How-Tos","type":"categories"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/tags/tutorial/","section":"Tags","summary":"","title":"Tutorial","type":"tags"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/tags/vpn/","section":"Tags","summary":"","title":"VPN","type":"tags"},{"content":"","date":"1 July 2024","externalUrl":null,"permalink":"/categories/vpn-and-security/","section":"Categories","summary":"","title":"VPN and Security","type":"categories"},{"content":"","externalUrl":null,"permalink":"/contact/","section":"Contact Me","summary":"","title":"Contact Me","type":"contact"},{"content":"","externalUrl":null,"permalink":"/pages/","section":"Pages","summary":"","title":"Pages","type":"pages"},{"content":" Responsibility of Contributors # Lorem ipsum dolor sit amet, consectetur adipiscing elit. Purus, donec nunc eros, ullamcorper id feugiat quisque aliquam sagittis. Sem turpis sed viverra massa gravida pharetra. Non dui dolor potenti eu dignissim fusce. Ultrices amet, in curabitur a arcu a lectus morbi id. Iaculis erat sagittis in tortor cursus. Molestie urna eu tortor, erat scelerisque eget. Nunc hendrerit sed interdum lacus. Lorem quis viverra sed\npretium, aliquam sit. Praesent elementum magna amet, tincidunt eros, nibh in leo. Malesuada purus, lacus, at aliquam suspendisse tempus. Quis tempus amet, velit nascetur sollicitudin. At sollicitudin eget amet in. Eu velit nascetur sollicitudin erhdfvssfvrgss eget viverra nec elementum. Lacus, facilisis tristique lectus in.\nGathering of Personal Information # Lorem ipsum dolor sit amet, consectetur adipiscing elit. Purus, donec nunc eros, ullamcorper id feugiat quisque aliquam sagittis. Sem turpis sed viverra massa gravida pharetra. Non dui dolor potenti eu dignissim fusce. Ultrices amet, in curabitur a arcu a lectus morbi id. Iaculis erat sagittis in tortor cursus. Molestie urna eu tortor, erat scelerisque eget. Nunc hendrerit sed interdum lacus. Lorem quis viverra sed\nProtection of Personal- Information # Lorem ipsum dolor sit amet, consectetur adipiscing elit. Purus, donec nunc eros, ullamcorper id feugiat quisque aliquam sagittis. Sem turpis sed viverra massa gravida pharetra. Non dui dolor potenti eu dignissim fusce. Ultrices amet, in curabitur a arcu a lectus morbi id. Iaculis erat sagittis in tortor cursus.\nMolestie urna eu tortor, erat scelerisque eget. Nunc hendrerit sed interdum lacus. Lorem quis viverra sed Lorem ipsum dolor sit amet, consectetur adipiscing elit. Purus, donec nunc eros, ullamcorper id feugiat\nPrivacy Policy Changes # Sll the Themefisher items are designed to be with the latest , We check all comments that threaten or harm the reputation of any person or organization personal information including, but limited to, email addresses, telephone numbers Any Update come in The technology Customer will get automatic Notification. ","externalUrl":null,"permalink":"/privacy-policy/","section":"Pages","summary":"Responsibility of Contributors # Lorem ipsum dolor sit amet, consectetur adipiscing elit. Purus, donec nunc eros, ullamcorper id feugiat quisque aliquam sagittis. Sem turpis sed viverra massa gravida pharetra. Non dui dolor potenti eu dignissim fusce. Ultrices amet, in curabitur a arcu a lectus morbi id. Iaculis erat sagittis in tortor cursus. Molestie urna eu tortor, erat scelerisque eget. Nunc hendrerit sed interdum lacus. Lorem quis viverra sed\n","title":"Privacy","type":"pages"}]