Config Drift Checker

Author here. This came out of a simple question I could not answer: after a Claude Code release, does my setup still do what I think it does?

My CLAUDE.md, a conventions skill and a guard hook are how code gets written on my projects now. Claude Code shipped 25 versions in the last 30 days, the model behind an alias changes server-side with no changelog, and a teammate can edit a skill in a PR nobody tests. The only signal I had was "it feels different".

So I built config-drift-checker. It writes eval cases from your existing setup (Anthropic's own claude plugin eval format), runs the real agent against them in a throwaway copy of your repo, and a GitHub Action re-runs them on every Claude Code release and every PR that touches the setup. Red check, PR comment, an HTML report with every grader's verdict and the model's response. Your machine, your CI, your key. No server.