I'm not sure what you're doing differently, but I just created a Claude skill called /triage-alert -- All I have to do is basically call /triage-alert without any arguments, or if I want to, paste a specific alert that happens to be firing.
The Skill itself basically is a 6-step process and a simple contract with some guardrails:
(1) identify the alert, (2) size the problem, (3) find the signature in metrics and logs, (4) test the leading hypothesis against a second source until it is confirmed or dead, (5) write the finding + plan, (6) AskUserQuestion for the decision, (7) implement via the sibling skill and verify, (8) update memory if the cause was new.
There'a 3 scripts to automate some of the tasks so that they are deterministic and 3 data sources as references as well as MCP servers it has access to for metric, logs and alerting sources.
That's it. It not only diagnoses the problem every time, correctly, but also figures out what went wrong, and why, and proposes how to fix the underlying causes.
I then have a follow up skill called /incident-postmortem that writes up a full incident report and post-mortem for the next time to learn from and feed back into the system -- Which is basically Claude's own memory over time.
I'm not sure what you're doing differently, but I just created a Claude skill called /triage-alert -- All I have to do is basically call /triage-alert without any arguments, or if I want to, paste a specific alert that happens to be firing.
The Skill itself basically is a 6-step process and a simple contract with some guardrails:
(1) identify the alert, (2) size the problem, (3) find the signature in metrics and logs, (4) test the leading hypothesis against a second source until it is confirmed or dead, (5) write the finding + plan, (6) AskUserQuestion for the decision, (7) implement via the sibling skill and verify, (8) update memory if the cause was new.
There'a 3 scripts to automate some of the tasks so that they are deterministic and 3 data sources as references as well as MCP servers it has access to for metric, logs and alerting sources.
That's it. It not only diagnoses the problem every time, correctly, but also figures out what went wrong, and why, and proposes how to fix the underlying causes.
I then have a follow up skill called /incident-postmortem that writes up a full incident report and post-mortem for the next time to learn from and feed back into the system -- Which is basically Claude's own memory over time.