Experimentation & A/B Testing
Experiments have a quiet failure mode. The test ships, and then nobody looks at it for three weeks. When someone finally does, the result is ambiguous, the person who designed it has moved on, and the honest answer is that it was underpowered from the start — it never could have detected the effect you cared about. So the decision gets made on vibes anyway, and the test was theatre. Two quarters later, someone proposes the same test again, because there is no record that it ever ran.
Skynet closes both leaks. It does the sizing maths up front so you know whether the test is worth running, and it holds every past experiment in memory so the program compounds instead of resetting.
How it works
Size it before you build it
Give the agent the hypothesis and the metric. It checks your baseline against your actual traffic and tells you the runtime you need — or tells you plainly that at your volume this test cannot resolve, which saves you three weeks.
Write the hypothesis down
The agent drafts the test doc: what you expect, why, the primary metric, the guardrails, and what result would make you ship or kill it. Deciding that in advance is what stops the post-hoc rationalising later.
Watch it run
It monitors the experiment against your analytics and flags real problems — a broken variant, a guardrail metric moving badly — without letting anyone peek at the primary result and call it early.
Report and remember
At the end you get a plain reading: what happened, whether it is conclusive, and the recommendation you committed to in advance. It goes into memory, so next quarter’s proposal knows this test already ran.
Build it from a prompt
One instruction covers the whole lifecycle, from sizing to the write-up.
Over time the record is the real asset. You stop repeating tests, stop shipping on underpowered results, and start with a body of evidence about what moves your product and what never has.