Ponytail, tested: half the code and the help text it left behind

Single comparison, one model, one run per task. The three runs below used the same starting repo and the same request; the only thing that changed was whether a skill was in play. Every number is a direction, not a distribution, and I mark below which conclusions are measured and which are my judgment.

What it does

DietrichGebert/ponytail (167,323 stars, MIT, site ponytail.dev, last push 2026-10-18; figures read 2026-10-19) installs six skills: ponytail, plus ponytail-review, ponytail-audit, ponytail-debt, ponytail-gain and ponytail-help. The five satellites query the ledger the main skill produces. Only the main skill changes how code gets written, and the install took 12.9 seconds.

The main SKILL.md is a decision ladder, not a slogan. Take the first option that fully works: does this need to exist at all, is it already in the codebase, does the standard library or the platform have it, is there an installed dependency, can it be one line, otherwise the minimum that works. It also lists what never gets cut: validation at trust boundaries, error handling that prevents data loss, security, accessibility. New non-trivial logic, meaning a branch, a loop, a parser, money or security, needs one small test. Shortcuts get a comment in the form shortcut: <the limit>, <when to upgrade>. Three levels: lite, full by default, and ultra, where ultra also argues with the request before building anything.

One more rule sets up a problem below: the reply must end with one or two lines saying what you skipped and what risk the user should know about.

The test

Same request and same starting repo as the Superpowers run: a 44-line cli.js with add <a> <b> and --help, a 22-line test file, two passing tests. The job was to add a --csv export printing 2,3,5, working in any argument position, leaving the default output untouched, with tests and no dependencies.

Method: wall clock from the first command to the last, model thinking included; one call per read, write, or command; line counts from git diff --numstat.

Do not read the call counts as a win. The baseline’s seven includes two reads forced by the tool policy before editing, and the Ponytail runs include the same two, so those cancel. The one remaining difference is a repo status check I ran before the baseline. The real finding is the other column: Ponytail did not save tool calls, it saved lines.

Two changes account for the entire difference. First, it left --help alone; the baseline added a line documenting --csv. Second, it collapsed the output branch from a four-line if into one ternary:

console.log(csv ? `${a},${b},${add(a, b)}` : add(a, b));

Tests shrank the same way: three separate test() blocks in the baseline, one block with two assertions here, without losing the any-position case.

One process difference never shows up in the diff. Ponytail does not require a failing test first, so I wrote the implementation and then the test, and it went green on the first run. Green proves the code matches my assertions; it does not prove the assertions can catch a mistake, which is exactly what the Superpowers run demonstrated with its red light.

The ultra setting produced a byte-identical diff to full: +12/-2 both times. Its only trace was a sentence before I started, noting that echo "2,3,$(node cli.js add 2 3)" would have solved the request with no code at all. That thought changed nothing in the repository.

Where it helps

1. It cut the right thing. The line it dropped was not a test or an error path; it was help text that nobody had asked for. That is the kind of change most likely to pass review as good hygiene while changing nothing for any user.
2. The test rule is executable rather than aspirational. It says new non-trivial logic gets one small test, not “write good tests”, which is why the result was one function with two assertions instead of three functions with three.
3. The closing “what I skipped” line turned an omission into a disclosure. I wrote that --help was not updated, so the reader learns the cost instead of discovering it later.

Problems

1. It does not switch itself on. After installing, nothing about my session changed until I opened the SKILL.md and followed it by hand. In practice it needs the trigger vocabulary, ponytail, be lazy, yagni, simplest solution, or an explicit /ponytail. Say “add a --csv export” to an agent with Ponytail installed and you get the same agent as before. The copy this machine already had, from September 2026, did show up in the session’s skill list as ponytail, but listed is not loaded: it fires only when the model chooses to read and follow it. That is also why my baseline is “unloaded” rather than “unaware”, since the one-line description was in front of me the whole time.

2. The levels make no difference at this size. Full and ultra produced the same file byte for byte. The documentation lists three intensities, which invites the assumption that ultra saves more; on this task it saved nothing extra and cost one extra judgment call. I can only prove that for one task, so I cannot tell you whether the levels matter on larger work.

3. Skipping the documentation is a real cost, and a one-line warning does not pay it. A CLI that documents itself through --help now has a flag users cannot find there. This is not hypothetical: the baseline build and the Ponytail build differ in exactly that, and a user reading help would conclude the feature does not exist. The skill’s fallback is a note to the operator, not to the user, and nobody in a team is assigned to convert that note into docs.

4. It contradicts Superpowers by design. Superpowers wants a spec, a plan and a test-first loop before code. Ponytail wants to question whether the feature should exist and to let the shortest diff win. Install both and the same sentence gets two opposite first moves. This one is my judgment from reading both SKILL.md files, not a measurement.

5. The same install wall applies. This is not Ponytail’s bug, but any skill installed through npx skills add on this machine hits it first: the command died in git clone with

   fatal: unable to access 'https://github.com/DietrichGebert/ponytail.git/': Failed to connect to github.com port 443 via 127.0.0.1 after 2059 ms: Could not connect to server

   because git’s global proxy pointed at 127.0.0.1:7890 with nothing listening. Overriding it with GIT_CONFIG_* variables to the port that was listening is what made the install possible.

When not to use it

1. Requirements that are still moving. Ponytail’s default is to build the smallest working version immediately. It will not help you discover what you actually want, and when the direction turns out wrong, the ten lines you saved become a rewrite.
2. Public surfaces: a library API or a CLI’s flags and help. YAGNI will remove extension points nobody uses today, and it will also remove the help text, exit codes and error messages that users do hit. This review is the example.
3. Projects with hard documentation rules. If your team requires synchronized help text, a doc entry per flag, or a test per new parameter, Ponytail’s default priority conflicts with that and you will re-explain the exception every time.

Verdict

Use it when the requirement is settled and the code is small; leave it out when the requirement is the open question. On this task it cut net new code from 20 lines to 10 with the same test coverage, and the price was a help screen that no longer mentions --csv. One model, one run per task: do not read 20-to-10 as a stable ratio.