Zoo 3.4 with code summaries

Presenting Zoo 3.4.

Code summaries and surprise detection

I still review production code, but I no longer care about style minutiae — instead, I’m looking for:

• surprise abstractions, complications and over-engineering
• decisions I disagree about
• naming and organizational choices I dislike
• inconsistent use of existing abstractions and helpers

Naming, abstractions and helpers I know about get caught at the low-level spec stage.

However, agents keep surprising me at the implementation stage, so I cannot forgo code reviews yet. In almost every task I execute, I do a post-implementation revision after code review.

…So, those code reviews are a real bottleneck. I’m starting to experiment with making them easier to do.

In Zoo 3.4, I am trying out code summaries: instead of reading the code, you read LLM-generated descriptions that intelligently recognize patterns, fold related changes and omit boilerplate, ex:

• GiveAccountCreationPoints runs its checks through the new corewelcomepts.EvaluateWelcomePoints.
  • Passes proc.Venue(), ForceLaunched: proc.group.forceLaunched, RewardsDisabled: proc.group.disableImmediateRewards. Check suppressions are honored.
  • On a non-empty eval.Denial, logs "GiveAccountCreationPoints: " + eval.Denial under debugLogWelcomePointDecisions and returns false. Step names match the removed inline debug lines.
  • The point change takes eval.Source.ID and eval.Pts. Welcome event unchanged.
  • IsWelcomePointsEnabled is now corewelcomepts.IsWelcomeSourceEnabledInVenue(proc.Venue()) && !disableImmediateRewards.

This isn’t an exact retelling of the source code; it lists 10 relevant lines out of a page-long diff.

I’ve also asked it to prefix SURPRISE to items that might seem controversial or unexpected given the spec, e.g.

• SURPRISE: the list includes language (LegacyLocale), which the apidocs change does not list.

I’m happy to report that surprise marking works very well! Most of the things I revise are indeed marked as surprises.

I’m gonna expand surprise criteria over time. The goal is for every post-implementation revision to be caused by something marked as a surprise. If/when I reach that, I will be able to only review the list of surprises.

(A term more accurate than surprise would be welcome too.)

Other tweaks

• Shorter specs. More pressure on agents to keep explanations brief, with details where they belong. Explicit sections for backwards-incompatible changes, functionality regressions and performance regressions, so those don’t get buried in a wall of text. I’m actually seeing agents doing more performance measurements now to fill it in!

• Zoo Revision skill. Some agents weren’t consistently running simple revisions through Zoo, and had to be reminded; the skill takes care of that, and replaces the old Zoo Add which didn’t prove itself useful.

• More monitoring support. Specs track stages, tickets and revision rounds; agents emit consistent chat events for task progress, reviews and Git operations. Monitoring tools can tell which chat owns the task and which ones are reviewing it.

• Uber-reviews do not implicitly include the initiating agent. Agents no longer interpret “uber-review with codex” as running Codex and themselves.

To install or upgrade

Install or upgrade Zoo skills from https://github.com/andreyvit/zoo, and run the setup or upgrade skills as necessary.