From Coder to Coach: Working with AI Agents in Open Source

This post is the companion to my presentation “From Coder to Coach”, given to my IBM colleagues. The slides are built with Slidev and available here.

The talk covers what I learned using Claude Code as my primary development tool on Apache Camel for the past few weeks. This post adds the practical details that didn’t fit in a 15-minute presentation.

The short version

I raised dozens of PRs on Apache Camel, all written by AI agents. Some were great. Some were really wrong. The failures taught me three rules:

1. High effort mode for planning – without it, the agent’s reasoning is too shallow and it picks the obvious-but-wrong approach.
2. Make the agent gather context – git history, JIRA tickets, design docs. The agent doesn’t know what it doesn’t know.
3. Don’t trust – the tunnel problem – when an agent commits to a wrong approach, it doesn’t self-correct. It doubles down. Catch it early.

When I follow these rules, the results are genuinely impressive: full PR review + CI triage + JIRA creation in a single conversation, parallel workstreams with zero context-switching cost, and an iteration loop where I direct every step but never write code myself.

The toolkit

Claude Code

A CLI-based AI coding agent that reads code, runs commands, edits files, and interacts with GitHub and JIRA APIs. Key features I rely on:

• /effort max – deep reasoning mode. Critical for planning. See the workaround below if the setting doesn’t take effect.
• CLAUDE.md – a file at the root of your project (or in ~/.claude/) with persistent instructions the agent follows in every conversation. This is where I encode the context-gathering rules.
• Custom commands – Markdown files in ~/.claude/commands/ that define reusable multi-step workflows. I’ve open-sourced mine: github.com/gnodet/claude-code-commands.

Superset

A terminal workspace manager built on top of git worktrees (not to be confused with Apache Superset). Each workspace gets its own branch, working directory, and dedicated agent. This enables parallel workstreams – one JIRA issue per workspace, full isolation, no git stash chaos.

The three rules

Rule 1: High effort mode for planning

Without /effort max, the agent’s reasoning is too shallow for non-trivial tasks. It skips edge cases, misses architectural constraints, and picks the obvious-but-wrong approach. Plans look reasonable on the surface but fall apart when you read the generated code.

Always use /effort max before any planning step. The difference is night and day. In default mode, the agent rushes to a solution. In max effort, it considers alternatives, checks constraints, and catches its own mistakes.

I had multiple PRs that looked fine on the surface but were fundamentally wrong because the agent didn’t think deeply enough about the problem.

Workaround: effort max in settings.json doesn’t reliably work

The effortLevel: "max" setting in ~/.claude/settings.json doesn’t seem to take effect – sessions still start at medium/auto effort. The --effort max CLI flag does work, so the workaround is to inject it into the launch command.

For plain CLI users:

Just run claude --effort max instead of claude, or add a shell alias:

alias claude='command claude --effort max'

For Superset users:

1. Edit ~/.superset/bin/claude and change the last line from:

   exec "$REAL_BIN" "$@"
   

   to:

   exec "$REAL_BIN" --effort max "$@"
   

2. Make it read-only so Superset doesn’t overwrite it on restart:

   chmod a-w ~/.superset/bin/claude
   

3. Also update the DB preset so new workspace initializations pick it up:

   python3 -c "
   import json, sqlite3
   conn = sqlite3.connect('$HOME/.superset/local.db')
   cur = conn.cursor()
   cur.execute('SELECT terminal_presets FROM settings')
   data = json.loads(cur.fetchone()[0])
   for p in data:
       if p['name'] == 'claude':
           p['commands'] = ['claude --effort max --dangerously-skip-permissions']
   cur.execute('UPDATE settings SET terminal_presets = ?', [json.dumps(data)])
   conn.commit()
   conn.close()
   print('Done')
   "
   

Root cause: Known issue – fix in progress (see mastra#14831). The effortLevel in settings.json is documented and accepted but doesn’t propagate to the model’s reasoning effort parameter.

Rule 2: Make the agent gather context

The agent doesn’t know what it doesn’t know. Apache Camel is a 17-year-old project with thousands of intentional design decisions. Code that looks “wrong” often exists for a good reason. Without git blame and JIRA context, the agent will “fix” things that aren’t broken.

I had PRs that effectively reverted intentional prior commits because the agent didn’t check why the code was written that way.

Fix: Encode context-gathering rules in CLAUDE.md:

Before implementing, you MUST:
- Check git history and git blame on affected files
- Search for related JIRA tickets
- Read commit messages for prior changes
- Look for design documents in proposals/

I’ve since formalized this into reusable custom commands. The /work-on command, for example, encodes a full investigation phase before any code is written: fetch the JIRA issue, check git history, search for related tickets, validate assumptions, and present findings for approval before proceeding. See the full set at github.com/gnodet/claude-code-commands.

Rule 3: Don’t trust – the tunnel problem

When an agent commits to a wrong approach, it doesn’t self-correct. It doubles down – writing code that’s internally consistent but fundamentally wrong. Like a junior dev who’s too confident and won’t stop to question their assumptions.

This is the scariest failure mode. The agent picks an approach, and from that point on, everything it does reinforces that approach – even if it’s wrong. It won’t stop and say “wait, maybe I should reconsider.”

Fix:

• Review the plan before implementation.
• Question assumptions early.
• If something feels off, stop and re-evaluate – don’t let the agent dig deeper.
• It’s cheaper to start a fresh conversation than to try to redirect 500 lines of wrong code.

Custom commands

The commands I use daily are open-sourced at github.com/gnodet/claude-code-commands. To install them:

cp commands/*.md ~/.claude/commands/

Key design principles:

• Investigation before implementation – the agent gathers context before writing any code.
• Human-in-the-loop – the agent presents findings and waits for approval at critical steps.
• Self-review – /work-on runs /review-pr on its own changes before creating a PR.
• Composability – commands reference each other (e.g., /work-on calls /monitor-ci and /review-pr).

These commands are tailored for Apache Camel, but the patterns are generic. Swap the JIRA API for your issue tracker, mvn for your build tool, and adjust the conventions.

What’s next

Agent swarms and context pollution

A single agent doing everything fills its context window with implementation details and forgets the rules from the beginning of the conversation. The literature calls this context pollution. Claude Code already spawns sub-agents with isolated context, which helps, but the main agent still accumulates context over a long session, and sub-agents delegate – they don’t cross-check.

The full vision: independent agents reviewing each other’s work, with a coordinator that stays at a high level and doesn’t get bogged down in implementation details.

Automatic PR code review

Tools like CodeRabbit already do automatic PR review – free for open source, reads CLAUDE.md, configurable with project-specific rules. An alternative would be a custom GitHub bot delegating to a Claude instance with full project context. Both need investigation.

Self-spawning workspaces

Today I manually create a new Superset workspace when a side issue comes up. The goal: the agent encounters a side issue, creates a JIRA ticket, spins up a new workspace, and kicks off a parallel agent – with just a human confirmation to proceed.

Resources

• Slides: From Coder to Coach (Slidev) | PDF
• Custom commands: github.com/gnodet/claude-code-commands
• Claude Code: docs.anthropic.com
• Superset: superset.sh
• CodeRabbit: coderabbit.ai