CapstonePython (agent), TypeScript (GitHub App), YAML (Actions)

Capstone 16 — GitHub Issue-to-PR Autonomous Agent

AWS Remote SWE Agents, Cursor Background Agents, OpenAI Codex cloud, and Google Jules all ship the same 2026 product shape: label an issue, get a PR. An agent runs in a cloud sandbox, verifies the tests pass, and posts a review-ready PR with its rationale. The hard parts are rebuilding the repo's environment automatically, keeping credentials from leaking, enforcing per-repo budgets, and making sure the agent can never force-push. This capstone builds the self-hosted version and compares it with the hosted ones on cost and pass rate.

TIME: 30 hoursPREREQUISITES: Phase 11 (LLM engineering), Phase 13 (tools), Phase 14 (agents), Phase 15 (autonomous), Phase 17 (infrastructure)

Learning Objectives

Design an async issue-to-PR pipeline: GitHub webhook to Lambda to SQS to a Fargate worker running the agent loop in a Daytona or E2B sandbox.
Scope credentials and writes correctly: a short-lived GitHub App installation token, branch protection (with the app off the bypass list) for main and force-pushes, and a worker-side allow-list for `.github/workflows`.
Gate every PR on verification: full CI in the sandbox, a coverage-delta check, and per-repo daily dollar and PR-count budgets enforced at the dispatcher.
Evaluate the agent on 30 seeded issues for pass rate, PR quality, cost, and latency, and compare the results against hosted agents such as Cursor Background Agents.

Problem

An async cloud coding agent is a worker that takes a GitHub issue, does the work in a cloud sandbox with no human in the loop, and hands back a pull request. It is a separate product category from the interactive coding agents in capstone 01. The UX is a GitHub label: you tag an issue @agent fix this, a worker spins up in a sandbox, clones the repo, runs tests, edits files, verifies, and opens a PR with the agent's rationale in the body. There is no interactive loop and no terminal. AWS Remote SWE Agents, Cursor Background Agents, OpenAI Codex cloud, Google Jules, and Factory Droids have all converged on this shape.

The engineering challenges are concrete. The agent has to build the repo from scratch without a cached dev image. Flaky tests must be re-run or isolated. Credentials need a GitHub App with minimal fine-grained permissions, budgets have to be enforced per repo per day, and force-pushing is off the table. The capstone measures pass rate, cost, and safety against the hosted alternatives.

Warm-up0/3

1. Why run an autonomous coding agent inside a disposable sandbox instead of on a developer's own machine?Warm-up

2. Why put a queue between a webhook receiver and the slow worker that does the real job?Warm-up

3. Why should a background worker use a short-lived, narrowly scoped credential rather than a long-lived broad one?Warm-up

Check: Problem0/2

1. What is an async cloud coding agent?Define

2. Why does the worker need environment inference instead of a prepared image per repo?Explain

Concept

The trigger is a GitHub webhook, fired by an issue label or a PR comment. A dispatcher enqueues the work to ECS Fargate or Lambda, and the worker pulls the repo into a Daytona or E2B sandbox using a generic Dockerfile inferred from the repo (language, framework). Inside the sandbox, a mini-swe-agent or SWE-agent v2 loop runs against Claude Opus 4.7 or GPT-5.4-Codex. It iterates: read code, propose a fix, apply the patch, run tests.

Verification is the gating step. Full CI must pass in the sandbox before the PR opens. The worker computes the coverage delta, and if it is negative beyond a threshold, the PR still opens but gets labeled needs-review. The agent posts its rationale as the PR description, plus an @agent thread the reviewer can ping for follow-ups.

Safety is scoped through two different GitHub surfaces. The App provides a short-lived installation token with workflows: read and narrow repo contents and PR scopes. Branch protection, not app permissions, enforces "no direct writes to main" and "no force-push", and the app is never added to the bypass list. Path-scoped read-only access to .github/workflows is not a real GitHub App primitive, so the allow-list on file edits has to enforce it at the worker. Budget ceilings per repo per day are enforced at the dispatcher (for example, max 5 PRs and $50 per repo per day, on top of a $20 cap per task).

Check: Concept0/13

1. What is a GitHub App in this pipeline?Define

2. What is branch protection's role in the safety design?Define

3. What gating happens after the agent loop concludes, before a PR is opened?Define

4. Which budget limits does the dispatcher enforce per repo per day?Define

5. Which mechanism actually stops the agent from force-pushing to `main`?Explain

6. Why must the app stay off the branch protection bypass list?Explain

7. Why enforce per-repo budgets at the dispatcher instead of inside the worker?Explain

8. A repo has already used its 5 PRs for the day and a sixth issue gets the label. Where should it be stopped?Apply

9. A repo has a limit of 5 PRs per day and the dispatcher has already started 4 tasks today. Two new issues get the label. What happens?Apply

10. The agent's branch overwrote history on `main`. The token had contents write and the app was on the bypass list. What is the right fix?Analyze

11. One repo received 40 small, cheap PRs overnight and swamped its reviewers. Which control should have stopped it?Analyze

12. To block force-pushes to `main` while the agent still has to push its own branch, one design removes contents write from the token and another uses branch protection. Which is sound?Analyze

13. You onboard many repos with different usage. How do you keep one noisy repo from consuming spend or reviewer attention?Design

Architecture

GitHub issue labeled `@agent fix` or PR comment
            |
            v
    GitHub App webhook -> AWS Lambda dispatcher
            |
            v
    ECS Fargate task (or GitHub Actions self-hosted runner)
       - pull repo
       - infer Dockerfile (language, package manager)
       - Daytona / E2B sandbox with target runtime
       - clone -> git worktree -> agent branch
            |
            v
    mini-swe-agent / SWE-agent v2 loop
       Claude Opus 4.7 or GPT-5.4-Codex
       tools: ripgrep, tree-sitter, read/edit, run_tests, git
            |
            v
    verify CI passes in-sandbox + coverage delta check
            |
            v (verified)
    git push + open PR via GitHub App
       PR body = rationale + diff summary + trace URL
       label: needs-review (only if coverage dropped > 2%)
            |
            v
    operator reviews; can @-mention agent for follow-ups

Check: Architecture0/1

1. Which order does a task follow after the worker starts?Apply

Stack

Trigger: GitHub App with fine-grained token; webhook receiver via Lambda or Fly.io
Worker: ECS Fargate task (or GitHub Actions self-hosted runner)
Sandbox: Daytona devcontainer or E2B sandbox per task
Agent loop: mini-swe-agent baseline or SWE-agent v2 over Claude Opus 4.7 / GPT-5.4-Codex
Retrieval: tree-sitter repo-map + ripgrep
Verification: full CI in-sandbox + coverage delta gate
Observability: Langfuse with per-PR trace archive linked from the PR body
Budget: per-repo daily dollar ceiling; max PRs per repo per day

Build It

GitHub App. Issue a fine-grained installation token: issues read+write, pull_requests write, contents read+write, workflows read. Branch protection is the only surface that can enforce "no direct push to main" and "no force-push", and the app stays off the bypass list. GitHub App permissions are not path-scoped, so the worker enforces "no writes under .github/workflows" as an allow-list check on the proposed diff.
Webhook receiver. A Lambda function accepts issue label and PR comment webhooks, filters on the label @agent fix this, and enqueues to SQS.
Dispatcher. It pops tasks from SQS and enforces the per-repo per-day budget. Then it spins up an ECS Fargate task with the repo URL, the issue body, and a fresh Daytona sandbox.
Environment inference. Detect the language (Python, Node, Go, Rust) and package manager (uv, pnpm, go mod, cargo). Generate a Dockerfile on the fly if the repo does not have one.
Agent loop. Run mini-swe-agent or SWE-agent v2 with Claude Opus 4.7. Tools: ripgrep, tree-sitter repo-map, read_file, edit_file, run_tests, git. Hard limits: $20 cost, 30 min wall-clock, 30 agent turns.
Verification. When the loop concludes, run the full test suite in-sandbox and compute the coverage delta via jacoco or coverage.py. If CI is red, halt and do not open a PR. If coverage drops more than 2%, open the PR with the needs-review label.
PR posting. Push the agent branch and open the PR through the GitHub API with title, rationale, diff summary, trace URL, cost, and turns.
Credential hygiene. The worker runs on a short-lived GitHub App installation token, and logs are scrubbed for secrets before archival.
Eval. Run 30 seeded internal issues of varying difficulty. Measure pass rate, PR quality (diff size, style, coverage), cost, and latency. Run Cursor Background Agents and AWS Remote SWE Agents on the same issues for comparison.

Check: Build It0/22

1. What does the dispatcher do in this pipeline?Define

2. What does environment inference do when the target repo has no Dockerfile?Define

3. What does coverage delta measure?Define

4. What is the rationale in an agent's pull request?Define

5. Which sequence carries a labeled issue from GitHub to a running worker?Explain

6. Why is the `.github/workflows` write restriction enforced at the worker rather than as a GitHub App permission?Explain

7. Why write the workflow allow-list check before tuning anything in the agent loop?Explain

8. Why does a coverage drop add `needs-review` while a red CI run blocks the PR entirely?Explain

9. What do the hard limits of $20, 30 minutes and 30 turns protect against?Explain

10. Because no human watches the run, what should the PR body give the reviewer?Explain

11. Why run Cursor Background Agents and AWS Remote SWE Agents on the same 30 seeded issues?Explain

12. The label `needs-triage` is added to an issue, but `@agent fix this` is not. What happens?Apply

13. A repo has `Cargo.toml` and no Dockerfile. What does the worker do?Apply

14. A proposed diff edits `src/dedupe.go`, `dedupe_test.go` and `.github/workflows/ci.yml`. What happens?Apply

15. A run has used 30 agent turns, $6.50 and 14 minutes without finishing. What ends it?Apply

16. An agent run finishes with all tests green, but coverage fell by 3.5%. What should the worker do?Apply

17. PRs keep appearing with failing tests on GitHub. The logs show each PR opened right after the agent loop ended. What was skipped?Analyze

18. A team reports an 80% pass rate for its agent, but ran hosted agents on different issues and recorded no cost or latency. What is wrong?Analyze

19. A security review finds a live API key inside the archived agent logs. Which Build It control was missing?Analyze

20. Security now requires that the agent never modify `deploy/`. Where do you enforce it?Design

21. In the first eval, 8 of 30 runs hit the 30-turn limit. What should you do next?Design

22. Verification fails on a test that flakes and has nothing to do with the agent's change. Which design fits?Design

Use It

# on github.com
  - user labels issue #842 with `@agent fix this`
  - PR #1903 appears 14 minutes later
  - body:
    > Fixed NPE in widget.dedupe() caused by null comparator entry.
    > Added regression test widget_test.go::TestDedupeNullComparator.
    > Coverage delta: +0.12%
    > Turns: 7  Cost: $1.80  Trace: langfuse:...
    > Labels: none (coverage delta is above -2%)

Check: Use It0/1

1. In the Use It run, the agent took 7 turns, spent $1.80 and finished in 14 minutes. Did it hit a hard limit?Apply

Ship It

outputs/skill-issue-to-pr.md is the deliverable: a GitHub App plus an async cloud worker that turns labeled issues into review-ready PRs, with bounded cost and scoped credentials.

Weight	Criterion	How it is measured
25	Pass rate on 30 issues	End-to-end success (CI green + coverage OK)
20	PR quality	Diff size, coverage delta, style conformance
20	Cost and latency per resolved issue	$ and wall-clock per PR
20	Safety	Scoped token, per-repo budget, no force-push, credential hygiene
15	Operator UX	Rationale comments, retry affordance, @-mention follow-up
100

Check: Ship It0/2

1. Of 30 seeded issues, 24 end with green CI and 4 of those fail the coverage check. What is the pass rate?Apply

2. A submission earns full marks on Pass rate and Safety and nothing else. What is its total?Apply

Author's Note

I'd write the .github/workflows allow-list check before tuning anything in the agent loop. Branch protection covers main and force-pushes, but GitHub App permissions are not path-scoped, so the worker has to enforce that rule itself. If the check is missing, the worker is not enforcing it at all, whatever the token scopes say.

My bet is that you should keep the hard limits from step 5 ($20, 30 minutes, 30 turns) as they are for the first runs, and raise them only once the 30-issue eval shows where the agent needs more room. A ceiling you loosen with evidence is easier to defend than one you set high on optimism.

Exercises

Add a "fix flaky test" mode. The label @agent stabilize-flake TestX runs the test 50 times in-sandbox and proposes a minimal change that stabilizes it.
Compare cost against Cursor Background Agents on three shared issues. Report which tool wins where.
Implement a budget dashboard with per-repo per-day cost and per-user cost, and alert on anomalies.
Build a "dry-run" mode that opens a draft PR without running CI, so reviewers can examine the plan cheaply.
Add a retention policy: PR branches older than 7 days without a merge get deleted automatically.

Key Terms

Term	What people say	What it actually means
GitHub App	"Scoped bot identity"	App with fine-grained permissions + short-lived installation token
Async cloud agent	"Background agent"	Non-interactive worker that runs in a cloud sandbox, not a terminal
Environment inference	"Dockerfile synthesis"	Detect language + package manager, generate a Dockerfile if absent
Verification	"CI-in-sandbox"	Run the full test suite inside the worker before opening a PR
Coverage delta	"Coverage preservation"	Change in test coverage % from base to agent branch
Per-repo budget	"Daily ceiling"	Dollar and PR-count cap enforced at the dispatcher
Rationale	"PR body explanation"	Agent's summary of what changed and why; required in the PR body