# Code Reviewer

Review GitHub pull requests from a native GitHub App channel. Mention `@code-reviewer` on a pull request to publish a GitHub review with inline comments, optional suggestion blocks, and Upstash-backed rate limiting for public repositories.

- Install: `npx shadcn@latest add @evex/code-reviewer`
- Category: coding
- Author: [TommyBez](https://www.evex.sh/authors/TommyBez)
- Updated: 2026-07-04
- Dependencies: @upstash/ratelimit@^2.0.8, @upstash/redis@^1.38.0, ai@^7.0.38, eve@^0.31.3, zod@4.3.6
- Web page: https://www.evex.sh/agents/code-reviewer
- This document: https://www.evex.sh/agents/code-reviewer.md

## Overview

Code Reviewer is an eve agent that reviews GitHub pull requests when someone mentions @code-reviewer in a PR comment or a review thread. It runs behind a native GitHub App channel: GitHub delivers the comment webhook to your deployed eve app at /eve/v1/github, and the agent replies by publishing a real GitHub review with inline comments anchored to the diff.

The agent reviews changed behavior, not style. Its instructions direct it at concrete bugs, regressions, security risks, rollout risk, and materially missing tests, and explicitly forbid naming nits and speculative rewrites. It labels each finding blocking, warning, or nit, caps each review at 10 inline comments, and attaches GitHub suggestion blocks for small local fixes the author applies manually.

It is read-only toward your repository: the channel checks the code out into the eve sandbox for inspection, but the agent never pushes commits, opens branches, or edits the pull request. Built-in Upstash-backed rate limiting keeps public deployments safe, with stricter defaults for public repositories than private ones.

## How it works

1. A user comments @code-reviewer (the mention must match GITHUB_APP_SLUG) on a pull request timeline or in a Files changed review thread, and GitHub sends the webhook to the eve app.
2. The channel verifies the mention targets a pull request conversation, then checks Upstash rate limits: per-PR cooldown, per-user-per-PR cooldown, and daily repository quotas; if blocked, it posts at most one cooldown reply per 15 minutes instead of running the model.
3. When allowed, the channel injects PR metadata and diff context (excluding lockfiles and build output like dist, .next, and coverage) and checks the repository out into the eve sandbox.
4. The agent, running on the zai/glm-5.2 model, inspects only the context needed to validate findings using read_file, grep, glob, and targeted bash, loading the bundled review-calibration skill when severity is ambiguous.
5. It calls the submit_pr_review tool exactly once with a summary and up to 10 severity-labeled inline comments, each validated by a Zod schema (path, line, side, optional suggestion block).
6. The channel claims the publication in Redis to prevent duplicates, then posts a batch GitHub review; if batch creation fails, it falls back to individual inline comments plus a timeline summary, and reviews with no findings become a short timeline comment.

## Use cases

### On-demand review for team pull requests

Install the GitHub App on your team repositories and mention @code-reviewer when a PR is ready. You get a structured review with blocking, warning, and nit findings anchored to the exact diff lines, focused on bugs and regressions rather than style.

### Safe reviews on public open-source repos

The Upstash-backed limiter defaults to 10 reviews per public repository per day, one review per PR every 15 minutes, and a per-user cooldown of 30 minutes, so drive-by mentions on a public repo cannot exhaust your model budget.

### Quick-fix suggestions authors can apply in one click

For short, local, near-certain fixes the agent attaches a GitHub suggestion block to the inline comment. The PR author applies it from the GitHub UI; the agent itself never commits or modifies the branch.

### Second pass on risky surfaces before merge

The instructions prioritize auth, permissions, user data, schemas, cache invalidation, concurrency, and billing code paths. Mention the agent on high-risk PRs to get a focused pass on those surfaces plus residual-risk notes in the review summary.

## Requirements

- `GITHUB_APP_ID`: The App ID of the GitHub App you create under GitHub Settings, Developer settings, GitHub Apps. The app needs Metadata read, Contents read, Pull requests read/write, and Issues read/write permissions.
- `GITHUB_APP_PRIVATE_KEY`: The PEM private key generated from the GitHub App settings page. When stored as a single-line variable, replace literal newlines with \n; eve normalizes that form at runtime.
- `GITHUB_WEBHOOK_SECRET`: A long random value set both in the GitHub App webhook configuration and in your deployment. A mismatch produces HTTP 401 responses at /eve/v1/github.
- `GITHUB_APP_SLUG`: The mention users type on GitHub, defaulting to code-reviewer. It must match your GitHub App name so @code-reviewer mentions trigger the agent.
- `KV_REST_API_URL`: Upstash Redis REST endpoint used by @upstash/ratelimit for cooldowns, daily quotas, and duplicate-publication claims. Provision it via Upstash or the Vercel Redis Marketplace integration.
- `KV_REST_API_TOKEN`: The matching Upstash REST token. Use a read-write token, not the read-only one, because the agent writes cooldown and review publication keys.
- `AI_GATEWAY_API_KEY`: A model credential for the deployment, either a Vercel AI Gateway API key or AI Gateway OIDC, so the agent can call the zai/glm-5.2 model.

## FAQ

### How do I install and trigger it?

Run npx shadcn@latest add @evex/code-reviewer inside an eve app, deploy it over HTTPS, create a GitHub App pointing its webhook at /eve/v1/github, subscribe to Issue comments and Pull request review comments, install it on your repositories, then comment @code-reviewer review this on any pull request.

### Can it modify my pull request?

No. The agent only publishes review comments and optional suggestion blocks. It may test small patches inside the eve sandbox to validate a suggestion, but it never pushes commits, opens branches, or edits the PR; authors apply suggestions manually.

### Which model does it use and can I change it?

The agent is pinned to zai/glm-5.2 in agent/agent.ts. Since the file is installed into your app, you can edit defineAgent to point at any model available through your AI Gateway credential.

### How do the rate limits work and can I tune them?

Defaults are one review per PR every 15 minutes, one per user per PR every 30 minutes, 25 daily reviews per private repository, and 10 per public repository. Every limit is tunable via CODE_REVIEWER_* environment variables, and CODE_REVIEWER_RATE_LIMIT_ENABLED=false disables limiting for local development.

### What happens if Upstash is unreachable?

The failure mode defaults to public_closed: reviews on public repositories are blocked while private repositories continue working. You can change CODE_REVIEWER_RATE_LIMIT_FAILURE_MODE to closed or open depending on how conservative you want the deployment to be.

## Files installed

- `agent/agent.ts`
- `agent/channels/github.ts`
- `agent/instructions.md`
- `agent/lib/review-rate-limit.ts`
- `agent/skills/review-calibration/references/review-checklist.md`
- `agent/skills/review-calibration/references/severity-scale.md`
- `agent/skills/review-calibration/SKILL.md`
- `agent/tools/submit_pr_review.ts`
- `evals/evals.config.ts`
- `evals/pr-review-input-contract.eval.ts`
- `evals/pr-review-no-findings.eval.ts`
- `evals/pr-review-with-inline-finding.eval.ts`
- `agent/README.md`
- `.env.example`

## File contents

### `agent/agent.ts`

```ts
import { defineAgent } from "eve";

export default defineAgent({
  model: "zai/glm-5.2",
});

```

### `agent/channels/github.ts`

````ts
import {
  defaultGitHubAuth,
  type GitHubJsonObject,
  githubChannel,
  type GitHubChannelState,
  type GitHubEventContext,
  type GitHubInboundContext,
  type GitHubThread,
} from "eve/channels/github";
import { toolResultFrom } from "eve/tools";
import {
  checkCodeReviewRateLimit,
  claimReviewPublication,
  type RateLimitDecision,
  shouldPostCooldownReply,
} from "../lib/review-rate-limit";
import submitPrReviewTool, {
  type SubmitPrReviewComment,
  type SubmitPrReviewOutput,
} from "../tools/submit_pr_review";

const BOT_NAME = process.env.GITHUB_APP_SLUG || "code-reviewer";
const BOT_MENTION_PATTERN = new RegExp(
  `@${escapeRegExp(BOT_NAME)}(?=$|[^A-Za-z0-9_-])`,
  "i",
);
const GITHUB_COMMENT_CHUNK_SIZE = 60_000;

type CodeReviewerGitHubState = GitHubChannelState & {
  codeReviewerReviewSubmitted?: boolean;
};

export default githubChannel({
  botName: BOT_NAME,
  pullRequestContext: {
    excludedFiles: [
      "**/pnpm-lock.yaml",
      "**/package-lock.json",
      "**/yarn.lock",
      "**/bun.lockb",
      "**/dist/**",
      "**/build/**",
      "**/.next/**",
      "**/coverage/**",
    ],
  },
  async onComment(ctx, comment) {
    if (!BOT_MENTION_PATTERN.test(comment.body)) {
      return null;
    }

    if (!isPullRequestConversation(ctx)) {
      return null;
    }

    const pullRequestNumber = ctx.conversation.pullRequestNumber;
    if (pullRequestNumber === null) {
      return null;
    }

    const decision = await checkCodeReviewRateLimit({
      installationId: ctx.github.installationId,
      isPrivateRepository: ctx.repository.private,
      pullRequestNumber,
      repositoryId: ctx.repository.id,
      senderId: ctx.sender.id,
      senderLogin: ctx.sender.login,
    });

    if (decision.allowed) {
      return { auth: defaultGitHubAuth(ctx) };
    }

    await maybePostCooldownReply(ctx, decision);
    return null;
  },
  events: {
    async "action.result"(data, channel) {
      const match = toolResultFrom(data.result, submitPrReviewTool);
      if (!match) {
        return;
      }

      const state = channel.state as CodeReviewerGitHubState;
      if (state.codeReviewerReviewSubmitted) {
        return;
      }

      const claimed = await claimReviewPublication({
        headSha: state.headSha,
        installationId: channel.github.installationId,
        pullRequestNumber: state.pullRequestNumber ?? 0,
        repositoryId: channel.repository.id,
        toolCallId: match.callId,
      });

      if (!claimed) {
        state.codeReviewerReviewSubmitted = true;
        return;
      }

      try {
        await publishReview(channel, match.output);
        state.codeReviewerReviewSubmitted = true;
      } catch {
        state.codeReviewerReviewSubmitted = false;
      }
    },
    async "message.completed"(data, channel) {
      if (data.finishReason === "tool-calls" || !data.message) {
        return;
      }

      const state = channel.state as CodeReviewerGitHubState;
      if (state.codeReviewerReviewSubmitted) {
        return;
      }

      await postCommentChunks(channel.thread, data.message);
    },
  },
});

function isPullRequestConversation(ctx: GitHubInboundContext) {
  return (
    ctx.conversation.kind === "pull_request" ||
    ctx.conversation.kind === "review_thread"
  );
}

async function maybePostCooldownReply(
  ctx: GitHubInboundContext,
  decision: Extract<RateLimitDecision, { allowed: false }>,
) {
  const pullRequestNumber = ctx.conversation.pullRequestNumber;
  if (pullRequestNumber === null) {
    return;
  }

  const canReply = await shouldPostCooldownReply({
    installationId: ctx.github.installationId,
    pullRequestNumber,
    repositoryId: ctx.repository.id,
  });

  if (!canReply) {
    return;
  }

  try {
    await ctx.thread.post(formatCooldownReply(decision));
  } catch {
    // If the cooldown notice cannot be posted, still suppress the model run.
  }
}

function formatCooldownReply(decision: Extract<RateLimitDecision, { allowed: false }>) {
  if (decision.reason === "rate_limit_unavailable") {
    return `\`${BOT_NAME}\` cannot run because rate limiting is unavailable for this repository.`;
  }

  const retryAfter = formatRetryAfter(decision.retryAfterSeconds);
  return `\`${BOT_NAME}\` is cooling down for this pull request. Try again ${retryAfter}.`;
}

function formatRetryAfter(retryAfterSeconds: number | undefined) {
  if (!retryAfterSeconds) {
    return "later";
  }

  if (retryAfterSeconds <= 90) {
    return "in about 1 minute";
  }

  return `in about ${Math.ceil(retryAfterSeconds / 60)} minutes`;
}

async function publishReview(
  channel: GitHubEventContext,
  review: SubmitPrReviewOutput,
) {
  const pullRequestNumber = channel.state.pullRequestNumber;
  if (pullRequestNumber === null || review.comments.length === 0) {
    await postCommentChunks(channel.thread, review.summary);
    return;
  }

  try {
    await createBatchReview(channel, review, pullRequestNumber);
    return;
  } catch {
    await publishFallbackReview(channel, review, pullRequestNumber);
  }
}

async function createBatchReview(
  channel: GitHubEventContext,
  review: SubmitPrReviewOutput,
  pullRequestNumber: number,
) {
  const body: GitHubJsonObject = {
    body: review.summary,
    comments: review.comments.map(toGitHubReviewComment),
    event: "COMMENT",
    ...(channel.state.headSha ? { commit_id: channel.state.headSha } : {}),
  };

  await channel.github.request({
    body,
    method: "POST",
    path: githubPullRequestReviewPath(channel.state, pullRequestNumber),
  });
}

async function publishFallbackReview(
  channel: GitHubEventContext,
  review: SubmitPrReviewOutput,
  pullRequestNumber: number,
) {
  const failedComments: SubmitPrReviewComment[] = [];
  let publishedCount = 0;

  if (channel.state.headSha) {
    for (const comment of review.comments) {
      try {
        await channel.github.request({
          body: {
            ...toGitHubReviewComment(comment),
            commit_id: channel.state.headSha,
          },
          method: "POST",
          path: githubPullRequestCommentPath(channel.state, pullRequestNumber),
        });
        publishedCount += 1;
      } catch {
        failedComments.push(comment);
      }
    }
  } else {
    failedComments.push(...review.comments);
  }

  await postCommentChunks(
    channel.thread,
    formatFallbackSummary(review.summary, failedComments, publishedCount),
  );
}

function toGitHubReviewComment(comment: SubmitPrReviewComment): GitHubJsonObject {
  return {
    body: formatInlineCommentBody(comment),
    line: comment.line,
    path: comment.path,
    side: comment.side,
    ...(comment.startLine
      ? {
          start_line: comment.startLine,
          start_side: comment.startSide ?? comment.side,
        }
      : {}),
  };
}

function formatInlineCommentBody(comment: SubmitPrReviewComment) {
  const parts = [`**${comment.severity}:** ${comment.body}`];

  if (comment.suggestion) {
    parts.push("", "```suggestion", comment.suggestion, "```");
  }

  return parts.join("\n");
}

function formatFallbackSummary(
  summary: string,
  failedComments: readonly SubmitPrReviewComment[],
  publishedCount: number,
) {
  if (failedComments.length === 0) {
    return publishedCount > 0
      ? `${summary}\n\nPosted ${publishedCount} inline comments.`
      : summary;
  }

  const failedList = failedComments.map(
    (comment) =>
      `- ${comment.path}:${comment.line} **${comment.severity}:** ${comment.body}`,
  );

  const prefix =
    publishedCount > 0
      ? `Posted ${publishedCount} inline comments. These findings could not be anchored inline:`
      : "These findings could not be anchored inline:";

  return [summary, "", prefix, ...failedList].join("\n");
}

async function postCommentChunks(thread: GitHubThread, message: string) {
  for (const chunk of splitCommentBody(message)) {
    await thread.post(chunk);
  }
}

function splitCommentBody(message: string) {
  if (message.length <= GITHUB_COMMENT_CHUNK_SIZE) {
    return [message];
  }

  const chunks: string[] = [];
  for (
    let startIndex = 0;
    startIndex < message.length;
    startIndex += GITHUB_COMMENT_CHUNK_SIZE
  ) {
    chunks.push(message.slice(startIndex, startIndex + GITHUB_COMMENT_CHUNK_SIZE));
  }

  return chunks;
}

function escapeRegExp(value: string) {
  return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}

function githubPullRequestReviewPath(
  state: GitHubChannelState,
  pullRequestNumber: number,
) {
  return `/repos/${encodeURIComponent(state.owner)}/${encodeURIComponent(
    state.repo,
  )}/pulls/${pullRequestNumber}/reviews`;
}

function githubPullRequestCommentPath(
  state: GitHubChannelState,
  pullRequestNumber: number,
) {
  return `/repos/${encodeURIComponent(state.owner)}/${encodeURIComponent(
    state.repo,
  )}/pulls/${pullRequestNumber}/comments`;
}

````

### `agent/instructions.md`

```md
# Mission
You review GitHub pull request diffs to find concrete bugs, regressions,
security risks, rollout risk, and materially missing tests.

# Default stance
Prefer changed behavior over style commentary. Do not spend time on naming nits,
formatting, or speculative rewrites unless they hide a concrete failure mode.
You are a reviewer, not a patch author: you may suggest fixes, but you do not
claim to have changed the pull request.

# Workflow
1. Start from the GitHub pull request context injected by the channel.
2. Identify risky surfaces in the diff: auth, permissions, user data, external
   side effects, schemas, cache invalidation, concurrency, billing, and runtime
   behavior.
3. Use read_file, grep, and glob to inspect only the context needed to validate
   or reject a finding.
4. Use bash for targeted tests or read-only verification when it materially
   increases confidence.
5. Use write_file only inside the sandbox when trying a small local patch helps
   validate a suggestion. Never state that the pull request itself was changed.
6. Load the review-calibration skill when severity is ambiguous or multiple
   issues compete.
7. Put only concrete, actionable, diff-anchored findings in inline comments.
8. Put non-anchorable concerns, residual risk, and no-finding summaries in the
   review summary.
9. Call submit_pr_review exactly once when the review is ready.
10. After submit_pr_review, do not produce a second substantive final answer.

# Inline comment rules
Each inline comment must include:
- user or system impact
- what triggers the issue
- the smallest evidence needed to justify it

Use at most 10 inline comments. Prefer fewer, higher-confidence comments over a
long review.

# Severity
- blocking: probable bug, security issue, data loss, broken runtime behavior, or
  user-visible regression.
- warning: real risk that depends on runtime context, rollout conditions, or a
  materially missing test.
- nit: small correctness improvement. Do not use this for style.

# Suggestions
Use a suggestion only when the fix is short, local, and almost certainly
correct. Do not suggest broad refactors, migrations, API redesigns, or large
test-suite changes. A suggestion must contain only the replacement code for the
GitHub line or range.

# No findings
If you find no actionable issues, still call submit_pr_review with comments: []
and a summary that says no actionable findings were found, plus any residual
risk that deserves manual verification before merge.

```

### `agent/lib/review-rate-limit.ts`

```ts
import { createHash } from "node:crypto";
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";

const DEFAULT_RATE_LIMIT_PREFIX = "evex:code-reviewer";
const DEFAULT_PR_COOLDOWN_SECONDS = 900;
const DEFAULT_USER_PR_COOLDOWN_SECONDS = 1800;
const DEFAULT_PRIVATE_REPO_DAILY_LIMIT = 25;
const DEFAULT_PUBLIC_REPO_DAILY_LIMIT = 10;
const DEFAULT_COOLDOWN_REPLY_SECONDS = 900;
const REVIEW_PUBLICATION_TTL_SECONDS = 86_400;
const TRUE_VALUES = new Set(["1", "true", "yes", "on"]);
const FALSE_VALUES = new Set(["0", "false", "no", "off"]);

type FailureMode = "public_closed" | "closed" | "open";
type RateLimitReason =
  | "user_pr_cooldown"
  | "pr_cooldown"
  | "repo_daily_limit"
  | "rate_limit_unavailable";

type RateLimitConfig = {
  cooldownReply: boolean;
  cooldownReplySeconds: number;
  enabled: boolean;
  failureMode: FailureMode;
  prefix: string;
  privateRepoDailyLimit: number;
  prCooldownSeconds: number;
  publicRepoDailyLimit: number;
  userPrCooldownSeconds: number;
};

type RateLimiter = ReturnType<typeof createRateLimiter>;
type RateLimiterBundle = ReturnType<typeof createRateLimiterBundle>;

export type CodeReviewRateLimitInput = {
  installationId: number | null | undefined;
  isPrivateRepository: boolean;
  pullRequestNumber: number;
  repositoryId: number;
  senderId: number | null | undefined;
  senderLogin: string | null | undefined;
};

export type CooldownReplyRateLimitInput = {
  installationId: number | null | undefined;
  pullRequestNumber: number;
  repositoryId: number;
};

export type ReviewPublicationClaimInput = {
  headSha: string | null | undefined;
  installationId: number | null | undefined;
  pullRequestNumber: number;
  repositoryId: number;
  toolCallId: string;
};

export type RateLimitDecision =
  | { allowed: true }
  | {
      allowed: false;
      reason: RateLimitReason;
      resetAt?: number;
      retryAfterSeconds?: number;
    };

let cachedRedis: Redis | null = null;
let cachedLimiters: RateLimiterBundle | null = null;

export async function checkCodeReviewRateLimit(
  input: CodeReviewRateLimitInput,
): Promise<RateLimitDecision> {
  const config = readRateLimitConfig();

  if (!config.enabled) {
    return { allowed: true };
  }

  if (!hasUpstashEnvironment()) {
    return unavailableDecision(config, input.isPrivateRepository);
  }

  try {
    if (!input.isPrivateRepository && config.publicRepoDailyLimit <= 0) {
      return { allowed: false, reason: "repo_daily_limit" };
    }

    if (input.isPrivateRepository && config.privateRepoDailyLimit <= 0) {
      return { allowed: false, reason: "repo_daily_limit" };
    }

    const limiters = getRateLimiters(config);
    const checks = [
      await checkLimiter(
        limiters.userPr,
        identifierForUserPr(input),
        "user_pr_cooldown",
      ),
      await checkLimiter(limiters.pr, identifierForPr(input), "pr_cooldown"),
      await checkLimiter(
        input.isPrivateRepository ? limiters.privateRepoDaily : limiters.publicRepoDaily,
        identifierForRepoDaily(input),
        "repo_daily_limit",
      ),
    ];

    return checks.find((decision) => !decision.allowed) ?? { allowed: true };
  } catch {
    return unavailableDecision(config, input.isPrivateRepository);
  }
}

export async function shouldPostCooldownReply(
  input: CooldownReplyRateLimitInput,
): Promise<boolean> {
  const config = readRateLimitConfig();

  if (!(config.enabled && config.cooldownReply && hasUpstashEnvironment())) {
    return false;
  }

  try {
    const { cooldownReply } = getRateLimiters(config);
    const decision = await checkLimiter(
      cooldownReply,
      identifierForCooldownReply(input),
      "pr_cooldown",
    );

    return decision.allowed;
  } catch {
    return false;
  }
}

export async function claimReviewPublication(
  input: ReviewPublicationClaimInput,
): Promise<boolean> {
  if (!hasUpstashEnvironment()) {
    return true;
  }

  try {
    const config = readRateLimitConfig();
    const redis = getRedis();
    const key = `${config.prefix}:review-publish:${hashParts([
      "review-publish",
      input.installationId ?? "unknown-installation",
      input.repositoryId,
      input.pullRequestNumber,
      input.headSha ?? "unknown-head",
      input.toolCallId,
    ])}`;
    const result = await redis.set(key, "1", {
      ex: REVIEW_PUBLICATION_TTL_SECONDS,
      nx: true,
    });

    return result === "OK";
  } catch {
    return true;
  }
}

function checkLimiter(
  limiter: RateLimiter,
  identifier: string,
  reason: RateLimitReason,
): Promise<RateLimitDecision> {
  return limiter.limit(identifier).then((result) => {
    void result.pending.catch(() => undefined);

    if (result.success) {
      return { allowed: true };
    }

    return {
      allowed: false,
      reason,
      resetAt: result.reset,
      retryAfterSeconds: retryAfterSeconds(result.reset),
    };
  });
}

function createRateLimiter(redis: Redis, limit: number, windowSeconds: number, prefix: string) {
  return new Ratelimit({
    analytics: false,
    limiter: Ratelimit.slidingWindow(limit, `${windowSeconds} s`),
    prefix,
    redis,
  });
}

function createRateLimiterBundle(config: RateLimitConfig) {
  const redis = getRedis();

  return {
    configKey: JSON.stringify(config),
    cooldownReply: createRateLimiter(
      redis,
      1,
      config.cooldownReplySeconds,
      `${config.prefix}:cooldown-reply`,
    ),
    pr: createRateLimiter(redis, 1, config.prCooldownSeconds, `${config.prefix}:pr`),
    privateRepoDaily: createRateLimiter(
      redis,
      Math.max(1, config.privateRepoDailyLimit),
      86_400,
      `${config.prefix}:repo-private-day`,
    ),
    publicRepoDaily: createRateLimiter(
      redis,
      Math.max(1, config.publicRepoDailyLimit),
      86_400,
      `${config.prefix}:repo-public-day`,
    ),
    userPr: createRateLimiter(
      redis,
      1,
      config.userPrCooldownSeconds,
      `${config.prefix}:user-pr`,
    ),
  };
}

function getRateLimiters(config: RateLimitConfig) {
  const configKey = JSON.stringify(config);
  if (cachedLimiters?.configKey === configKey) {
    return cachedLimiters;
  }

  cachedLimiters = createRateLimiterBundle(config);
  return cachedLimiters;
}

function getRedis() {
  if (cachedRedis) {
    return cachedRedis;
  }

  cachedRedis = Redis.fromEnv();
  return cachedRedis;
}

function hasUpstashEnvironment() {
  return Boolean(process.env.KV_REST_API_URL && process.env.KV_REST_API_TOKEN);
}

function unavailableDecision(
  config: RateLimitConfig,
  isPrivateRepository: boolean,
): RateLimitDecision {
  if (config.failureMode === "open") {
    return { allowed: true };
  }

  if (config.failureMode === "closed") {
    return { allowed: false, reason: "rate_limit_unavailable" };
  }

  return isPrivateRepository
    ? { allowed: true }
    : { allowed: false, reason: "rate_limit_unavailable" };
}

function identifierForUserPr(input: CodeReviewRateLimitInput) {
  return hashParts([
    "user-pr",
    input.installationId ?? "unknown-installation",
    input.repositoryId,
    input.pullRequestNumber,
    input.senderId ?? input.senderLogin ?? "unknown-sender",
  ]);
}

function identifierForPr(input: CodeReviewRateLimitInput) {
  return hashParts([
    "pr",
    input.installationId ?? "unknown-installation",
    input.repositoryId,
    input.pullRequestNumber,
  ]);
}

function identifierForRepoDaily(input: CodeReviewRateLimitInput) {
  return hashParts([
    "repo-day",
    input.installationId ?? "unknown-installation",
    input.repositoryId,
    input.isPrivateRepository ? "private" : "public",
  ]);
}

function identifierForCooldownReply(input: CooldownReplyRateLimitInput) {
  return hashParts([
    "cooldown-reply",
    input.installationId ?? "unknown-installation",
    input.repositoryId,
    input.pullRequestNumber,
  ]);
}

function hashParts(parts: readonly (number | string)[]) {
  return createHash("sha256").update(parts.map(String).join(":")).digest("hex");
}

function retryAfterSeconds(resetAt: number) {
  return Math.max(1, Math.ceil((resetAt - Date.now()) / 1000));
}

function readRateLimitConfig(): RateLimitConfig {
  return {
    cooldownReply: readBoolean(process.env.CODE_REVIEWER_COOLDOWN_REPLY, true),
    cooldownReplySeconds: readPositiveInteger(
      process.env.CODE_REVIEWER_COOLDOWN_REPLY_SECONDS,
      DEFAULT_COOLDOWN_REPLY_SECONDS,
    ),
    enabled: readBoolean(process.env.CODE_REVIEWER_RATE_LIMIT_ENABLED, true),
    failureMode: readFailureMode(
      process.env.CODE_REVIEWER_RATE_LIMIT_FAILURE_MODE,
    ),
    prefix:
      process.env.CODE_REVIEWER_RATE_LIMIT_PREFIX?.trim() ||
      DEFAULT_RATE_LIMIT_PREFIX,
    privateRepoDailyLimit: readNonNegativeInteger(
      process.env.CODE_REVIEWER_PRIVATE_REPO_DAILY_LIMIT,
      DEFAULT_PRIVATE_REPO_DAILY_LIMIT,
    ),
    prCooldownSeconds: readPositiveInteger(
      process.env.CODE_REVIEWER_PR_COOLDOWN_SECONDS,
      DEFAULT_PR_COOLDOWN_SECONDS,
    ),
    publicRepoDailyLimit: readNonNegativeInteger(
      process.env.CODE_REVIEWER_PUBLIC_REPO_DAILY_LIMIT,
      DEFAULT_PUBLIC_REPO_DAILY_LIMIT,
    ),
    userPrCooldownSeconds: readPositiveInteger(
      process.env.CODE_REVIEWER_USER_PR_COOLDOWN_SECONDS,
      DEFAULT_USER_PR_COOLDOWN_SECONDS,
    ),
  };
}

function readBoolean(value: string | undefined, fallback: boolean) {
  if (!value) {
    return fallback;
  }

  const normalizedValue = value.trim().toLowerCase();
  if (TRUE_VALUES.has(normalizedValue)) {
    return true;
  }

  if (FALSE_VALUES.has(normalizedValue)) {
    return false;
  }

  return fallback;
}

function readFailureMode(value: string | undefined): FailureMode {
  if (value === "closed" || value === "open" || value === "public_closed") {
    return value;
  }

  return "public_closed";
}

function readPositiveInteger(value: string | undefined, fallback: number) {
  const parsedValue = Number.parseInt(value ?? "", 10);
  return Number.isInteger(parsedValue) && parsedValue > 0
    ? parsedValue
    : fallback;
}

function readNonNegativeInteger(value: string | undefined, fallback: number) {
  const parsedValue = Number.parseInt(value ?? "", 10);
  return Number.isInteger(parsedValue) && parsedValue >= 0
    ? parsedValue
    : fallback;
}

```

### `agent/skills/review-calibration/references/review-checklist.md`

```md
# Review lenses
- Can a valid request now fail because of ordering, retries, or stale cache state?
- Does any new branch bypass authz, rate limiting, or ownership checks?
- Can the new code create partial writes or inconsistent state on failure?
- Does the patch need a migration, rollback, or a feature flag to be safe?
- What is the smallest test that would catch the highest-risk regression?
- Does a changed external side effect need idempotency, approval, or throttling?
- Does any user-controlled input reach file, network, shell, or markup surfaces?
- Can the finding be anchored to a changed diff line, or should it stay in the summary?

```

### `agent/skills/review-calibration/references/severity-scale.md`

```md
# Severity lenses
- blocking: data loss, privilege bypass, high-probability incident trigger, or user-facing regression without an easy workaround
- warning: correctness issue with limited blast radius, missing validation, fragile recovery, or materially missing tests
- nit: small correctness improvement that is safe and local; never style-only

```

### `agent/skills/review-calibration/SKILL.md`

```md
---
name: review-calibration
description: Calibrate review severity when impact is ambiguous and the review needs a consistent bar.
---

# Review calibration

**Calibrate** each finding against user harm, exploitability, reversibility, and
detection speed. Escalate issues that can corrupt state, leak data, bypass
authorization, or silently ship broken behavior. De-escalate issues that are
recoverable, obvious, and tightly scoped.

Use the review tool severity labels:

- **blocking** — probable bugs, security issues, data loss, broken runtime
  behavior, or user-visible regressions
- **warning** — real risks that depend on runtime context, rollout conditions,
  or materially missing tests
- **nit** — small correctness improvements only; never style

For severity lenses and review checklist detail, see
[severity-scale](./references/severity-scale.md) and
[review-checklist](./references/review-checklist.md).

```

### `agent/tools/submit_pr_review.ts`

```ts
import { defineTool } from "eve/tools";
import { z } from "zod";

const reviewSide = z.enum(["RIGHT", "LEFT"]);
const reviewSeverity = z.enum(["blocking", "warning", "nit"]);

const reviewComment = z.object({
  body: z.string().min(1).max(4000),
  line: z.number().int().positive(),
  path: z.string().min(1),
  severity: reviewSeverity,
  side: reviewSide.default("RIGHT"),
  startLine: z.number().int().positive().optional(),
  startSide: reviewSide.optional(),
  suggestion: z.string().min(1).max(8000).optional(),
});

const submitPrReviewInput = z.object({
  comments: z.array(reviewComment).max(10),
  summary: z.string().min(1).max(4000),
});

export type SubmitPrReviewComment = z.infer<typeof reviewComment>;
export type SubmitPrReviewOutput = z.infer<typeof submitPrReviewInput>;

export default defineTool({
  description:
    "Submit the final pull request review as a structured summary plus inline comments. Use this exactly once when the review is ready to publish.",
  inputSchema: submitPrReviewInput,
  execute(input) {
    return input;
  },
  toModelOutput(output) {
    return {
      type: "json",
      value: {
        hasSummary: output.summary.length > 0,
        inlineCommentCount: output.comments.length,
        readyToPublish: true,
      },
    };
  },
});

```

### `evals/evals.config.ts`

```ts
import { defineEvalConfig } from "eve/evals";

export default defineEvalConfig({
  timeoutMs: 120_000,
});

```

### `evals/pr-review-input-contract.eval.ts`

```ts
import { defineEval } from "eve/evals";
import { equals } from "eve/evals/expect";

const HUNK_NEW_START = 10;
const HUNK_NEW_END = 13;

type ReviewComment = {
  line?: number;
  path?: string;
  severity?: string;
  side?: string;
};

export default defineEval({
  description:
    "Submits exactly one review whose inline comments anchor to the changed hunk with correct path, in-range lines, and a blocking or warning severity for an authorization bypass.",
  async test(t) {
    const turn = await t.send(`
<github_context>
repository: example/widget
pull_request_number: 44
sender: maintainer
head_sha: fed789
</github_context>

Pull request diff:

diff --git a/src/auth.ts b/src/auth.ts
@@ -10,4 +10,4 @@ export function getUserForRequest(session: Session, requestedUserId: string) {
-  if (session.userId !== requestedUserId) {
-    throw new Error("forbidden");
-  }
-  return getUser(requestedUserId);
+  if (session.userId) {
+    return getUser(requestedUserId);
+  }
+  return null;

Review this diff and publish the PR review with submit_pr_review.
`);

    t.succeeded();
    t.calledTool("submit_pr_review", { count: 1 }).gate();

    const call = turn.requireToolCall("submit_pr_review");
    const comments = (call.input.comments ?? []) as readonly ReviewComment[];
    t.check(comments.length >= 1, equals(true).gate());

    let allAnchoredToHunk = true;
    let hasActionableSeverity = false;
    for (const comment of comments) {
      if (comment.path !== "src/auth.ts") {
        allAnchoredToHunk = false;
      }
      const line = comment.line;
      if (
        typeof line !== "number" ||
        !Number.isInteger(line) ||
        line < HUNK_NEW_START ||
        line > HUNK_NEW_END
      ) {
        allAnchoredToHunk = false;
      }
      if (comment.severity === "blocking" || comment.severity === "warning") {
        hasActionableSeverity = true;
      }
    }
    t.check(allAnchoredToHunk, equals(true).gate());
    t.check(hasActionableSeverity, equals(true).gate());
    t.check(
      comments.some((comment) => comment.severity === "blocking"),
      equals(true).soft(),
    );
    t.check(
      typeof call.input.summary === "string" && call.input.summary.length > 0,
      equals(true).gate(),
    );
  },
});

```

### `evals/pr-review-no-findings.eval.ts`

```ts
import { defineEval } from "eve/evals";
import { equals, includes } from "eve/evals/expect";

export default defineEval({
  description:
    "Submits a no-findings review with an empty comments list for an innocuous PR diff.",
  async test(t) {
    const turn = await t.send(`
<github_context>
repository: example/widget
pull_request_number: 43
sender: maintainer
head_sha: def456
</github_context>

Pull request diff:

diff --git a/src/copy.ts b/src/copy.ts
@@
-export const emptyState = "No items";
+export const emptyState = "No matching items";

Review this diff and publish the PR review with submit_pr_review.
`);

    t.succeeded();
    t.calledTool("submit_pr_review");
    const call = turn.requireToolCall("submit_pr_review");
    const comments = (call.input.comments ?? []) as readonly unknown[];
    t.check(comments.length === 0, equals(true).gate());
    t.check(t.reply, includes("No actionable").soft());
  },
});

```

### `evals/pr-review-with-inline-finding.eval.ts`

```ts
import { defineEval } from "eve/evals";
import { includes } from "eve/evals/expect";

export default defineEval({
  description: "Finds an actionable PR issue and submits an inline review.",
  async test(t) {
    await t.send(`
<github_context>
repository: example/widget
pull_request_number: 42
sender: maintainer
head_sha: abc123
</github_context>

Pull request diff:

diff --git a/src/auth.ts b/src/auth.ts
@@
- if (session.userId !== requestedUserId) {
-   throw new Error("forbidden");
- }
+ if (session.userId) {
+   return getUser(requestedUserId);
+ }

Review this diff and publish the PR review with submit_pr_review.
`);

    t.succeeded();
    t.calledTool("submit_pr_review");
    t.check(t.reply, includes("submit_pr_review").soft());
  },
});

```

### `agent/README.md`

````md
# Code Reviewer

Review GitHub pull requests from a native GitHub App channel. Mention
`@code-reviewer` on a pull request and it publishes a GitHub review with inline
comments for concrete findings. Small, local fixes may include GitHub suggestion
blocks that the PR author can apply manually.

The agent reviews changed behavior rather than style. It looks for bugs,
regressions, security issues, rollout risk, and materially missing tests.

## How it works

1. Install this agent into an existing Eve app.
2. Deploy the Eve app so GitHub can reach it over HTTPS.
3. Create and install a GitHub App for the repositories you want reviewed.
4. Point the GitHub App webhook to `/eve/v1/github`.
5. Subscribe the GitHub App to PR comment events.
6. Comment on a pull request:

```md
@code-reviewer review this
```

The GitHub channel injects PR metadata and diff context, checks out the
repository into the Eve sandbox, and lets the agent inspect relevant files. The
agent does not push commits, open branches, or modify the pull request. It only
publishes review comments and optional suggestion blocks.

## GitHub App setup

Create the GitHub App from **GitHub Settings -> Developer settings ->
GitHub Apps -> New GitHub App**.

Use these settings:

- **GitHub App name**: `code-reviewer`, or another name that matches
  `GITHUB_APP_SLUG`.
- **Homepage URL**: your deployed Eve app URL.
- **Callback URL**: leave blank.
- **Request user authorization (OAuth) during installation**: disabled.
- **Webhook**: active.
- **Webhook URL**: `https://<your-eve-deployment>/eve/v1/github`.
- **Webhook secret**: a long random value. Save the same value as
  `GITHUB_WEBHOOK_SECRET`.

The webhook URL must be publicly reachable by GitHub. Localhost URLs do not work
unless you expose them through a tunnel.

After creating the app:

1. Copy the **App ID** into `GITHUB_APP_ID`.
2. Generate a private key from the app settings.
3. Copy the private key PEM into `GITHUB_APP_PRIVATE_KEY`.
4. Install the app on the target repositories from **Install App**.

When storing the private key as a single-line environment variable, replace
literal newlines with `\n`. Eve normalizes that form at runtime.

## GitHub permissions

Use the narrowest permissions that support review comments:

- Metadata: read
- Contents: read
- Pull requests: read/write
- Issues: read/write

The issues permission is needed for PR timeline comments and fallback replies.

## GitHub events

Subscribe only to the events this agent consumes:

- Issue comments
- Pull request review comments

The first event lets users mention `@code-reviewer` from the PR conversation
timeline. The second lets users mention it from a code review thread in the
Files changed view.

You do not need the broader Issues or Pull requests events for the default
mention-driven workflow.

## Environment

The registry installs a `.env.example` template. Put real secret values in your
deployment environment.

Set the GitHub App credentials:

```bash
GITHUB_APP_ID=
GITHUB_APP_PRIVATE_KEY=
GITHUB_WEBHOOK_SECRET=
GITHUB_APP_SLUG=code-reviewer
```

`GITHUB_APP_SLUG` must match the mention users type in GitHub. With the default
value above, the trigger is:

```md
@code-reviewer review this
```

Set Vercel Redis/Upstash Marketplace REST credentials for rate limiting:

```bash
KV_REST_API_URL=
KV_REST_API_TOKEN=
```

Do not use the read-only token for this agent. Rate limiting writes cooldown and
review publication keys. `KV_URL` and `REDIS_URL` are Redis protocol URLs and
are not used by this REST client.

The default rate limits are intentionally stricter on public repositories:

- one review every 15 minutes per PR
- one review every 30 minutes per user per PR
- 25 reviews per private repository per day
- 10 reviews per public repository per day
- one cooldown reply every 15 minutes per PR

For local development only, you can disable rate limiting:

```bash
CODE_REVIEWER_RATE_LIMIT_ENABLED=false
```

Production public deployments should keep rate limiting enabled. If Upstash is
unavailable, the default failure mode blocks public repositories and allows
private repositories.

## Deployment checklist

Before testing from GitHub, confirm:

- The Eve app is deployed and has a POST webhook route at
  `https://<your-eve-deployment>/eve/v1/github`.
- The deployment has GitHub App credentials, Upstash Redis REST credentials,
  and a model credential such as Vercel AI Gateway OIDC or
  `AI_GATEWAY_API_KEY`.
- The GitHub App is installed on the repository that contains the pull request.
- The app has **Contents: read**, **Issues: read/write**, and **Pull requests:
  read/write** on that repository.
- The webhook is active and subscribed to **Issue comments** and **Pull request
  review comments**.

GitHub sends a `ping` webhook when you create or update the app. A successful
delivery should return HTTP 200. Then open a pull request and comment:

```md
@code-reviewer review this
```

Expected behavior:

- GitHub accepts the webhook delivery.
- The agent adds an `eyes` reaction unless progress reactions are disabled.
- The agent posts a GitHub pull request review with inline comments when it
  finds concrete issues.
- If no inline finding is warranted, it posts a short PR timeline summary.

Common setup failures:

- HTTP 401 from `/eve/v1/github`: `GITHUB_WEBHOOK_SECRET` does not match the
  GitHub App webhook secret.
- No response to a mention: the app is not subscribed to the comment event, the
  mention does not match `GITHUB_APP_SLUG`, or the app is not installed on that
  repository.
- GitHub API 403: the app installation is missing repository access or one of
  the required write permissions.
- Public repository reviews are blocked: Upstash is unavailable and the default
  failure mode is `public_closed`.

## Development

```bash
pnpm install
pnpm dev
```

Run `pnpm info` to inspect the Eve surface and `pnpm build` before opening a PR.
Use `pnpm eval -- --skip-report` for lightweight agent behavior checks.

````

### `.env.example`

```
GITHUB_APP_ID=
GITHUB_APP_PRIVATE_KEY=
GITHUB_WEBHOOK_SECRET=
GITHUB_APP_SLUG=code-reviewer

# Vercel Redis/Upstash Marketplace REST credentials.
KV_REST_API_URL=
KV_REST_API_TOKEN=

CODE_REVIEWER_RATE_LIMIT_ENABLED=true
CODE_REVIEWER_RATE_LIMIT_PREFIX=evex:code-reviewer
CODE_REVIEWER_PR_COOLDOWN_SECONDS=900
CODE_REVIEWER_USER_PR_COOLDOWN_SECONDS=1800
CODE_REVIEWER_PRIVATE_REPO_DAILY_LIMIT=25
CODE_REVIEWER_PUBLIC_REPO_DAILY_LIMIT=10
CODE_REVIEWER_COOLDOWN_REPLY=true
CODE_REVIEWER_COOLDOWN_REPLY_SECONDS=900
CODE_REVIEWER_RATE_LIMIT_FAILURE_MODE=public_closed

```