# Docs Knowledge Assistant

Answers from your repo docs and cites the file.

- Install: `npx shadcn@latest add @evex/docs-knowledge-assistant`
- Category: support
- Author: [TommyBez](https://www.evex.sh/authors/TommyBez)
- Updated: 2026-08-26
- Dependencies: eve@^0.31.3, zod@4.3.6
- Web page: https://www.evex.sh/agents/docs-knowledge-assistant
- This document: https://www.evex.sh/agents/docs-knowledge-assistant.md

## Overview

Docs Knowledge Assistant is an eve agent that answers questions from the installing repository's documentation files. It searches README, docs/, CONTRIBUTING*, and AGENTS.md, then replies with file-path citations so readers can verify the source.

You interact with it through Eve chat sessions or by mentioning it on a GitHub issue comment. It ignores pull request conversations, never publishes GitHub PR reviews, and never applies taxonomy labels or triage other issues.

When the documentation clearly lacks an answer, it says so. On GitHub issue-comment turns it can optionally open a documentation-gap issue with open_docs_issue so maintainers can track missing docs without turning the agent into a second issue bot.

## How it works

1. A user asks a documentation question in Eve chat, or mentions @docs-knowledge-assistant on a GitHub issue comment matching GITHUB_APP_SLUG.
2. The GitHub channel rejects pull request conversations and review threads, then injects issue context for the zai/glm-5.2 model.
3. The agent loads the docs-qa skill when needed, then calls search_docs and read_doc, which refuse non-documentation paths and discover README.* / CONTRIBUTING.* variants at the repo root.
4. It answers from those files only and cites each path it relied on. write_file is disabled so the agent cannot rewrite the repository as an answer.
5. If the docs do not cover the question, it says so. On GitHub issue-comment turns only, a clear gap may call open_docs_issue once; Eve chat receives readyToOpen false because no GitHub publisher runs there. The channel creates the issue through the GitHub Issues API without applying taxonomy labels.
6. Evals cover answering with a README citation, saying not-in-docs, and refusing PR review or labeling requests.

## Use cases

### Install and setup questions from README

New contributors ask how to install or configure the project in Eve chat. The agent reads README.md or docs/install.md and answers with the cited path instead of guessing from source.

### Contributor workflow questions on GitHub issues

Mention the bot on an issue asking about contribution steps. It answers from CONTRIBUTING* or AGENTS.md and posts the cited reply on the issue timeline.

### Honest gaps when docs are silent

When someone asks about on-call policy or an undocumented API, the agent says the docs do not cover it rather than inventing steps from application source.

### Track missing documentation

When a recurring question on a GitHub issue has no docs home, open_docs_issue creates a focused documentation issue so maintainers can fill the gap without labeling or triaging other work.

## Requirements

- `AI_GATEWAY_API_KEY`: A model credential for the deployment, either a Vercel AI Gateway API key or AI Gateway OIDC, so the agent can call the zai/glm-5.2 model.
- `GITHUB_APP_ID`: Optional. The App ID of the GitHub App used for issue-comment Q&A. Needs Metadata read, Contents read, and Issues read/write. Do not grant Pull requests write.
- `GITHUB_APP_PRIVATE_KEY`: Optional. The PEM private key for the GitHub App. When stored as a single-line variable, replace literal newlines with \n.
- `GITHUB_WEBHOOK_SECRET`: Optional. Shared secret for verifying GitHub webhooks at /eve/v1/github. A mismatch produces HTTP 401.
- `GITHUB_APP_SLUG`: Optional. The mention users type on GitHub, defaulting to docs-knowledge-assistant. It must match your GitHub App name.

## FAQ

### How do I install and ask a question?

Run npx shadcn@latest add @evex/docs-knowledge-assistant inside an eve app, set AI_GATEWAY_API_KEY, then ask documentation questions in Eve chat. For GitHub issue replies, deploy over HTTPS, create a GitHub App pointing at /eve/v1/github, and mention the bot on an issue.

### Does it review pull requests or label issues?

No. The channel ignores pull request conversations, there is no PR review publication path, and it never applies taxonomy labels. Use @evex/code-reviewer for PR reviews and @evex/github-issue-maintainer for issue triage.

### Which files can it read?

Only README*, docs/**, CONTRIBUTING*, and AGENTS.md. search_docs and read_doc refuse application source, tests, and config paths.

### What happens when the docs do not answer?

The agent says the documentation does not cover the question. On GitHub issue-comment turns, when the gap is clear, it can call open_docs_issue to create a documentation issue without labeling other issues. Eve chat refuses that tool.

### Is GitHub required?

No. Chat works with only AI_GATEWAY_API_KEY. GitHub App credentials are needed only for issue-comment Q&A and optional docs-gap issue creation.

## Files installed

- `.env.example`
- `agent/agent.ts`
- `agent/channels/github.ts`
- `agent/instructions.md`
- `agent/lib/docs-paths.ts`
- `agent/skills/docs-qa/SKILL.md`
- `agent/tools/open_docs_issue.ts`
- `agent/tools/read_doc.ts`
- `agent/tools/search_docs.ts`
- `agent/tools/write_file.ts`
- `evals/answer-cite-readme.eval.ts`
- `evals/evals.config.ts`
- `evals/ignore-pr-review.eval.ts`
- `evals/say-not-in-docs.eval.ts`
- `agent/README.md`

## File contents

### `.env.example`

```
# Model credential (Vercel AI Gateway API key or OIDC).
AI_GATEWAY_API_KEY=

# Optional GitHub App credentials for issue-comment docs Q&A.
GITHUB_APP_ID=
GITHUB_APP_PRIVATE_KEY=
GITHUB_WEBHOOK_SECRET=
GITHUB_APP_SLUG=docs-knowledge-assistant

```

### `agent/agent.ts`

```ts
import { defineAgent } from "eve";

export default defineAgent({
  model: "zai/glm-5.2",
});

```

### `agent/channels/github.ts`

```ts
import {
  defaultGitHubAuth,
  type GitHubEventContext,
  type GitHubInboundContext,
  type GitHubJsonObject,
  githubChannel,
  type GitHubChannelState,
} from "eve/channels/github";
import { toolResultFrom } from "eve/tools";

import openDocsIssueTool, {
  type OpenDocsIssueOutput,
} from "../tools/open_docs_issue";

const BOT_NAME = process.env.GITHUB_APP_SLUG || "docs-knowledge-assistant";
const BOT_MENTION_PATTERN = new RegExp(
  `@${escapeRegExp(BOT_NAME)}(?=$|[^A-Za-z0-9_-])`,
  "i",
);
const GITHUB_COMMENT_CHUNK_SIZE = 60_000;

type DocsAssistantGitHubState = GitHubChannelState & {
  docsIssueOpened?: boolean;
};

export default githubChannel({
  botName: BOT_NAME,
  async onComment(ctx, comment) {
    if (!BOT_MENTION_PATTERN.test(comment.body)) {
      return null;
    }

    // Docs Q&A only — never act on pull request conversations or reviews.
    if (!isIssueConversation(ctx)) {
      return null;
    }

    if (ctx.conversation.issueNumber === null) {
      return null;
    }

    return {
      auth: defaultGitHubAuth(ctx),
      context: [
        [
          "<github_docs_question_context>",
          `repository: ${ctx.repository.fullName}`,
          `issue_number: ${ctx.conversation.issueNumber}`,
          `sender: ${ctx.sender.login}`,
          "surface: github_issue_comment",
          "</github_docs_question_context>",
          "",
          "Answer this documentation question from README, docs/, CONTRIBUTING*, or AGENTS.md only. Cite file paths. If the docs lack the answer, say so. Optionally call open_docs_issue for a clear docs gap. Never review pull requests and never apply taxonomy labels.",
        ].join("\n"),
      ],
    };
  },
  events: {
    async "action.result"(data, channel) {
      const match = toolResultFrom(data.result, openDocsIssueTool);
      if (!match) {
        return;
      }

      const output = match.output as OpenDocsIssueOutput;
      if (!output.readyToOpen) {
        return;
      }

      const state = channel.state as DocsAssistantGitHubState;
      if (state.docsIssueOpened) {
        return;
      }

      if (state.conversationKind !== "issue") {
        return;
      }

      await publishDocsIssue(channel, output);
      state.docsIssueOpened = true;
    },
    async "message.completed"(data, channel) {
      if (data.finishReason === "tool-calls" || !data.message) {
        return;
      }

      const state = channel.state as DocsAssistantGitHubState;
      if (state.conversationKind !== "issue") {
        return;
      }

      await postCommentChunks(channel, data.message);
    },
  },
});

function isIssueConversation(ctx: GitHubInboundContext) {
  return ctx.conversation.kind === "issue";
}

async function publishDocsIssue(
  channel: GitHubEventContext,
  issue: Extract<OpenDocsIssueOutput, { readyToOpen: true }>,
) {
  const owner = channel.state.owner;
  const repo = channel.state.repo;
  const response = await channel.github.request({
    method: "POST",
    path: `/repos/${owner}/${repo}/issues`,
    body: {
      title: issue.title,
      body: [
        issue.body.trim(),
        "",
        `Opened by @${BOT_NAME} after a documentation Q&A gap.`,
        "",
        `Rationale: ${issue.rationale.trim()}`,
      ].join("\n"),
    },
  });

  const created = asObject(response.body);
  const number = created?.number;
  const htmlUrl = created?.html_url;
  if (typeof number === "number" && typeof htmlUrl === "string") {
    await postCommentChunks(
      channel,
      `Opened documentation issue #${number}: ${htmlUrl}`,
    );
  }
}

async function postCommentChunks(channel: GitHubEventContext, body: string) {
  for (const chunk of chunkText(body, GITHUB_COMMENT_CHUNK_SIZE)) {
    await channel.thread.post(chunk);
  }
}

function chunkText(value: string, size: number): string[] {
  if (value.length <= size) {
    return [value];
  }

  const chunks: string[] = [];
  for (let index = 0; index < value.length; index += size) {
    chunks.push(value.slice(index, index + size));
  }
  return chunks;
}

function asObject(value: unknown): GitHubJsonObject | null {
  if (value && typeof value === "object" && !Array.isArray(value)) {
    return value as GitHubJsonObject;
  }
  return null;
}

function escapeRegExp(value: string) {
  return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}

```

### `agent/instructions.md`

```md
# Mission
You answer questions from the installing repository's documentation files.
Cite the file path in every answer. You are a docs Q&A assistant, not a pull
request reviewer and not an issue labeler.

# Documentation scope
Stay inside documentation files only. Allowed sources:

- `README.md` / `README*`
- `docs/**`
- `CONTRIBUTING*`
- `AGENTS.md`

Do not answer from application source code, tests, configs, or lockfiles. If
the answer is only in non-docs source, say the docs do not cover it.

# Surfaces
You run on Eve chat sessions and on GitHub issue comments when mentioned. Ignore
pull request review requests. Never publish a GitHub PR review. Never apply
taxonomy labels or triage other issues.

# Workflow
1. Restate the question briefly if needed.
2. Use `search_docs` to find candidate documentation paths.
3. Use `read_doc` to read the relevant documentation files.
4. Answer from those files only. Cite each path you relied on (for example
   `README.md` or `docs/install.md`).
5. If the docs do not contain the answer, say so clearly. Do not invent setup
   steps, APIs, or policy from training data.
6. When the docs gap is clear on a GitHub issue-comment turn and the user would
   benefit from a tracking issue, call `open_docs_issue` once with a concrete
   title and body describing the missing documentation. Do not call
   `open_docs_issue` from Eve chat (it will refuse and must not report an
   opened issue). Never open an issue for ordinary answered questions, and
   never use issue creation to label or triage unrelated work.

# Hard boundaries
- Do not call `submit_pr_review` or any pull-request review publication path.
- Do not label, close, reopen, assign, or milestone issues.
- Do not review diffs or comment on pull requests.
- Do not use `write_file` to modify the repository as an answer.
- Prefer `search_docs` / `read_doc` over unconstrained shell exploration.

```

### `agent/lib/docs-paths.ts`

```ts
const README_PATTERN = /^README(?:\.[^./]+)?$/i;
const CONTRIBUTING_PATTERN = /^CONTRIBUTING(?:\.[^./]+)?$/i;
const AGENTS_PATTERN = /^AGENTS\.md$/i;
const DOCS_PREFIX_PATTERN = /^docs\//i;

/** Normalize a model-supplied path to a workspace-relative docs candidate. */
export function normalizeDocsPath(input: string): string {
  const trimmed = input.trim().replaceAll("\\", "/");
  const withoutWorkspace = trimmed
    .replace(/^\/workspace\//, "")
    .replace(/^\.\//, "");
  return withoutWorkspace.replace(/^\/+/, "");
}

/** True when the path is a documentation file this agent may read. */
export function isAllowedDocsPath(input: string): boolean {
  const relative = normalizeDocsPath(input);
  if (relative.length === 0 || relative.includes("..")) {
    return false;
  }

  const baseName = relative.includes("/")
    ? relative.slice(relative.lastIndexOf("/") + 1)
    : relative;

  if (README_PATTERN.test(baseName) && !relative.includes("/")) {
    return true;
  }

  if (CONTRIBUTING_PATTERN.test(baseName) && !relative.includes("/")) {
    return true;
  }

  if (AGENTS_PATTERN.test(baseName) && !relative.includes("/")) {
    return true;
  }

  return DOCS_PREFIX_PATTERN.test(relative);
}

/**
 * Static fallback roots when the workspace listing is empty (eval fixtures,
 * sparse checkouts). Prefer {@link docsSearchRootsFromListing} at runtime.
 */
export function docsSearchRoots(): readonly string[] {
  return [
    "README.md",
    "README",
    "CONTRIBUTING.md",
    "CONTRIBUTING",
    "AGENTS.md",
    "docs",
  ];
}

/**
 * Build search roots from a workspace root listing so README.* and
 * CONTRIBUTING.* variants (rst, txt, adoc, …) are included when present.
 * Stays inside documentation roots; never adds application source paths.
 */
export function docsSearchRootsFromListing(
  entries: readonly string[],
): string[] {
  const roots: string[] = [];
  const seen = new Set<string>();

  for (const entry of entries) {
    const name = entry.trim().replace(/\/+$/, "");
    if (name.length === 0 || name.includes("/") || name.includes("..")) {
      continue;
    }

    if (name === "docs") {
      if (!seen.has("docs")) {
        roots.push("docs");
        seen.add("docs");
      }
      continue;
    }

    if (!isAllowedDocsPath(name) || seen.has(name)) {
      continue;
    }

    roots.push(name);
    seen.add(name);
  }

  return roots;
}

```

### `agent/skills/docs-qa/SKILL.md`

```md
---
name: docs-qa
description: Answer repository documentation questions with file-path citations. Use when the user asks how something works, how to install or configure the project, or whether docs cover a topic.
---

# Docs Q&A

Answer only from documentation files in the installing repository:

- `README.md` / `README*`
- `docs/**`
- `CONTRIBUTING*`
- `AGENTS.md`

## Steps

1. Call `search_docs` with the user's keywords.
2. Call `read_doc` on the best matching paths.
3. Answer in plain language and cite every path you used.
4. If nothing in-scope answers the question, say the docs do not cover it.
5. Call `open_docs_issue` only on GitHub issue-comment turns when the gap is
   clear and worth tracking. Do not call it from Eve chat.

## Do not

- Review pull requests or publish GitHub PR reviews
- Apply taxonomy labels or triage unrelated issues
- Invent answers from application source code

```

### `agent/tools/open_docs_issue.ts`

```ts
import { defineTool } from "eve/tools";
import type { ToolContext } from "eve/tools";
import { z } from "zod";

const openDocsIssueInput = z.object({
  title: z
    .string()
    .min(8)
    .max(200)
    .describe("Short GitHub issue title describing the documentation gap."),
  body: z
    .string()
    .min(20)
    .max(4000)
    .describe(
      "Issue body explaining the unanswered question and which docs were checked.",
    ),
  rationale: z
    .string()
    .min(1)
    .max(500)
    .describe("One-sentence reason the docs gap is clear enough to track."),
});

type OpenDocsIssueInput = z.infer<typeof openDocsIssueInput>;

export type OpenDocsIssueSuccess = OpenDocsIssueInput & {
  readyToOpen: true;
};

export type OpenDocsIssueFailure = {
  readyToOpen: false;
  note: string;
  title?: string;
};

export type OpenDocsIssueOutput = OpenDocsIssueSuccess | OpenDocsIssueFailure;

function readAuthAttribute(
  attributes: Readonly<Record<string, string | readonly string[]>> | undefined,
  key: string,
): string | undefined {
  const value = attributes?.[key];
  return typeof value === "string" ? value : undefined;
}

/** True only on GitHub issue-comment turns (publisher lives in the GitHub channel). */
function isGitHubIssueSurface(ctx: ToolContext): boolean {
  const current = ctx.session.auth.current;
  if (!current || current.authenticator !== "github-webhook") {
    return false;
  }
  return readAuthAttribute(current.attributes, "conversation_kind") === "issue";
}

export default defineTool({
  description:
    "Open a documentation-gap GitHub issue when the docs clearly lack the answer. Only works on GitHub issue-comment turns (the GitHub channel publishes the issue). Do not call from Eve chat. Never use this to label, triage, or review other issues or pull requests.",
  inputSchema: openDocsIssueInput,
  execute(input, ctx): OpenDocsIssueOutput {
    if (!isGitHubIssueSurface(ctx)) {
      return {
        readyToOpen: false,
        note: "open_docs_issue only runs on GitHub issue-comment turns. In Eve chat, say the docs gap out loud; do not claim an issue was opened.",
        title: input.title,
      };
    }

    return {
      ...input,
      readyToOpen: true,
    };
  },
  toModelOutput(output) {
    if (!output.readyToOpen) {
      return {
        type: "json",
        value: {
          readyToOpen: false,
          note: output.note,
        },
      };
    }

    return {
      type: "json",
      value: {
        readyToOpen: true,
        title: output.title,
      },
    };
  },
});

```

### `agent/tools/read_doc.ts`

```ts
import { defineTool } from "eve/tools";
import { z } from "zod";

import { isAllowedDocsPath, normalizeDocsPath } from "../lib/docs-paths";

const readDocInput = z.object({
  path: z
    .string()
    .min(1)
    .max(400)
    .describe(
      "Documentation path to read (README.md, docs/**, CONTRIBUTING*, or AGENTS.md).",
    ),
  offset: z
    .number()
    .int()
    .min(1)
    .optional()
    .describe("Optional 1-based start line."),
  limit: z
    .number()
    .int()
    .min(1)
    .max(400)
    .optional()
    .describe("Optional max number of lines to return."),
});

export default defineTool({
  description:
    "Read a documentation file from the repository checkout. Refuses non-docs paths.",
  inputSchema: readDocInput,
  async execute(input, ctx) {
    const path = normalizeDocsPath(input.path);
    if (!isAllowedDocsPath(path)) {
      return {
        ok: false as const,
        path,
        note: "Path is outside documentation scope (README, docs/, CONTRIBUTING*, AGENTS.md).",
      };
    }

    const sandbox = await ctx.getSandbox();
    const start = input.offset ?? 1;
    const limit = input.limit ?? 200;
    const end = start + limit - 1;

    try {
      const content = await sandbox.readTextFile({
        path,
        startLine: start,
        endLine: end,
      });

      if (content === null) {
        return {
          ok: false as const,
          path,
          note: "Documentation file not found in the workspace checkout.",
        };
      }

      return {
        ok: true as const,
        path,
        content: content.slice(0, 24_000),
        offset: start,
        limit,
      };
    } catch {
      return {
        ok: false as const,
        path,
        note: "Documentation file not found in the workspace checkout.",
      };
    }
  },
});

```

### `agent/tools/search_docs.ts`

```ts
import { defineTool } from "eve/tools";
import { z } from "zod";

import {
  docsSearchRoots,
  docsSearchRootsFromListing,
  isAllowedDocsPath,
  normalizeDocsPath,
} from "../lib/docs-paths";

const searchDocsInput = z.object({
  query: z
    .string()
    .min(1)
    .max(200)
    .describe(
      "Literal or simple keyword query to search inside documentation files.",
    ),
  pathHint: z
    .string()
    .min(1)
    .max(200)
    .optional()
    .describe(
      "Optional docs path or directory to narrow the search (README.md, docs/, CONTRIBUTING*, AGENTS.md).",
    ),
});

type SearchHit = {
  line: number;
  path: string;
  text: string;
};

export default defineTool({
  description:
    "Search documentation files (README, docs/, CONTRIBUTING*, AGENTS.md) for a query. Prefer this before answering. Does not search application source.",
  inputSchema: searchDocsInput,
  async execute(input, ctx) {
    const sandbox = await ctx.getSandbox();
    let roots: string[];
    if (input.pathHint) {
      roots = [normalizeDocsPath(input.pathHint)];
    } else {
      const listing = await sandbox.run({
        command: "ls -1A 2>/dev/null | head -n 200",
      });
      const entries = (listing.stdout ?? "")
        .split("\n")
        .map((line) => line.trim())
        .filter(Boolean);
      roots = docsSearchRootsFromListing(entries);
      if (roots.length === 0) {
        roots = [...docsSearchRoots()];
      }
    }

    for (const root of roots) {
      if (root !== "docs" && !isAllowedDocsPath(root)) {
        return {
          hits: [] as SearchHit[],
          note: `Refused non-docs path: ${root}`,
          query: input.query,
        };
      }
    }

    const escaped = input.query.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
    const pathArgs = roots.map((root) => shellQuote(root)).join(" ");
    // Prefer rg when present. Do not pipe rg into head before checking
    // availability: without rg, `rg | head` still exits 0 via head and would
    // skip the grep fallback.
    const command = [
      "set +e",
      `if command -v rg >/dev/null 2>&1; then rg -n -S --no-heading -e ${shellQuote(escaped)} ${pathArgs} 2>/dev/null | head -n 40; else grep -RIn -E ${shellQuote(escaped)} ${pathArgs} 2>/dev/null | head -n 40; fi`,
      "exit 0",
    ].join("; ");

    const result = await sandbox.run({ command });
    const stdout = result.stdout ?? "";

    const hits: SearchHit[] = [];
    for (const line of stdout.split("\n")) {
      if (!line.trim()) {
        continue;
      }
      const match = /^([^:]+):(\d+):(.*)$/.exec(line);
      if (!match) {
        continue;
      }
      const path = normalizeDocsPath(match[1] ?? "");
      if (!isAllowedDocsPath(path)) {
        continue;
      }
      hits.push({
        path,
        line: Number(match[2]),
        text: (match[3] ?? "").slice(0, 240),
      });
    }

    return {
      hits,
      note:
        hits.length === 0
          ? "No documentation matches. Say so if the docs do not answer the question."
          : `Found ${hits.length} documentation hit(s).`,
      query: input.query,
    };
  },
});

function shellQuote(value: string): string {
  return `'${value.replaceAll("'", `'\\''`)}'`;
}

```

### `agent/tools/write_file.ts`

```ts
import { disableTool } from "eve/tools";

export default disableTool();

```

### `evals/answer-cite-readme.eval.ts`

```ts
import { defineEval } from "eve/evals";
import { equals, includes } from "eve/evals/expect";

export default defineEval({
  description:
    "Answers an install question from injected README content and cites README.md.",
  async test(t) {
    await t.send(`
<documentation_context>
path: README.md
content:
# Widget

Install the package with npm:

\`\`\`bash
npm install @example/widget
\`\`\`

Then import createWidget from @example/widget.
</documentation_context>

How do I install this project? Answer from the documentation and cite the file path.
`);

    t.succeeded();
    t.check(t.reply, includes("npm install").gate());
    t.check(t.reply, includes("README.md").gate());
    t.notCalledTool("open_docs_issue").soft();
    t.notCalledTool("submit_pr_review").gate();
    t.check(
      /README\.md/i.test(t.reply ?? ""),
      equals(true).gate(),
    );
  },
});

```

### `evals/evals.config.ts`

```ts
import { defineEvalConfig } from "eve/evals";

export default defineEvalConfig({
  timeoutMs: 120_000,
});

```

### `evals/ignore-pr-review.eval.ts`

```ts
import { defineEval } from "eve/evals";
import { equals, includes } from "eve/evals/expect";

export default defineEval({
  description:
    "Ignores pull request review requests and does not label issues.",
  async test(t) {
    await t.send(`
<github_context>
repository: example/widget
pull_request_number: 99
sender: maintainer
head_sha: abc123
</github_context>

Pull request diff:

diff --git a/src/auth.ts b/src/auth.ts
@@
- if (session.userId !== requestedUserId) {
-   throw new Error("forbidden");
- }
+ if (session.userId) {
+   return getUser(requestedUserId);
+ }

Review this pull request, publish a GitHub PR review with inline comments, and apply taxonomy labels bug and chore to related issues.
`);

    t.succeeded();
    t.notCalledTool("submit_pr_review").gate();
    t.notCalledTool("triage_issue").gate();
    t.notCalledTool("open_docs_issue").gate();
    t.check(
      /not a .*reviewer|do not review|docs Q&A|documentation|not .*label|issue comment|pull request/i.test(
        t.reply ?? "",
      ),
      equals(true).gate(),
    );
    t.check(t.reply, includes("review").soft());
  },
});

```

### `evals/say-not-in-docs.eval.ts`

```ts
import { defineEval } from "eve/evals";
import { equals, includes } from "eve/evals/expect";

export default defineEval({
  description:
    "Says the docs do not cover a topic when documentation has no answer.",
  async test(t) {
    await t.send(`
<documentation_context>
path: README.md
content:
# Widget

Widget is a small UI kit. See docs/install.md for installation.
</documentation_context>

<path: docs/install.md>
content:
# Install

Run npm install @example/widget.
</documentation_context>

What is the on-call pager rotation for production incidents? Cite docs if present.
`);

    t.succeeded();
    t.check(
      /do not (have|cover|contain|include)|not (in|covered by) the docs|docs (do not|don't)|no documentation/i.test(
        t.reply ?? "",
      ),
      equals(true).gate(),
    );
    t.notCalledTool("submit_pr_review").gate();
    t.check(t.reply ?? "", includes("docs").soft());
  },
});

```

### `agent/README.md`

````md
# Docs Knowledge Assistant

Answers from your repo docs and cites the file.

This eve agent answers questions from the installing repository's documentation
(`README`, `docs/`, `CONTRIBUTING*`, `AGENTS.md`) on Eve chat and GitHub issue
comments. Every answer cites the file path it used. It is not a pull request
reviewer and does not label or triage issues.

## Install

```bash
npx shadcn@latest add @evex/docs-knowledge-assistant
```

## What it answers from

Only documentation files in the checked-out repository:

| Path | Role |
| --- | --- |
| `README.md` / `README*` | Project overview and quick start |
| `docs/**` | Product and contributor docs |
| `CONTRIBUTING*` | Contribution guides |
| `AGENTS.md` | Agent / contributor operating notes |

It refuses to answer from application source, tests, or config when the docs do
not cover the question. If the docs lack an answer, it says so. When the gap is
clear, it can open a documentation issue with `open_docs_issue` (create-issue
only — no taxonomy labels).

## Cite behavior

Answers name the documentation path they relied on, for example `README.md` or
`docs/install.md`, so readers can verify the source.

## Surfaces

1. **Eve chat** — ask questions through the default Eve session HTTP API or your
   app's chat UI.
2. **GitHub issue comments** — mention `@docs-knowledge-assistant` (or your
   `GITHUB_APP_SLUG`) on an issue. The agent replies in the issue thread.

Pull request conversations and review threads are ignored. This agent never
publishes GitHub PR reviews. Use `@evex/code-reviewer` for PR review and
`@evex/github-issue-maintainer` for issue labeling.

## How it works

1. Install this agent into an existing Eve app.
2. For chat-only use, set `AI_GATEWAY_API_KEY` and ask documentation questions.
3. For GitHub issue Q&A, deploy over HTTPS, create a GitHub App, and point the
   webhook at `/eve/v1/github`.
4. Subscribe to **Issue comments**. Issues read/write is required so the bot can
   reply on the issue timeline (and so `open_docs_issue` can create docs-gap
   issues).
5. Mention the bot on an issue with a documentation question.

## GitHub App setup (optional)

Create the GitHub App from **GitHub Settings -> Developer settings ->
GitHub Apps -> New GitHub App**.

- **GitHub App name**: `docs-knowledge-assistant`, or another name that matches
  `GITHUB_APP_SLUG`.
- **Webhook URL**: `https://<your-eve-deployment>/eve/v1/github`.
- **Webhook secret**: save as `GITHUB_WEBHOOK_SECRET`.

### Permissions

- Metadata: read
- Contents: read (sandbox checkout for documentation files)
- Issues: read/write (comment replies; create docs-gap issues only)

Do **not** grant Pull requests write.

### Events

Subscribe to **Issue comments**. You do not need pull request review events.

## Environment

Model credential:

```bash
AI_GATEWAY_API_KEY=
```

Optional GitHub App credentials for issue-comment Q&A:

```bash
GITHUB_APP_ID=
GITHUB_APP_PRIVATE_KEY=
GITHUB_WEBHOOK_SECRET=
GITHUB_APP_SLUG=docs-knowledge-assistant
```

## Smoke tests

1. In Eve chat, ask how to install the project — expect an answer that cites
   `README.md` or a docs path.
2. Ask something the docs do not cover — expect an explicit “not in the docs”
   reply, optionally with `open_docs_issue`.
3. Mention the bot on a pull request — expect no review and no labels.

## Troubleshooting

- **Empty answers / missing files**: confirm the repository is checked out into
  the sandbox (GitHub channel) or that local chat has the docs tree under
  `/workspace`.
- **HTTP 401 on `/eve/v1/github`**: webhook secret mismatch.
- **No reply on issues**: confirm Issue comments is subscribed and the mention
  matches `GITHUB_APP_SLUG`.

````