GitHub AI app scanning: scope the preview pilot

GitHub announced AI applications code scanning in public preview on 28 October 2026. Evaluate it as an advisory pull-request detector with a specific scope: the current documentation limits preview detection to indirect prompt injection in AI agents. It does not scan the repository backlog or block merges. Review eligibility, separate enablement and usage costs before designing a pilot around the announcement. ^{[1] [2]}

This is a dated adoption analysis of the October announcement, checked on 3 November 2026 in Europe/Amsterdam. It addresses how an engineering team can evaluate the detector without turning missing findings into a safety claim. The broader announcement and narrower current coverage description should both be visible in the decision record. ^{[1] [2]}

Scenario: an internal support agent reads issue text and retrieved troubleshooting documents, then proposes maintenance actions. Its owners want a pull-request review signal when code connects untrusted content to a model that can reach tools. They already use code scanning. They need to establish what this additional detector covers, what it skips, who reviews its output, and what evidence would justify wider adoption.

What did GitHub announce?#

The announcement introduces scanning for risks in AI applications and agents. It names indirect prompt injection, sensitive information disclosure and excessive agency, describes pull-request analysis, and identifies the feature as public preview. The operational documentation provides the narrower current coverage needed to plan a test. ^{[1] [2]}

In that documentation, the preview currently detects indirect prompt injection vulnerabilities in AI agents. It covers pull requests rather than a full repository or existing backlog. Treat the broader risk list as product context, not a promise that the current preview detects every listed category in a pilot repository. ^[2]

This distinction changes the acceptance procedure. A test that introduces an excessive-permission problem and sees no finding cannot establish that the detector malfunctioned if that case is outside documented preview coverage. It also cannot establish that the permission design is safe. Record the case as outside the claimed test scope and keep the independent security review.

The announcement and documentation do not establish detection quality for your architecture. They describe availability and intended behavior. A useful evaluation therefore starts with an explicit risky data flow and a control that repairs that flow, while preserving the uncertainty around model-based detection.

Which preview limits change the pilot?#

Pull-request provenance matters. The current documentation excludes fork pull requests and pull requests created by Dependabot. Findings are advisory and cannot be used in rulesets to enforce merge requirements. A clean merge check should therefore not be interpreted as proof that this detector evaluated and approved the change. ^[2]

Separate four states in the pilot log: a qualifying pull request was examined and a finding appeared; a qualifying pull request produced no observed finding; the change was outside documented coverage; or scanning execution could not be established. These are our proposed evaluation states, not GitHub status names or API fields.

The detector's use of an LLM-based approach is distinct from CodeQL analysis. GitHub describes no language or framework limitation for that approach, but that is not a guarantee that every abstraction, wrapper or dynamic dispatch pattern will be recognized. Keep language eligibility separate from observed detection on the code path you care about. ^[2]

For the support-agent scenario, the security owner should identify where untrusted text enters, where it reaches model context, and where model output can influence actions. The pilot should preserve those code locations in its record. Merely adding an alarming string to a file does not establish a meaningful architecture test.

How is this different from AI Scan?#

AI applications scanning is a separate detector from AI Scan for pull requests and from CodeQL. AI Scan focuses on traditional code-security vulnerabilities and complements CodeQL coverage. The AI applications detector addresses risks particular to AI applications and agents, with the current preview limit described above. ^{[2] [3]}

A likely settings mistake is specific: a team enables AI Scan and assumes the AI applications detector is included. The documentation says each organization-level feature must be enabled separately, even though enterprise policy is shared. Verify the effective control for the detector under evaluation. ^{[2] [4] [5]}

Do not copy licensing assumptions from a neighboring feature. The AI applications documentation says it requires GitHub Advanced Security on GitHub Enterprise Cloud, including data residency, and consumes AI credits in addition to that license. Its requirements also apply to eligible public repositories. The separate AI Scan documentation has its own availability and licensing description. ^{[2] [3]}

For readers who have already reviewed AI Scan for pull requests, this is an additional evaluation decision. It is not a renaming of the same scanner or a reason to discard existing analysis. Write the actual detector name in the pilot record so later readers can connect findings to the control that produced them.

Who must enable it and pay for usage?#

Coordinate the enterprise policy owner, organization security owner, repository administrator and billing owner before broad enablement. The enterprise policy controls whether organizations may use the features; allowing that policy does not itself enable either detector. Organization administrators enable them separately, and eligible repositories inherit the organization setting with repository opt-outs. ^{[4] [5]}

Repository inheritance determines the actual pilot scope. An organization-level switch can affect more than the single repository named in an experiment plan. Identify eligible repositories, existing opt-outs and the intended rollout scope before changing settings. A billing or security owner should not discover the actual scope from the first usage report.

This article does not enable any setting or provide a click-by-click tenant procedure. Record the effective state in the real environment and follow the current administrative documentation. The purpose is to make the decision reviewable before a setting begins producing scans and possible charges.

Separate active-committer licensing from metered AI usage. GitHub's billing guidance distinguishes product licensing, AI credits and budget behavior. Review the applicable billing account, included allowances and hard-budget controls rather than inferring a free trial from public repository visibility or a neighboring Copilot entitlement. No price estimate is asserted here. ^{[2] [6]}

Ask the billing owner what evidence will show usage stopped, was prevented or remained within the approved scope. A budget alert and an enforcement limit are different controls. Document their actual configured behavior and the person responsible for responding when the pilot cannot scan because of a limit. ^[6]

Design a pilot that can produce useful negative evidence#

Begin with a small fixture repository or isolated test branch containing no sensitive data and no live action credentials. The fixture should represent the support agent's real architecture at a useful level: an external document reader, model context assembly and a proposed tool dispatch. Keep the example deliberately non-operational so its purpose is code review rather than an action against business systems.

The internal record below is a proposed test description, not a supported GitHub configuration or API request. The states are deliberately explicit so the absence of a comment cannot silently become a pass.

{
  "pilot": "support-agent-xpia-review",
  "detector": "AI applications scanning",
  "documentedScope": "PR indirect prompt injection in agents",
  "fixtures": [
    "untrusted-document-to-model-to-tool-proposal",
    "same-path-with-independent-action-validation",
    "benign-document-with-security-vocabulary"
  ],
  "recordPerCommit": [
    "origin-and-eligibility",
    "effective-setting-evidence",
    "scan-observation",
    "finding-or-no-observed-finding",
    "reviewer-disposition",
    "separate-runtime-test-result"
  ]
}

For the risky fixture, have a reviewer explain the data flow before viewing detector output. The explanation should identify the trust boundary and possible consequence. Then inspect whether the detector points to that path and whether its explanation helps the reviewer choose a repair. Avoid tuning the fixture repeatedly until it produces a screenshot that looks successful.

For the repaired control, enforce action validation through trusted application code with a constrained destination and operation. A prompt requesting careful behavior is insufficient evidence of that repair. Runtime tests should show what happens when retrieved text asks for an unauthorized action, including how the system rejects or escalates it. The scanner remains a review aid rather than the executing permission boundary.

Include a benign control that contains the same security vocabulary in ordinary documentation. This helps expose a pilot that rewards any warning regardless of usefulness. Record false positives, unclear explanations and missing findings, with the exact commit and case. A detector that generates more comments is not automatically a better fit for the team's review process.

Keep the real PR origin in the record. A fork exclusion should not be worked around by quietly reclassifying the original contribution as a successful detector test. Instead, use an explicitly authorized internal fixture for the in-scope test and retain the excluded case as a coverage limit. ^[2]

For a non-finding, establish only what the evidence supports: the documented eligibility was checked, relevant settings were observed, and no finding was seen for the saved case and commit. If scan execution itself remains uncertain, mark that uncertainty. Do not merge the uncertainty into the benign-control outcome.

What should happen to a finding and an autofix?#

Assign an engineering disposition to each finding: confirmed path, false positive, insufficient context, accepted risk under a named owner, or a proposed repair awaiting verification. Preserve the detector explanation alongside the reviewer's reasoning. The two should remain distinguishable when the pilot is evaluated later.

Current documentation says findings appear as PR comments with an AI app label and feedback controls. It also says Copilot Autofix suggestions are generated automatically for this detector, independently of the setting that controls CodeQL Autofix. A team that previously disabled or reviewed another autofix feature should verify this separate behavior. ^[2]

Review a suggested patch before accepting it. Review which authority it changes, whether it breaks legitimate workflows, and whether it moves untrusted input to another unreviewed path. Execute the relevant acceptance tests in an isolated environment. A patch can remove a visible warning while leaving a different route to the same action.

The release owner still needs a merge decision outside these advisory findings. Define how an unresolved confirmed path is escalated and which existing controls actually enforce that decision. The documented inability to use these findings as ruleset requirements makes that process especially important. ^[2]

The pilot design here was checked against the announcement and current primary documentation; its JSON record was parsed locally. No tenant setting was changed, no fixture PR was opened and no scanner result was generated for this article. Those are the next environment-specific steps for a team choosing to evaluate the preview.

Before adopting later, reread coverage, eligibility and billing. This is analysis of the dated announcement, not a promise that preview behavior remains fixed. The related Agentic Workflows review addresses workflow execution authority; this article addresses an additional code-review detector and its limits.

Does the preview detect every AI security risk?

The announcement names several AI risk families, but current documentation limits preview detection to indirect prompt injection vulnerabilities in AI agents. Plan tests against that documented scope and retain separate review for other risk categories. ^{[1] [2]}

Does enabling AI Scan enable AI applications scanning?

No. The features share an enterprise-level policy but have separate organization enablement controls. Verify the effective Scan for AI applications setting and inherited repository state before interpreting the absence of findings as an evaluated change. ^{[2] [4] [5]}

Can its findings block a merge through rulesets?

The documented findings are advisory and cannot serve as ruleset merge requirements. Define a separate review and escalation process for confirmed issues, and verify which existing controls enforce the team's release decision. ^[2]

Does it scan old repository code and fork contributions?

The documented detector covers pull requests rather than the full repository backlog, and excludes fork and Dependabot-created pull requests. Record those exclusions in coverage reports; missing findings for excluded changes do not establish safety. ^[2]