Skip to main content

AI Code Review FAQs

Last updated on

Questions about how AI Code Review behaves, followed by the failures teams hit most often.

General​

Does AI Code Review replace human reviewers?

No. AI Code Review reports findings as status checks alongside your existing review process. It does not approve, merge, or block a pull request on its own, and it has no push permission, so it cannot commit a fix.

Can AI Code Review block a merge?

Not by itself. It reports one status check per review criterion. Whether a failing check blocks a merge is decided by your branch protection rules. Because there is no aggregate check, requiring AI Code Review means requiring the individual check identifiers.

Does the review agent run as me, or as its own identity?

As its own identity. Onboarding creates a dedicated service account named aicr_service_account with four permissions: read the repository and its checks, add pull request comments and reviewers, report status checks, and resolve users. Push permission is deliberately excluded.

Which source control platforms are supported?

GitHub Cloud, GitHub Enterprise Server, and Harness Code Repository. GitLab and Bitbucket are not supported. GitHub repositories are linked into Harness Code and reviewed there rather than in place.

Do I need to install an AI Code Review GitHub App?

No. AI Code Review uses standard Harness GitHub connectors with any supported authentication type, including personal access token and OAuth. Most teams already have a connector that works.

Which models can AI Code Review use?

Anthropic models only at beta, including Sonnet, Opus, and Fable. Opus is recommended. Amazon Bedrock is supported. Bring your own key, or use the Harness AI gateway.

Where do the suggested labels and reviewers come from?

Labels are picked from the labels that already exist in the repository. Reviewer suggestions come primarily from the git blame of the files in the patch. Both are suggestions and neither overrides a CODEOWNERS rule or a required reviewer policy.

Can an administrator stop a team overriding an inherited setting?

No. A lower scope can always set its own connector or system prompt and win. The practical control is who holds repo_edit at each scope.

Is my source code sent to a third-party model provider?

The agent clones the repository and calls an Anthropic model, either through the Harness AI gateway or through your own key. Amazon Bedrock is supported for accounts that require models to stay inside AWS. Confirm data handling terms with your Harness account team before rolling out to a regulated codebase.

Configuration​

Can review criteria be stored in Git alongside the code?

Not today. Criteria, the connector, and the system prompt live in Harness and are managed through the UI or the settings API. There is no configuration-as-code path for them. The generated pipeline is editable YAML, but it is stored in Harness rather than in your repository.

Can a repository override a criterion it inherits from a parent space?

No. Criteria are combined down the hierarchy, never replaced. A repository can add criteria but cannot disable or change one it inherits. Restating an inherited criterion produces two checks rather than overriding it. Change it at the scope that owns it.

Why does a review criterion appear twice on a pull request?

Criteria are not deduplicated across scopes. The same title defined at both project and repository level runs twice and produces two checks. Remove it from one scope.

Does offboarding remove everything AI Code Review created?

No. Offboarding deletes the repository pull request trigger and disables the repository review flag. The pipeline, service account, role, token, secret, and the repository settings and criteria all remain. If you onboarded at organization or account scope, triggers in other projects are also left in place.

How long is the token AI Code Review creates valid for?

Fifteen years. Onboarding creates a service account token stored as the secret aicr_token_secret and it is not rotated automatically. Include it in whatever credential review process you already run.

Reviews do not run​

Harness AI Code Review does not run when a pull request is opened on an onboarded repository

Confirm the trigger aicr_pr_trigger_<repository> exists in the target project. The trigger fires only on pull request create, update, and reopen, so a pull request that existed before onboarding is not reviewed until the next push.

Harness AI Code Review reports no status checks although the aicr pipeline ran successfully

A repository with no enabled review criteria produces no checks. Confirm the repository, or a space above it, defines at least one criterion with enabled set to true. A non-recursive settings read only shows criteria defined at that exact scope, so read with recursive set to true.

Onboarding a GitHub repository fails because the repository identifier is missing the provider owner

An account-level connector requires the full provider path in owner/repo form for each repository. A bare name produces an ownerless URL that GitHub returns as not found. Supply owner/repo, or use a repository-level connector and leave the identifier empty.

Onboarding a GitHub repository times out while the repository is still importing

A newly linked repository must finish importing before the pull request trigger can be registered, because webhook registration is rejected during import. Large repositories take longer. Re-run onboarding once the import completes; onboarding is idempotent.

Results look wrong​

A Harness AI Code Review status check reports error rather than a pass or fail result

Error means no verdict was produced for that criterion, and a check that is still pending or running also reads as error through the API. Confirm the pipeline execution completed, then push a commit to re-run the review.

Harness AI Code Review status checks disappear from a pull request after a force push

Results are recorded against a specific head commit. A force push moves the head past the reviewed commit, so the stored results no longer apply and are not shown. The next run reports against the new head commit.

Harness AI Code Review findings are generic and reviewers are ignoring them

This is almost always the criterion rather than the agent. A description that names a topic produces observations; one that names a specific failure condition, bounds the paths it applies to, and asks for the location produces actionable findings. Start with two tight criteria rather than a broad set.

Review criteria disappeared after updating Harness AI Code Review settings

An update replaces the entire criteria set for that scope rather than patching it. Criteria absent from the update payload are deleted, and omitting the enabled field is the same as sending false. Read the setting, modify it, and write the whole object back.

No Harness AI Code Review comment appears on the pull request although status checks are reported

Comment posting is best effort and never fails a review. It is skipped when the Harness UI base URL is not configured, or when the repository has no linked provider connector. Status checks are unaffected.

Harness AI Code Review posts a second comment on the same pull request instead of updating the first

A comment is posted per review round, keyed to the head commit. A new commit starts a new round and a new comment. Earlier comments are left in place because there is no delete step.

A reviewer cannot open the View results link from a Harness AI Code Review comment

The link points at the AI Code Review page in the Harness UI and requires a Harness session with access to that project. A GitHub-only reviewer still sees the comment and the status checks, but not the full written overview.