---
title: "Service Agents"
description: "Autonomous AI agents deployed inside your Kubernetes clusters for 24/7 policy violation detection and remediation."
diataxis: how-to
applies_to:
  product: "nirmata-ai-agents"
audience: ["platform-engineer"]
last_updated: 2026-09-08
url: https://docs.nirmata.io/docs/control-hub/agent-hub/service-agents/
---


> **Applies to:** Nirmata AI Agents 1.0 and later

Service Agents are autonomous AI agents that run inside your Kubernetes clusters. Unlike [Cloud Agents](../cloud-agents/), which are launched on-demand from the Control Hub, Service Agents are deployed directly into the cluster and operate continuously — detecting policy violations, generating remediation plans, and opening pull requests in your Git repositories without human intervention.

The primary Service Agent is the **Remediator Agent**: it monitors Kyverno policy reports, uses AI to generate compliant fixes, and integrates with your GitOps workflow by creating PRs against the affected repository.

For a conceptual overview of Service Agents, see [Service Agents in Nirmata AI Agents](/docs/ai/service-agents/).

## In This Section

- [**Getting Started**]({{< relref "getting-started/" >}}) — Install the Remediator Agent and verify your first remediation
- [**Configuration**]({{< relref "configuration/" >}}) — Configure ToolConfig (Git credentials), LLMConfig (AI provider), and the Remediator resource
- [**Observability**]({{< relref "observability/" >}}) — Prometheus metrics, status fields, and monitoring the agent in production
- [**GitHub Authentication**]({{< relref "github-authentication.md" >}}) — Connect to GitHub using the Nirmata GitHub App

## How It Works

1. The Remediator Agent is installed in the `nirmata` namespace via Helm.
2. It reads Kyverno `ClusterPolicyReport` resources to discover violations.
3. For each violation, it generates a fix — using a configured AI provider, or deterministically with no AI at all in `prescriptive` mode.
4. It opens a pull request in the target Git repository with the proposed change.
5. You review and merge the PR — the agent never pushes directly to a protected branch.

## Key Capabilities

| Capability | Description |
|------------|-------------|
| **Continuous monitoring** | Runs on a cron schedule or triggered on-demand via the Kubernetes API |
| **Multi-cluster support** | Hub Mode uses ArgoCD or FluxCD to manage violations across hundreds of clusters |
| **GitOps integration** | All changes go through PRs — no direct cluster mutations |
| **Confidence-based actions** | Configure whether PRs are opened for high-confidence fixes, low-confidence, or both |
| **Split PR** | Split a multi-policy PR into separate PRs for independent review workflows |
| **Policy exception requests** | Request an exception from the PR instead of a fix, routed through Control Hub approval |
| **AI provider choice** | Nirmata AI (default), AWS Bedrock, Azure OpenAI, or the Anthropic API |
| **Optional AI** | `prescriptive` remediation mode right-sizes workload resources deterministically, with no LLM in the path |


---

## Getting Started


## Prerequisites

### Required Components

- **Kubernetes cluster** running Kubernetes 1.20+
- **Helm** 3.x installed and configured
- **kubectl** configured to access your cluster
- **Kyverno** installed and running with policy reports enabled
- **ArgoCD** or **FluxCD** (optional) — required for the `argoHub` / `fluxHub` multi-cluster modes

### Required Credentials

- **Nirmata Control Hub Service Account Token** — create a Service Account in Nirmata Control Hub and copy its secret

This token is required for the default configuration, which uses Nirmata AI as the LLM provider and the Nirmata GitHub App for Git access. If you supply **both** your own LLM provider (AWS Bedrock, Azure OpenAI, or the Anthropic API) **and** your own Git credentials (a PAT or [your own GitHub App](../github-authentication/#3-custom-github-app)), you can install with `nirmata.auth=none` and skip this step entirely — see [Nirmata Platform Authentication](../configuration/#nirmata-platform-authentication) for what that turns off.

### Create a Service Account

1. Log in to [Nirmata Control Hub](https://nirmata.io)
2. Navigate to **Identity & Access** from the left sidebar
3. Go to **Service Accounts** and create a new one
4. Copy the generated secret — you'll use it in the next step

---

## Installation

### 1. Create Namespace and Secrets

```bash
kubectl create namespace nirmata

kubectl create secret generic nirmata-service-account-token \
  --from-literal=service-account-token=YOUR_NCH_SERVICE_ACCOUNT_TOKEN \
  --namespace nirmata
```

### 2. Add the Helm Repository

```bash
helm repo add nirmata https://nirmata.github.io/kyverno-charts
helm repo update nirmata
```

### 3. Install the Agent

```bash
helm install nirmata-agent nirmata/nirmata-agent --devel \
  --namespace nirmata \
  --create-namespace
```json

{{% alert title="Note" %}}
The chart uses `serviceAccountToken` authentication by default and expects the `nirmata-service-account-token` secret created above. To use API token authentication instead, set:

```bash
--set nirmata.auth="apiToken" \
--set nirmata.apiTokenSecret="nirmata-api-token"
```

To run with no Nirmata platform credential at all, set `--set nirmata.auth="none"` and configure your own LLM provider and Git credentials.
{{% /alert %}}

{{% alert title="Configuring through Helm values" %}}
This guide creates the three custom resources by hand so each one is visible. The chart can also create them from values: `llm.*` renders the LLMConfig and `tool.*` renders the ToolConfig (both enabled by default), and `remediator.enabled=true` renders the Remediator. `remediator.enabled` defaults to `false`, so the install above deploys the controller without a Remediator until you create one.
{{% /alert %}}

### 4. Verify the Installation

```bash
# Check pods are running
kubectl get pods -n nirmata -l app.kubernetes.io/name=nirmata-agent

# Check custom resource definitions were installed
kubectl get llmconfigs,toolconfigs,remediators -n nirmata

# Tail the agent logs
kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent --tail=50
```yaml

---

## Quick Configuration

The agent requires three custom resources before it will start remediating. See [Configuration](../configuration/) for full details and all available options.

### 1. ToolConfig (Git credentials)

**GitHub Personal Access Token:**

```bash
kubectl create secret generic github-pat-token \
  --from-literal=token=GITHUB_PAT_TOKEN \
  --namespace nirmata

kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: github
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: github-pat-token
        namespace: nirmata
        key: token
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations: "
        commitMessageTemplate: "Auto-fix: Remediate policy violations: "
        customLabels:
          - "auto-remediation"
        systemLabels:
          - "clusterName"
          - "namespace"
EOF
```

Prefer using the [Nirmata GitHub App](../github-authentication/) over a personal access token — it avoids managing secrets manually and provides automatic token rotation.

### 2. LLMConfig (AI provider)

The Helm chart creates a default LLMConfig using Nirmata AI automatically. If you need to create it manually:

```bash
kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: nirmataAI
  nirmataAI:
    model: ""
EOF
```

### 3. Remediator

**Local Mode** (scans the cluster where the agent is installed):

```bash
# Create a ConfigMap mapping Git repos to namespaces
kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: repo-namespace-mapping
  namespace: nirmata
data:
  mapping: |
    [
      {
        "repo": "https://github.com/your-org/your-repo",
        "branch": "main",
        "path": "k8s/",
        "targetNamespace": "default"
      }
    ]
EOF

# Create the Remediator
kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: Remediator
metadata:
  name: remediator-local
  namespace: nirmata
spec:
  environment:
    type: localCluster
  target:
    localCluster:
      repoNamespaceMappingRef:
        name: repo-namespace-mapping
        namespace: nirmata
        key: mapping
  remediation:
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    gitCredentials:
      name: toolconfig-sample
      namespace: nirmata
    eventPolling:
      enabled: true
      intervalMinutes: 5
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
EOF
```yaml

---

## Uninstalling

```bash
helm uninstall nirmata-agent -n nirmata
```

Secrets you created manually are not removed — clean those up separately if needed.


---

## Configuration


The Remediator Agent is configured through three Kubernetes custom resources:

| Resource | Purpose |
|----------|---------|
| **ToolConfig** | Git provider credentials and PR defaults |
| **LLMConfig** | AI provider settings (Nirmata AI, AWS Bedrock, Azure OpenAI) |
| **Remediator** | What to scan, when to run, and what actions to take |

---

## Nirmata Platform Authentication

The Helm chart's `nirmata.auth` value selects how — or whether — the agent authenticates to Nirmata Control Hub.

| `nirmata.auth` | Credential injected | Use when |
|---|---|---|
| `serviceAccountToken` (default) | `SERVICE_ACCOUNT_TOKEN` from `nirmata.serviceAccountTokenSecret` | Standard install |
| `apiToken` | `API_TOKEN` from `nirmata.apiTokenSecret` | You use an API token instead of a service account |
| `none` | None | You supply your own LLM provider **and** your own Git credentials |

Set `nirmata.auth=none` to run the agent with no Nirmata platform credential at all — for example an install with no Control Hub connectivity, using AWS Bedrock plus a GitHub PAT. The agent still reaches your LLM provider and Git provider, so this removes the dependency on Control Hub, not on outbound network access. The following Remediator features require a credential and are unavailable with `none`:

- The `nirmataAI` LLM provider
- The `nirmata-app` Git authentication method
- Policy exception requests (the `@nirmatabot request-exception` command)

The chart validates this at install time rather than letting it fail at runtime. Combining `nirmata.auth=none` with `llm.provider=nirmataAI` or `tool.credentials.method=nirmata-app` aborts the install with an explanatory error.

---

## ToolConfig

ToolConfig defines how the agent authenticates with your Git provider and sets defaults for the pull requests it creates.

### GitHub — Personal Access Token

```bash
kubectl create secret generic github-pat-token \
  --from-literal=token=GITHUB_PAT_TOKEN \
  --namespace nirmata

kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: github
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: github-pat-token
        namespace: nirmata
        key: token
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations: "
        commitMessageTemplate: "Auto-fix: Remediate policy violations: "
        customLabels:
          - "auto-remediation"
          - "security"
        systemLabels:
          - "clusterName"
          - "namespace"
EOF
```text

### GitHub — Nirmata GitHub App (Recommended)

Using the Nirmata GitHub App avoids managing secrets manually and provides automatic token rotation. Follow the [GitHub Authentication guide](../github-authentication/) to set up the App integration first.

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: github
  credentials:
    method: nirmata-app
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations: "
        commitMessageTemplate: "Auto-fix: Remediate policy violations: "
        customLabels:
          - "auto-remediation"
        systemLabels:
          - "clusterName"
          - "namespace"
```text

### GitHub — Custom GitHub App

Use your own GitHub App when it must be owned by your organization, or when the agent runs with `nirmata.auth=none`. See the [GitHub Authentication guide](../github-authentication/#3-custom-github-app) for creating the App and the permissions it needs.

```bash
kubectl create secret generic github-app-private-key \
  --from-file=private-key.pem=/path/to/your-app.private-key.pem \
  --namespace nirmata
```

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: github-app-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: app
    app:
      appId: 123456                 # integer, not a string
      # installationId: 78901234    # optional — auto-discovered when omitted
      privateKeySecretRef:
        name: github-app-private-key
        key: private-key.pem
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
```

The private key Secret must be in the same namespace as the ToolConfig.

### GitLab

```bash
kubectl create secret generic gitlab-pat-token \
  --from-literal=token=GITLAB_PAT_TOKEN \
  --namespace=nirmata

kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: gitlab
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: gitlab-pat-token
        namespace: nirmata
        key: token
EOF
```text

### Private Certificate Authorities

The agent reaches your Git provider at `github.com` or `gitlab.com` — there is no ToolConfig setting for a GitHub Enterprise Server or self-managed GitLab API endpoint. Use `spec.tls` when that traffic passes through a TLS-intercepting proxy or egress gateway presenting a corporate certificate. Without the CA bundle, the agent fails with a certificate verification error.

```bash
kubectl create secret generic git-ca-bundle \
  --from-file=ca.crt=/path/to/corporate-ca.pem \
  --namespace nirmata
```

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: gitlab
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: gitlab-pat-token
        namespace: nirmata
        key: token
  tls:
    caBundleSecretRef:
      name: git-ca-bundle
      namespace: nirmata
      key: ca.crt
```

`spec.tls.insecureSkipVerify: true` disables certificate verification entirely. Use it only to confirm that a failure is certificate-related — never in production.

### Pull Request Labels

ToolConfig supports two label types:

- **`customLabels`** — static labels always applied to every PR (e.g., `auto-remediation`, `security`)
- **`systemLabels`** — dynamic labels computed at runtime from remediation context:

| System Label | Value |
|---|---|
| `branch` | The Git branch being remediated |
| `clusterName` | The cluster where violations were found |
| `appName` | ArgoCD application name (Hub Mode only) |
| `namespace` | Kubernetes namespace containing violations |

---

## LLMConfig

LLMConfig specifies the AI provider used to generate remediation plans.

### Nirmata AI (Default)

The Helm chart creates this automatically. Authentication uses the Service Account token injected at deployment time — no additional secrets needed.

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: nirmataAI
  nirmataAI:
    model: ""   # empty string uses the current default model
```text

### AWS Bedrock

For EKS clusters with Pod Identity Agent:

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: bedrock
  bedrock:
    model: MODEL_ARN_OR_INFERENCE_ARN
    region: AWS_REGION
```text

Bedrock also accepts `credentialsSecretRef` (a Secret holding static AWS credentials), `roleArn`, and `externalId` if Pod Identity is not available.

<details>
<summary>Full AWS IAM setup</summary>

```bash
aws iam create-role \
  --role-name remediator-agent-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }]
  }'

aws iam put-role-policy \
  --role-name remediator-agent-role \
  --policy-name BedrockInvokePolicy \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": "arn:aws:bedrock:<REGION>:<ACCOUNT_ID>:application-inference-profile/<PROFILE>"
    }]
  }'

aws eks create-pod-identity-association \
  --cluster-name <CLUSTER_NAME> \
  --namespace nirmata \
  --service-account nirmata-agent \
  --role-arn arn:aws:iam::<ACCOUNT_ID>:role/remediator-agent-role
```text
</details>

### Azure OpenAI

```bash
kubectl create secret generic azure-openai-credentials \
  --from-literal=api-key=AZURE_API_KEY \
  -n nirmata
```

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: azure-openai
  azureOpenAI:
    endpoint: https://YOUR_RESOURCE_NAME.openai.azure.com/
    deploymentName: DEPLOYMENT_NAME
    apiKeySecretRef:
      name: azure-openai-credentials
      key: api-key
      namespace: nirmata
```yaml

### Anthropic API

Calls the Anthropic API directly with your own API key.

```bash
kubectl create secret generic anthropic-credentials \
  --from-literal=api-key=ANTHROPIC_API_KEY \
  -n nirmata
```

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: anthropic
  anthropic:
    model: claude-sonnet-5          # required — no default
    apiKeySecretRef:
      name: anthropic-credentials
      namespace: nirmata
      key: api-key
    # baseURL: https://ai-gateway.internal
```

`model` is required and has no default, so the model in use never changes silently on an agent upgrade. Set `baseURL` — including the scheme — to route requests through an internal AI gateway or proxy instead of `https://api.anthropic.com`.

---

## Remediator

The Remediator resource ties everything together: it defines the environment mode, which clusters or applications to target, the schedule, and what actions to take.

### Environment Modes

| Mode | Use Case |
|------|----------|
| `localCluster` | Scan the cluster where the agent is installed |
| `argoHub` | Use ArgoCD to manage violations across multiple clusters |
| `fluxHub` | Use FluxCD to manage violations across multiple clusters |

Direct repository scanning (`target.vcs`) is **not** an environment type — it is configured alongside any of the three modes above. See [VCS Target Mode](#vcs-target-mode-direct-repository-scanning).

### ArgoCD Hub Mode (Multi-Cluster)

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: Remediator
metadata:
  name: remediator-argo-hub
  namespace: nirmata
spec:
  environment:
    type: argoHub
  target:
    argoHub:
      appSelector:
        allApps: true    # or specify names / labelSelector
  remediation:
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    gitCredentials:
      name: toolconfig-sample
      namespace: nirmata
    eventPolling:
      enabled: true
      intervalMinutes: 5
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
```

**Targeting specific clusters or apps:**

```yaml
target:
  argoHub:
    clusterNames:
      - production-cluster
      - staging-cluster
    appSelector:
      names:
        - nginx-demo
        - web-app
      labelSelector:
        matchLabels:
          team: platform
          environment: production
```

### FluxCD Hub Mode (Multi-Cluster)

Targets workloads managed by Flux. The agent selects Flux `Kustomization` and/or `HelmRelease` objects, then follows each one's `sourceRef` to its `GitRepository` to discover the repository URL and branch to open the PR against — so no repo-to-namespace mapping is needed.

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: Remediator
metadata:
  name: remediator-flux-hub
  namespace: nirmata
spec:
  environment:
    type: fluxHub
  target:
    fluxHub:
      fluxNamespace: flux-system   # where Flux controllers run; reference only
      kustomizationSelector:
        names:
          - my-app
          - backend-api
  remediation:
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    gitCredentials:
      name: toolconfig-sample
      namespace: nirmata
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
```

Set at least one of `kustomizationSelector` or `helmReleaseSelector` — the Helm chart refuses to install without one, and a Remediator applied directly with neither is admitted but selects no targets. Set both to remediate both object types. Each selector accepts:

| Field | Behavior |
|---|---|
| `names` | Select specific objects by name |
| `namespaces` | Restrict which namespaces are searched. In hub-driven setups each spoke usually has a dedicated namespace on the hub. Empty means all namespaces |
| `labelSelector` | Select by labels, e.g. `nirmata.io/remediation: "enabled"` |
| `allKustomizations` / `allHelmReleases` | Select everything found in the searched namespaces; other fields are ignored |

```yaml
target:
  fluxHub:
    kustomizationSelector:
      namespaces:
        - production
        - staging
      labelSelector:
        matchLabels:
          nirmata.io/remediation: "enabled"
    helmReleaseSelector:
      allHelmReleases: true
```

Flux API versions are detected automatically, covering Flux 0.x through 2.7+ (`v1`/`v1beta2`/`v1beta1` for Kustomization and GitRepository, `v2`/`v2beta2`/`v2beta1` for HelmRelease).

{{% alert title="Prerequisite" %}}
The agent's ClusterRole must be able to read Flux resources. Verify before creating the Remediator:

```bash
kubectl auth can-i list kustomizations.kustomize.toolkit.fluxcd.io \
  --as=system:serviceaccount:nirmata:nirmata-agent -A
kubectl auth can-i get gitrepositories.source.toolkit.fluxcd.io \
  --as=system:serviceaccount:nirmata:nirmata-agent -A
```

If either returns `no`, grant `get`/`list`/`watch` on `kustomizations`, `helmreleases`, and `gitrepositories` to the agent's ClusterRole. Without it, the agent finds no targets.
{{% /alert %}}

### Local Cluster Mode

See the example in [Getting Started](../getting-started/).

### VCS Target Mode (Direct Repository Scanning)

Scan Git repositories directly without requiring a running cluster:

```yaml
spec:
  environment:
    type: localCluster
  target:
    vcs:
      policies:
        - name: pod-security-policy
          repo: https://github.com/your-org/policies
          path: kyverno/pod-security.yaml
          ref: main
      resources:
        - name: web-app-deployment
          repo: https://github.com/your-org/manifests
          path: deployments/web-app.yaml
          ref: main
          policyRefs:
            - pod-security-policy
  remediation:
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
```

### Remediation Modes

`spec.remediation.remediationMode` selects how fixes are generated.

| Mode | Behavior |
|------|----------|
| `ai` (default) | An LLM generates the remediation. `llmConfigRef` is required |
| `prescriptive` | Fixes are computed deterministically with no LLM call anywhere in the path. `llmConfigRef` is not required and is ignored if set |

Prescriptive mode is currently scoped to **resource-optimization** violations: it right-sizes container CPU and memory from VerticalPodAutoscaler recommendations and from bounds defined in the policy itself. Existing Remediators are unaffected — the field defaults to `ai`.

Scoping reuses the standard Kyverno category annotation, so nothing new has to be labelled on your workloads:

```yaml
spec:
  remediation:
    remediationMode: prescriptive
    resourceOptimization:
      policyCategories:
        - "Resource Management"   # default
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    gitCredentials:
      name: toolconfig-sample
      namespace: nirmata
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
```

A violation is remediated only when its policy's `policies.kyverno.io/category` annotation appears in `policyCategories`. Which *workloads* are in scope is decided by the policy's own `match` block — there is nothing to configure here for that.

**Prerequisites**

1. A Kyverno policy annotated with a matching category:

   ```yaml
   metadata:
     annotations:
       policies.kyverno.io/category: Resource Management
   ```

2. For usage-based right-sizing, a `VerticalPodAutoscaler` in recommendation-only mode for each workload:

   ```yaml
   apiVersion: autoscaling.k8s.io/v1
   kind: VerticalPodAutoscaler
   metadata:
     name: nginx
     namespace: my-app-ns
   spec:
     targetRef:
       apiVersion: apps/v1
       kind: Deployment
       name: nginx
     updatePolicy:
       updateMode: "Off"   # recommendation only — never restarts pods
   ```

   Boundary policies (for example "the CPU limit must not exceed 1") are satisfied from the policy bound alone and need no VPA.

**Current limits**

- Workloads with neither a VPA recommendation nor a computable policy bound are skipped and recorded — never guessed.
- Helm charts and kustomize-overlay value indirection are skipped in this phase.
- The Helm chart always renders `llmConfigRef` and has no `remediationMode` value, so apply a prescriptive Remediator as a manifest rather than through the chart.

### Actions

| Action | Behavior |
|--------|----------|
| `CreatePR` | Opens a pull/merge request with the fix |
| `DryRun` | Logs the planned file changes and creates nothing. Useful for a first evaluation. Raise `logging.verbosity` to `2` to log the proposed file contents |

`toolRef` is required by the schema for both action types.

### CI Checks

Every remediation PR the agent opens on GitHub also gets two check runs, so a reviewer can see whether the proposed fix is valid before merging. Nothing needs to be configured — they are created automatically alongside the PR.

| Check | Validates |
|-------|-----------|
| **Kubernetes Resource Validation** | The remediated manifest against the Kubernetes OpenAPI schema — that the change is structurally valid |
| **Policy Compliance** | The remediated manifest against the Kyverno policies, locally — that the change actually resolves the violation |

Each check reports one of three conclusions:

| Conclusion | Meaning |
|---|---|
| `success` | Validation passed |
| `failure` | The proposed change is invalid, or still violates a policy. Review before merging |
| `neutral` | Validation was skipped — for example the policy could not be evaluated locally |

Both checks are advisory: they annotate the PR but never block it, and a failure to create them is logged without affecting the PR. Policy Compliance runs the Kyverno validator directly, so it involves no LLM call and works the same in `prescriptive` mode.

Check runs are a GitHub API feature. GitLab merge requests get no equivalent, and checks are also skipped when no commit SHA is available.

### Confidence-Based Actions

Control when automated PRs are created based on the AI's confidence in its fix:

```yaml
remediation:
  actions:
    - type: CreatePR
      confidence:
        - high     # only create PRs when AI is highly confident
      toolRef:
        name: toolconfig-sample
        namespace: nirmata
```

| Confidence | Meaning |
|-----------|---------|
| `high` | The configured AI provider is highly confident the fix is correct and safe |
| `low` | A potential fix was found but human review is recommended |

Confidence is set by the AI provider, so gating applies to `ai` mode. Prescriptive plans are always `high`.

### Filtering by Severity

```yaml
remediation:
  filters:
    policySelector:
      matchSeverity:
        - high
        - critical
```yaml

---

## Split PR

The Split PR feature lets you split a pull request containing fixes for multiple policies into separate PRs — useful when different policies need different reviewers or approval workflows.

### How to Split

Add a comment to the PR mentioning `@nirmatabot`:

```text
@nirmatabot split-pr require-run-as-non-root disallow-privileged-containers
```

- One or more policy names (space-separated) after `split-pr`
- Use the exact policy names as they appear in the PR

### What Happens

- **Original PR** — updated to contain only the remaining policies; a comment is added with a link to the new PR
- **New PR** — created on a new branch (prefixed `splitpr-`) with only the specified policies; links back to the original PR
- Both PRs are tracked independently by the agent

---

## Request a Policy Exception

When a violation should not be fixed, request an exception from the PR instead. This raises a Policy Exception Request in Nirmata Control Hub for approval.

```text
@nirmatabot request-exception <policy-name> [<policy-name> ...] [duration] [reason]
```

| Argument | Required | Description |
|---|---|---|
| `policy-name` | Yes | One or more Kyverno policy names to except. Use the names as they appear in the PR |
| `duration` | No | A number of days (`7d`, `30d`) or `permanent`. Defaults to permanent when omitted |
| `reason` | No | Free-text justification — all text after the duration token. Shown in the Control Hub approval UI |

```text
@nirmatabot request-exception disallow-capabilities-strict
@nirmatabot request-exception disallow-capabilities-strict 30d
@nirmatabot request-exception disallow-host-ports restrict-apparmor-profiles 7d Approved by security team
```

A `reason` is only parsed after a duration token, so include a duration whenever you want to record a justification.

### What Happens

1. The named policies are removed from the remediation PR, and the agent comments with the request name and the policies still covered by the PR.
2. A Policy Exception Request is created in Control Hub and approvers are notified by email. Track it under **Policies → Exception Requests**.
3. On approval, Control Hub opens a **second PR** containing a Kyverno `PolicyException` manifest scoped to that exact resource. Merge that PR to activate the exception.

If a request already exists for a policy, the agent reports it and skips that policy rather than filing a duplicate.

{{% alert title="Requires Nirmata Control Hub credentials" %}}
This command calls the Control Hub API, so it is unavailable when the agent is installed with `nirmata.auth=none`. See [Nirmata Platform Authentication](#nirmata-platform-authentication).
{{% /alert %}}


---

## Observability


## Prometheus Metrics

The agent serves Prometheus metrics on container port `8080` (`metrics.port`), exposed through the `service-agent-metrics-service` Service on port `8443`. The endpoint is plain HTTP by default (`metrics.secure=false`).

### Available Metrics

Reconcile and GitOps metrics are shared by all Service Agent controllers and carry a `controller_type` label — filter on `controller_type="remediator"` for the Remediator Agent. All of these also carry `namespace` and `k8s_uid`, identifying the custom resource the run belongs to.

| Metric | Type | Labels | Description |
|--------|------|--------|-------------|
| `agent_reconcile_total` | Counter | `controller_type`, `result="success\|error"` | Reconciliation runs |
| `agent_reconcile_duration_seconds` | Histogram | `controller_type`, `result="success\|error"` | Duration of each reconciliation run |
| `agent_outcome_total` | Counter | `controller_type`, `outcome_type` | Generic outcomes. `outcome_type="REMEDIATION_PLANS_GENERATED"` counts plans produced |
| `agent_gitops_outcome_total` | Counter | `controller_type`, `outcome_type`, `repo`, `source_branch`, `target_branch` | GitOps outcomes: `PR_OPENED`, `PR_MERGED`, `ISSUE_CREATED` |
| `agent_gitops_outcome_latency_seconds` | Histogram | `controller_type`, `outcome_type`, `repo` | Time to reach a GitOps outcome. Currently observed for `PR_MERGED` (PR open to merge) |
| `remediator_actions_executed_total` | Counter | `type`, `status="success\|error"` | Action executions by action type |
| `remediator_violations_active` | Gauge | `instance_id`, `severity` | Active violations by severity |
| `remediator_resources_skipped_unchanged_total` | Counter | `collector_id` | Resources skipped because they have not changed since the last run |

Standard controller-runtime, workqueue, leader-election, and Kubernetes client metrics are exposed on the same endpoint.

### Enable ServiceMonitor

```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: nirmata-agent-metrics
  namespace: nirmata
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: nirmata-agent
  endpoints:
    - port: metrics
      path: /metrics
      scheme: http
```

### Access Metrics Directly

```bash
kubectl -n nirmata port-forward svc/service-agent-metrics-service 8443:8443

curl -s http://localhost:8443/metrics | grep '^agent_\|^remediator_'
```

### Example Queries

```promql
# Success rate over the last hour
sum(rate(agent_reconcile_total{controller_type="remediator",result="success"}[1h]))
/ sum(rate(agent_reconcile_total{controller_type="remediator"}[1h]))

# P95 reconciliation latency
histogram_quantile(0.95,
  sum by (le) (rate(agent_reconcile_duration_seconds_bucket{controller_type="remediator"}[1h]))
)

# PRs opened per day, by repository
sum by (repo) (
  increase(agent_gitops_outcome_total{controller_type="remediator",outcome_type="PR_OPENED"}[1d])
)

# Active critical violations
sum(remediator_violations_active{severity="critical"})
```

---

## Remediator Status

The `Remediator` resource reports detailed status about each run.

```bash
# View full status
kubectl get remediator remediator-argo-hub -n nirmata -o yaml

# View just the last run summary
kubectl get remediator remediator-argo-hub -n nirmata \
  -o jsonpath='{.status.lastRunSummary}' | jq
```

### Status Fields

| Field | Description |
|-------|-------------|
| `phase` | Current operational phase: `Running`, `Idle`, or `Failed` |
| `lastScheduleTime` | When the last remediation was scheduled |
| `lastSuccessfulTime` | When the last successful run completed |
| `nextScheduledTime` | When the next run is scheduled |
| `conditions` | Step-by-step workflow tracking with collector information |
| `lastRunSummary.startTime` / `endTime` | Run duration timestamps |
| `lastRunSummary.status` | Success or failure |
| `lastRunSummary.message` | Human-readable outcome |
| `lastRunSummary.targetsProcessed` | Number of targets scanned |
| `lastRunSummary.violationsFound` | Total violations discovered |
| `lastRunSummary.remediationPlans` | Number of remediation plans produced |
| `lastRunSummary.actionsExecuted` | Number of actions taken (PRs created, etc.) |
| `lastRunSummary.errors` | Any errors encountered |

### Example Status Query

```bash
kubectl get remediator remediator-argo-hub -n nirmata \
  -o jsonpath='{.status.lastRunSummary}' | jq '{
  status: .status,
  violations: .violationsFound,
  plans: .remediationPlans,
  actions: .actionsExecuted,
  errors: .errors
}'
```bash

---

## Logs

```bash
# Follow live logs
kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent -f

# Last 100 lines
kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent --tail=100
```yaml

---

## Support Matrix

| Component | Supported |
|-----------|-----------|
| **Kubernetes** | All CNCF-compliant distributions v1.20+, including on-prem |
| **AI providers** | Nirmata AI (default), AWS Bedrock, Azure OpenAI, Anthropic API — or no AI provider at all in `prescriptive` remediation mode |
| **GitOps** | ArgoCD, FluxCD |
| **VCS** | GitHub (App & PAT), GitLab (Enterprise & SaaS) |
| **Manifests** | YAML files, simple Helm charts |


---

## GitHub Authentication Guide


## Overview

Nirmata AI Agents support multiple GitHub authentication methods to integrate with your GitHub repositories. This guide helps you choose the right authentication method and configure it properly.

**Recommended Method**: Use Nirmata's GitHub App for the simplest, most secure integration. Install it once from Nirmata Control Hub, and it works across all GitOps operations.

## Authentication Methods

### 1. Nirmata GitHub App (Recommended)

**Best for**: Production environments and all GitOps operations

The Nirmata GitHub App method provides the simplest and most secure authentication. Simply install Nirmata's GitHub App from the Control Hub UI - no need to create your own GitHub App or manage secrets.

#### Benefits

- ✅ **Zero Secret Management**: No need to manage GitHub secrets in Kubernetes
- ✅ **Centralized Configuration**: All credentials managed in Nirmata Control Hub
- ✅ **Best Security**: Short-lived tokens with fine-grained permissions
- ✅ **Automatic Rotation**: Token refresh handled automatically
- ✅ **Enterprise Ready**: Built for organizational use

#### Prerequisites

1. **Install GitHub App in Nirmata Control Hub**: Follow the [GitHub App Integration guide](/docs/control-hub/settings/integrations/githubapp/) to install Nirmata's GitHub App
   - Navigate to Settings → Integrations in Nirmata Control Hub
   - Click **Connect** on the GitHub card
   - Authorize Nirmata to access your GitHub repositories
2. **Nirmata Control Hub Service Account Token**: Your Nirmata Control Hub Service Account token (`SERVICE_ACCOUNT_TOKEN`) or API token (`API_TOKEN`) must be configured in your cluster

#### Quick Setup

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: nirmata-github-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: nirmata-app  # That's all you need!
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "remediator: Fix policy violations in %s"
        commitMessageTemplate: "Auto-fix: Remediate policy violations in %s"
        systemLabels:
          - "clusterName"
          - "namespace"
        customLabels:
          - "security"
          - "compliance"
```text

#### How It Works

1. You install Nirmata's GitHub App once from the Control Hub UI
2. The agent retrieves GitHub App credentials from Nirmata Control Hub using your Nirmata API token
3. No additional secrets or configuration required in your Kubernetes cluster
4. Works for all GitOps operations including AI agents, policy management, and repository sync

---

### 2. Personal Access Token (PAT)

**Best for**: Development, testing, and proof-of-concept scenarios

Use PAT for quick testing or when GitHub App setup is not feasible.

#### Considerations

- ⚠️ **Long-lived Tokens**: Less secure than GitHub Apps
- ⚠️ **User-bound**: Tied to individual user accounts
- ⚠️ **Broader Permissions**: Less granular permission control
- ✅ **Simple Setup**: Quick to configure for testing

#### Quick Setup

**Step 1: Create PAT Secret**

```bash
kubectl create secret generic github-pat-token \
  --from-literal=token=YOUR_GITHUB_PAT \
  --namespace nirmata
```text

**Step 2: Create ToolConfig**

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: github-pat-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: github-pat-token
        namespace: nirmata
        key: token
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations"
        commitMessageTemplate: "Auto-fix: Remediate policy violations"
```text

---

### 3. Custom GitHub App

**Best for**: Organizations that must own the GitHub App themselves, or that run the agent without Nirmata Control Hub credentials

Use your own GitHub App when GitHub App security is required but installing Nirmata's App is not an option — for example a policy that all Apps be owned by your own organization, or an agent installed with `nirmata.auth=none`. You keep GitHub App benefits (short-lived tokens, fine-grained repository permissions) but manage the private key yourself.

{{% alert title="github.com only" %}}
All three authentication methods target `github.com`. The agent has no setting for a GitHub Enterprise Server API endpoint, so a custom GitHub App must be installed on `github.com` — it cannot authenticate against a self-hosted GHES instance.
{{% /alert %}}

#### Considerations

- ✅ **Short-lived Tokens**: Installation access tokens, same as the Nirmata App
- ✅ **Fine-grained Permissions**: Scoped per repository, not per user
- ✅ **No Control Hub Dependency**: Works with `nirmata.auth=none`
- ⚠️ **You Manage the Private Key**: Stored as a Kubernetes Secret and rotated by you
- ⚠️ **Requires `github.com` Reachability**: The agent mints tokens against the GitHub API

#### Prerequisites

1. Create a GitHub App in your organization and generate a private key (`.pem`).
2. Grant it repository permissions covering what the agent does: read and write **Contents** (commit the fix) and **Pull requests** (open and comment on PRs), read **Metadata**, and write **Checks** so the agent can report its [CI checks](../configuration/#ci-checks). Add write **Issues** only if you configure the `CreateIssue` action.
3. Install the App on the repositories the agent should remediate.
4. Note the **App ID** from the App's settings page.

#### Quick Setup

**Step 1: Create the private key Secret**

```bash
kubectl create secret generic github-app-private-key \
  --from-file=private-key.pem=/path/to/your-app.private-key.pem \
  --namespace nirmata
```

**Step 2: Create ToolConfig**

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: github-app-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: app
    app:
      appId: 123456
      # installationId: 78901234   # optional — auto-discovered when omitted
      privateKeySecretRef:
        name: github-app-private-key
        key: private-key.pem
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        systemLabels:
          - "clusterName"
          - "namespace"
```

`appId` is an integer, not a string. Omit `installationId` and the agent discovers it per repository at run time — it reads the installation for the target repo and caches the result; set it explicitly to pin one installation and skip that lookup. The private key may be PKCS#1 or PKCS#8 PEM; it must be RSA.

{{% alert title="Secret namespace" %}}
The private key Secret is read from the namespace of the **ToolConfig**, so it must live alongside it — `nirmata` in these examples. The same applies to PAT secrets. A Secret in another namespace produces a "failed to get private key Secret" error even if `privateKeySecretRef.namespace` names it.
{{% /alert %}}

**Via Helm values**

```yaml
tool:
  credentials:
    method: app
    app:
      appId: 123456
      privateKeySecret:
        name: github-app-private-key
        key: private-key.pem
```

---

## Comparison Matrix

| Feature | Nirmata GitHub App | Custom GitHub App | Personal Access Token |
|---------|-------------------|-------------------|-----------------------|
| **Security** | ✅ Excellent | ✅ Excellent | ⚠️ Moderate |
| **Token Lifetime** | 1 hour (auto-refresh) | 1 hour (auto-refresh) | Long-lived (requires manual rotation) |
| **Setup Complexity** | ✅ Simplest (one-click install) | ⚠️ Highest (create App, manage key) | ✅ Simple |
| **Secret Management** | ✅ Zero (managed by Nirmata Control Hub) | ⚠️ Manual (private key in a K8s secret) | ⚠️ Manual (K8s secrets) |
| **Permission Control** | ✅ Fine-grained | ✅ Fine-grained | ⚠️ Broad (user-level) |
| **Multi-Org Support** | ✅ Yes (install per org) | ✅ Yes (install per org) | ⚠️ User-bound |
| **Audit Trail** | ✅ Excellent (GitHub + Nirmata Control Hub) | ✅ Good (GitHub) | ⚠️ User-level only |
| **Enterprise Ready** | ✅ Yes | ✅ Yes | ⚠️ Limited |
| **Credential Rotation** | ✅ Automatic | ⚠️ Manual (private key) | ⚠️ Manual |
| **Needs Control Hub credentials** | ✅ Yes | ❌ No | ❌ No |
| **Works for All GitOps** | ✅ Yes | ⚠️ Agent only | ⚠️ Requires separate PATs |
| **Best Use Case** | Production & all environments | You must own the App, or run without Control Hub | Quick testing/POC only |

---

## Decision Guide

### Choose Nirmata GitHub App When:

- ✅ You're using Nirmata Control Hub (highly recommended)
- ✅ You want the simplest, most secure configuration
- ✅ You need enterprise-grade security without overhead
- ✅ You want zero secret management in Kubernetes
- ✅ You're deploying to production
- ✅ You want one integration for all GitOps operations
- ✅ You need automated credential rotation

**Recommendation**: Use Nirmata GitHub App for all environments (production, staging, development). It's designed specifically for GitOps workflows and requires minimal configuration.

### Choose a Custom GitHub App When:

- ✅ Your organization requires that all GitHub Apps be owned by your org
- ✅ You run the agent with `nirmata.auth=none` and still want App-grade security
- ✅ You want fine-grained, per-repository permissions without Control Hub
- ⚠️ You accept responsibility for storing and rotating the App private key
- ⚠️ Your repositories are on `github.com`, not a self-hosted GHES instance

**Recommendation**: Prefer the Nirmata GitHub App unless one of the above applies — it removes private key management entirely.

### Choose Personal Access Token When:

- ⚠️ You're doing quick testing or proof-of-concept
- ⚠️ You have simple, single-repository scenarios
- ⚠️ You need to get started immediately without Nirmata Control Hub setup
- ⚠️ Your organization explicitly allows PAT usage for automation

**Note**: PATs are not recommended for production use due to security limitations and manual credential management requirements.

---

## Complete Configuration Example

### Step 1: Install GitHub App (One-Time Setup)

1. Navigate to Nirmata Control Hub → Settings → Integrations
2. Click **Connect** on the GitHub card
3. Authorize Nirmata to access your repositories
4. Done! The GitHub App is now available for all GitOps operations

### Step 2: Create ToolConfig for AI Agents

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: nirmata-github-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: nirmata-app  # Uses GitHub App from Nirmata Control Hub
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Fix] {{.PolicyName}}"
        commitMessageTemplate: "Remediate: {{.PolicyName}} violations"
        systemLabels:
          - "clusterName"
          - "namespace"
        customLabels:
          - "automated"
          - "security"
```text

### Step 3: Use with Remediator Agent

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: Remediator
metadata:
  name: remediator-prod
  namespace: nirmata
spec:
  environment:
    type: argoHub
  
  target:
    argoHub:
      appSelector:
        allApps: true
  
  remediation:
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    gitCredentials:
      name: nirmata-github-tool  # References Nirmata GitHub App
      namespace: nirmata
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    eventPolling:
      enabled: true
      intervalMinutes: 5
    actions:
      - type: CreatePR
        toolRef:
          name: nirmata-github-tool
          namespace: nirmata
```text

### Complete Workflow

1. ✅ **Install GitHub App** in Nirmata Control Hub (one-time, works for all GitOps operations)
2. ✅ **Create ToolConfig** with `method: nirmata-app`
3. ✅ **Reference in Remediator** - agent automatically uses GitHub App credentials
4. ✅ **Zero secret management** - no GitHub secrets in Kubernetes
5. ✅ **Automatic token rotation** - handled by Nirmata Control Hub
6. ✅ **Audit trail** - tracked in both GitHub and Nirmata Control Hub

---

## Pull Request Configuration

### Understanding Labels

ToolConfig supports two types of PR labels:

#### System Labels (Dynamic)

Computed at runtime based on remediation context:
- `branch`: Git branch being remediated
- `clusterName`: Cluster where violations were found
- `appName`: ArgoCD application name (if applicable)
- `namespace`: Kubernetes namespace with violations

**Note**: System labels are only applied when data is available in the remediation context.

#### Custom Labels (Static)

Always added to every PR:
- `auto-remediation`
- `security`
- `compliance`
- Any custom labels you define

### Example with All Label Types

```yaml
defaults:
  git:
    pullRequests:
      branchPrefix: "fix/"
      titleTemplate: "[Auto-Fix] {{.PolicyName}}"
      commitMessageTemplate: "Remediate: {{.PolicyName}} violations"
      systemLabels:
        - "clusterName"
        - "namespace"
        - "appName"
      customLabels:
        - "automated"
        - "kyverno-policy"
        - "needs-review"
```text

---

## Best Practices

### Security

1. **Use Nirmata GitHub App** for production environments with Nirmata Control Hub
2. **Rotate credentials regularly** regardless of method
3. **Limit repository access** to only what's needed
4. **Use separate credentials** for different environments
5. **Monitor audit logs** for unexpected activity

### Configuration

1. **Use descriptive names** for ToolConfig resources
2. **Namespace isolation** - keep configs in appropriate namespaces
3. **Document your setup** - add comments explaining configuration choices
4. **Test in development** before deploying to production

### Maintenance

1. **Review permissions quarterly** - ensure they're still appropriate
2. **Update private keys** on a schedule (every 90 days recommended)
3. **Monitor token usage** through GitHub audit logs
4. **Keep documentation updated** when changing configurations

---

## Troubleshooting

### GitHub App Not Connected

**Symptom**: ToolConfig fails with "GitHub App not configured"

**Solutions**:
1. Verify GitHub App is installed in Nirmata Control Hub:
   - Go to Settings → Integrations
   - Check if GitHub shows "Connected" status
2. If not connected, click **Connect** and complete the authorization
3. Ensure you selected the repositories where you want Nirmata to work

### Authentication Failures

**Symptom**: Agent cannot authenticate with GitHub

**Common Causes & Solutions**:

**For Nirmata GitHub App**:
1. Verify GitHub App is connected in Nirmata Control Hub (Settings → Integrations)
2. Check your Nirmata Control Hub Service Account token (`SERVICE_ACCOUNT_TOKEN`) or API token (`API_TOKEN`) is valid:
   ```bash
   kubectl get secret nirmata-service-account-token -n nirmata -o yaml
   ```
3. Ensure the secret contains a valid, non-expired Nirmata Control Hub Service Account token
4. Verify network connectivity from cluster to Nirmata Control Hub
5. Check that the GitHub App has access to the target repository

**For PAT**:
1. Check token hasn't expired
2. Verify token has required scopes (`repo`, `write:discussion`)
3. Ensure token secret exists in correct namespace:
   ```bash
   kubectl get secret github-pat-token -n nirmata
   ```

### Cannot Create Pull Requests

**Symptom**: Authentication succeeds but PRs aren't created

**Solutions**:
1. Verify Nirmata GitHub App is installed on the target repository:
   - Go to GitHub → Settings → Applications → Installed GitHub Apps
   - Find "Nirmata" and check repository access
2. Ensure the repository is not archived or read-only
3. Check branch protection rules don't block the app
4. Review agent logs for detailed error messages:
   ```bash
   kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent --tail=100
   ```

### Missing Repository Access

**Symptom**: Nirmata cannot access a specific repository

**Solutions**:
1. In GitHub, go to Settings → Applications → Installed GitHub Apps
2. Click **Configure** next to Nirmata
3. Add the repository or select "All repositories"
4. Click **Save**
5. Wait a few minutes for permissions to propagate

### ToolConfig Method Not Working

**Symptom**: Error like "unsupported method: nirmata-app"

**Solutions**:
1. Verify you're using the latest Remediator Agent version
2. Check that your Helm chart is up to date
3. Ensure the agent has network access to Nirmata Control Hub
4. Review pod logs for version/compatibility information

---

## Related Documentation

- [GitHub App Integration in Nirmata Control Hub](/docs/control-hub/settings/integrations/githubapp/)
- [Remediator Agent Overview](/docs/control-hub/agent-hub/service-agents/)
- [ToolConfig Configuration](/docs/control-hub/agent-hub/service-agents/configuration/#toolconfig)
- [GitHub Apps Documentation](https://docs.github.com/en/developers/apps)

---

## Support

For additional help:

- **Nirmata Support**: support@nirmata.com
- **Documentation**: https://docs.nirmata.io
- **GitHub Apps**: https://docs.github.com/en/developers/apps