---
title: "Service Agents"
description: "Autonomous AI agents deployed inside your Kubernetes clusters for 24/7 policy violation detection and remediation."
diataxis: how-to
applies_to:
  product: "nirmata-ai-agents"
audience: ["platform-engineer"]
last_updated: 2026-04-16
url: https://docs.nirmata.io/docs/control-hub/agent-hub/service-agents/
---


> **Applies to:** Nirmata AI Agents 1.0 and later

Service Agents are autonomous AI agents that run inside your Kubernetes clusters. Unlike [Cloud Agents](../cloud-agents/), which are launched on-demand from the Control Hub, Service Agents are deployed directly into the cluster and operate continuously — detecting policy violations, generating remediation plans, and opening pull requests in your Git repositories without human intervention.

The primary Service Agent is the **Remediator Agent**: it monitors Kyverno policy reports, uses AI to generate compliant fixes, and integrates with your GitOps workflow by creating PRs against the affected repository.

For a conceptual overview of Service Agents, see [Service Agents in Nirmata AI Agents](/docs/ai/service-agents/).

## In This Section

- [**Getting Started**]({{< relref "getting-started/" >}}) — Install the Remediator Agent and verify your first remediation
- [**Configuration**]({{< relref "configuration/" >}}) — Configure ToolConfig (Git credentials), LLMConfig (AI provider), and the Remediator resource
- [**Observability**]({{< relref "observability/" >}}) — Prometheus metrics, status fields, and monitoring the agent in production
- [**GitHub Authentication**]({{< relref "github-authentication.md" >}}) — Connect to GitHub using the Nirmata GitHub App

## How It Works

1. The Remediator Agent is installed in the `nirmata` namespace via Helm.
2. It reads Kyverno `ClusterPolicyReport` resources to discover violations.
3. For each violation, it calls a configured AI provider to generate a fix.
4. It opens a pull request in the target Git repository with the proposed change.
5. You review and merge the PR — the agent never pushes directly to a protected branch.

## Key Capabilities

| Capability | Description |
|------------|-------------|
| **Continuous monitoring** | Runs on a cron schedule or triggered on-demand via the Kubernetes API |
| **Multi-cluster support** | Hub Mode uses ArgoCD to manage violations across hundreds of clusters |
| **GitOps integration** | All changes go through PRs — no direct cluster mutations |
| **Confidence-based actions** | Configure whether PRs are opened for high-confidence fixes, low-confidence, or both |
| **Split PR** | Split a multi-policy PR into separate PRs for independent review workflows |
| **AI provider choice** | Nirmata AI (default), AWS Bedrock, or Azure OpenAI |


---

## Getting Started


## Prerequisites

### Required Components

- **Kubernetes cluster** running Kubernetes 1.20+
- **Helm** 3.x installed and configured
- **kubectl** configured to access your cluster
- **Kyverno** installed and running with policy reports enabled
- **ArgoCD** (optional) — required for Hub Mode multi-cluster setups

### Required Credentials

- **Nirmata Control Hub Service Account Token** — create a Service Account in Nirmata Control Hub and copy its secret

### Create a Service Account

1. Log in to [Nirmata Control Hub](https://nirmata.io)
2. Navigate to **Identity & Access** from the left sidebar
3. Go to **Service Accounts** and create a new one
4. Copy the generated secret — you'll use it in the next step

---

## Installation

### 1. Create Namespace and Secrets

```bash
kubectl create namespace nirmata

kubectl create secret generic nirmata-service-account-token \
  --from-literal=service-account-token=YOUR_NCH_SERVICE_ACCOUNT_TOKEN \
  --namespace nirmata
```

### 2. Add the Helm Repository

```bash
helm repo add nirmata https://nirmata.github.io/kyverno-charts
helm repo update nirmata
```

### 3. Install the Agent

```bash
helm install nirmata-agent nirmata/nirmata-agent --devel \
  --namespace nirmata \
  --create-namespace
```json

{{% alert title="Note" %}}
The chart uses `serviceAccountToken` authentication by default and expects the `nirmata-service-account-token` secret created above. To use API token authentication instead, set:
```
--set nirmata.auth="apiToken" \
--set nirmata.apiTokenSecret="nirmata-api-token"
```json
{{% /alert %}}

### 4. Verify the Installation

```bash
# Check pods are running
kubectl get pods -n nirmata -l app.kubernetes.io/name=nirmata-agent

# Check custom resource definitions were installed
kubectl get llmconfigs,toolconfigs,remediators -n nirmata

# Tail the agent logs
kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent --tail=50
```yaml

---

## Quick Configuration

The agent requires three custom resources before it will start remediating. See [Configuration](../configuration/) for full details and all available options.

### 1. ToolConfig (Git credentials)

**GitHub Personal Access Token:**

```bash
kubectl create secret generic github-pat-token \
  --from-literal=token=GITHUB_PAT_TOKEN \
  --namespace nirmata

kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: github
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: github-pat-token
        namespace: nirmata
        key: token
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations: "
        commitMessageTemplate: "Auto-fix: Remediate policy violations: "
        customLabels:
          - "auto-remediation"
        systemLabels:
          - "clusterName"
          - "namespace"
EOF
```

Prefer using the [Nirmata GitHub App](../github-authentication/) over a personal access token — it avoids managing secrets manually and provides automatic token rotation.

### 2. LLMConfig (AI provider)

The Helm chart creates a default LLMConfig using Nirmata AI automatically. If you need to create it manually:

```bash
kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: nirmataAI
  nirmataAI:
    model: ""
EOF
```

### 3. Remediator

**Local Mode** (scans the cluster where the agent is installed):

```bash
# Create a ConfigMap mapping Git repos to namespaces
kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: repo-namespace-mapping
  namespace: nirmata
data:
  mapping: |
    [
      {
        "repo": "https://github.com/your-org/your-repo",
        "branch": "main",
        "path": "k8s/",
        "targetNamespace": "default"
      }
    ]
EOF

# Create the Remediator
kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: Remediator
metadata:
  name: remediator-local
  namespace: nirmata
spec:
  environment:
    type: localCluster
  target:
    localCluster:
      repoNamespaceMappingRef:
        name: repo-namespace-mapping
        namespace: nirmata
        key: mapping
  remediation:
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    gitCredentials:
      name: toolconfig-sample
      namespace: nirmata
    eventPolling:
      enabled: true
      intervalMinutes: 5
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
EOF
```yaml

---

## Uninstalling

```bash
helm uninstall nirmata-agent -n nirmata
```

Secrets you created manually are not removed — clean those up separately if needed.


---

## Configuration


The Remediator Agent is configured through three Kubernetes custom resources:

| Resource | Purpose |
|----------|---------|
| **ToolConfig** | Git provider credentials and PR defaults |
| **LLMConfig** | AI provider settings (Nirmata AI, AWS Bedrock, Azure OpenAI) |
| **Remediator** | What to scan, when to run, and what actions to take |

---

## ToolConfig

ToolConfig defines how the agent authenticates with your Git provider and sets defaults for the pull requests it creates.

### GitHub — Personal Access Token

```bash
kubectl create secret generic github-pat-token \
  --from-literal=token=GITHUB_PAT_TOKEN \
  --namespace nirmata

kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: github
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: github-pat-token
        namespace: nirmata
        key: token
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations: "
        commitMessageTemplate: "Auto-fix: Remediate policy violations: "
        customLabels:
          - "auto-remediation"
          - "security"
        systemLabels:
          - "clusterName"
          - "namespace"
EOF
```text

### GitHub — Nirmata GitHub App (Recommended)

Using the Nirmata GitHub App avoids managing secrets manually and provides automatic token rotation. Follow the [GitHub Authentication guide](../github-authentication/) to set up the App integration first.

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: github
  credentials:
    method: nirmata-app
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations: "
        commitMessageTemplate: "Auto-fix: Remediate policy violations: "
        customLabels:
          - "auto-remediation"
        systemLabels:
          - "clusterName"
          - "namespace"
```text

### GitLab

```bash
kubectl create secret generic gitlab-pat-token \
  --from-literal=token=GITLAB_PAT_TOKEN \
  --namespace=nirmata

kubectl apply -f - <<EOF
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: toolconfig-sample
  namespace: nirmata
spec:
  type: gitlab
  credentials:
    method: pat
    pat:
      secretRef:
        name: gitlab-pat-token
        namespace: nirmata
        key: token
EOF
```text

### Pull Request Labels

ToolConfig supports two label types:

- **`customLabels`** — static labels always applied to every PR (e.g., `auto-remediation`, `security`)
- **`systemLabels`** — dynamic labels computed at runtime from remediation context:

| System Label | Value |
|---|---|
| `branch` | The Git branch being remediated |
| `clusterName` | The cluster where violations were found |
| `appName` | ArgoCD application name (Hub Mode only) |
| `namespace` | Kubernetes namespace containing violations |

---

## LLMConfig

LLMConfig specifies the AI provider used to generate remediation plans.

### Nirmata AI (Default)

The Helm chart creates this automatically. Authentication uses the Service Account token injected at deployment time — no additional secrets needed.

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: nirmataAI
  nirmataAI:
    model: ""   # empty string uses the current default model
```text

### AWS Bedrock

For EKS clusters with Pod Identity Agent:

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: bedrock
  bedrock:
    model: MODEL_ARN_OR_INFERENCE_ARN
    region: AWS_REGION
```text

<details>
<summary>Full AWS IAM setup</summary>

```bash
aws iam create-role \
  --role-name remediator-agent-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }]
  }'

aws iam put-role-policy \
  --role-name remediator-agent-role \
  --policy-name BedrockInvokePolicy \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": "arn:aws:bedrock:<REGION>:<ACCOUNT_ID>:application-inference-profile/<PROFILE>"
    }]
  }'

aws eks create-pod-identity-association \
  --cluster-name <CLUSTER_NAME> \
  --namespace nirmata \
  --service-account nirmata-agent \
  --role-arn arn:aws:iam::<ACCOUNT_ID>:role/remediator-agent-role
```text
</details>

### Azure OpenAI

```bash
kubectl create secret generic azure-openai-credentials \
  --from-literal=api-key=AZURE_API_KEY \
  -n nirmata
```

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: LLMConfig
metadata:
  name: nirmata-agent-llm
  namespace: nirmata
spec:
  type: azure-openai
  azureOpenAI:
    endpoint: https://YOUR_RESOURCE_NAME.openai.azure.com/
    deploymentName: DEPLOYMENT_NAME
    apiKeySecretRef:
      name: azure-openai-credentials
      key: api-key
      namespace: nirmata
```yaml

---

## Remediator

The Remediator resource ties everything together: it defines the environment mode, which clusters or applications to target, the schedule, and what actions to take.

### Environment Modes

| Mode | Use Case |
|------|----------|
| `localCluster` | Scan the cluster where the agent is installed |
| `argoHub` | Use ArgoCD to manage violations across multiple clusters |

### ArgoCD Hub Mode (Multi-Cluster)

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: Remediator
metadata:
  name: remediator-argo-hub
  namespace: nirmata
spec:
  environment:
    type: argoHub
  target:
    argoHubTarget:
      argoAppSelector:
        allApps: true    # or specify names / labelSelector
  remediation:
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    gitCredentials:
      name: toolconfig-sample
      namespace: nirmata
    eventPolling:
      enabled: true
      intervalMinutes: 5
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
```

**Targeting specific clusters or apps:**

```yaml
target:
  argoHubTarget:
    clusterNames:
      - production-cluster
      - staging-cluster
    argoAppSelector:
      names:
        - nginx-demo
        - web-app
      labelSelector:
        matchLabels:
          team: platform
          environment: production
```

### Local Cluster Mode

See the example in [Getting Started](../getting-started/).

### VCS Target Mode (Direct Repository Scanning)

Scan Git repositories directly without requiring a running cluster:

```yaml
spec:
  environment:
    type: localCluster
  target:
    vcs:
      policies:
        - name: pod-security-policy
          repo: https://github.com/your-org/policies
          path: kyverno/pod-security.yaml
          ref: main
      resources:
        - name: web-app-deployment
          repo: https://github.com/your-org/manifests
          path: deployments/web-app.yaml
          ref: main
          policyRefs:
            - pod-security-policy
  remediation:
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    actions:
      - type: CreatePR
        toolRef:
          name: toolconfig-sample
          namespace: nirmata
```

### Confidence-Based Actions

Control when automated PRs are created based on the AI's confidence in its fix:

```yaml
remediation:
  actions:
    - type: CreatePR
      confidence:
        - high     # only create PRs when AI is highly confident
      toolRef:
        name: toolconfig-sample
        namespace: nirmata
```

| Confidence | Meaning |
|-----------|---------|
| `high` | Nirmata AI is highly confident the fix is correct and safe |
| `low` | A potential fix was found but human review is recommended |

### Filtering by Severity

```yaml
remediation:
  filters:
    policySelector:
      matchSeverity:
        - high
        - critical
```yaml

---

## Split PR

The Split PR feature lets you split a pull request containing fixes for multiple policies into separate PRs — useful when different policies need different reviewers or approval workflows.

### How to Split

Add a comment to the PR mentioning `@nirmatabot`:

```text
@nirmatabot split-pr require-run-as-non-root disallow-privileged-containers
```

- One or more policy names (space-separated) after `split-pr`
- Use the exact policy names as they appear in the PR

### What Happens

- **Original PR** — updated to contain only the remaining policies; a comment is added with a link to the new PR
- **New PR** — created on a new branch (prefixed `splitpr-`) with only the specified policies; links back to the original PR
- Both PRs are tracked independently by the agent


---

## Observability


## Prometheus Metrics

The Remediator Agent exposes Prometheus metrics at the controller manager's metrics endpoint.

### Available Metrics

| Metric | Type | Labels | Description |
|--------|------|--------|-------------|
| `remediator_reconciles_total` | Counter | `result="success\|error"` | Total number of reconciliation runs |
| `remediator_reconcile_duration_seconds` | Histogram | `result="success\|error"` | Duration of each reconciliation run |

### Enable ServiceMonitor

```yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: go-agent-remediator-metrics
  namespace: go-agent-remediator-system
spec:
  selector:
    matchLabels:
      control-plane: controller-manager
  endpoints:
    - port: https
      path: /metrics
      scheme: https
      tlsConfig:
        insecureSkipVerify: true
```

### Access Metrics Directly

```bash
kubectl -n go-agent-remediator-system port-forward \
  deploy/go-agent-remediator-controller-manager 8443:8443

SA=go-agent-remediator-controller-manager
NS=go-agent-remediator-system
TOKEN=$(kubectl -n $NS create token $SA)
curl -k -H "Authorization: Bearer $TOKEN" https://localhost:8443/metrics
```

### Example Queries

```promql
# Success rate over the last hour
sum(rate(remediator_reconciles_total{result="success"}[1h]))
/ sum(rate(remediator_reconciles_total[1h]))

# P95 reconciliation latency
histogram_quantile(0.95,
  sum by (le) (rate(remediator_reconcile_duration_seconds_bucket[1h]))
)
```yaml

---

## Remediator Status

The `Remediator` resource reports detailed status about each run.

```bash
# View full status
kubectl get remediator remediator-argo-hub -n nirmata -o yaml

# View just the last run summary
kubectl get remediator remediator-argo-hub -n nirmata \
  -o jsonpath='{.status.lastRunSummary}' | jq
```

### Status Fields

| Field | Description |
|-------|-------------|
| `phase` | Current operational phase: `Running`, `Idle`, or `Failed` |
| `lastScheduleTime` | When the last remediation was scheduled |
| `lastSuccessfulTime` | When the last successful run completed |
| `nextScheduledTime` | When the next run is scheduled |
| `conditions` | Step-by-step workflow tracking with collector information |
| `lastRunSummary.startTime` / `endTime` | Run duration timestamps |
| `lastRunSummary.status` | Success or failure |
| `lastRunSummary.message` | Human-readable outcome |
| `lastRunSummary.targetsProcessed` | Number of targets scanned |
| `lastRunSummary.violationsFound` | Total violations discovered |
| `lastRunSummary.remediationPlans` | Number of AI-generated plans produced |
| `lastRunSummary.actionsExecuted` | Number of actions taken (PRs created, etc.) |
| `lastRunSummary.errors` | Any errors encountered |

### Example Status Query

```bash
kubectl get remediator remediator-argo-hub -n nirmata \
  -o jsonpath='{.status.lastRunSummary}' | jq '{
  status: .status,
  violations: .violationsFound,
  plans: .remediationPlans,
  actions: .actionsExecuted,
  errors: .errors
}'
```bash

---

## Logs

```bash
# Follow live logs
kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent -f

# Last 100 lines
kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent --tail=100
```yaml

---

## Support Matrix

| Component | Supported |
|-----------|-----------|
| **Kubernetes** | All CNCF-compliant distributions v1.20+, including on-prem |
| **AI providers** | Nirmata AI (default), AWS Bedrock, Azure OpenAI |
| **GitOps** | ArgoCD |
| **VCS** | GitHub (App & PAT), GitLab (Enterprise & SaaS) |
| **Manifests** | YAML files, simple Helm charts |


---

## GitHub Authentication Guide


## Overview

Nirmata AI Agents support multiple GitHub authentication methods to integrate with your GitHub repositories. This guide helps you choose the right authentication method and configure it properly.

**Recommended Method**: Use Nirmata's GitHub App for the simplest, most secure integration. Install it once from Nirmata Control Hub, and it works across all GitOps operations.

## Authentication Methods

### 1. Nirmata GitHub App (Recommended)

**Best for**: Production environments and all GitOps operations

The Nirmata GitHub App method provides the simplest and most secure authentication. Simply install Nirmata's GitHub App from the Control Hub UI - no need to create your own GitHub App or manage secrets.

#### Benefits

- ✅ **Zero Secret Management**: No need to manage GitHub secrets in Kubernetes
- ✅ **Centralized Configuration**: All credentials managed in Nirmata Control Hub
- ✅ **Best Security**: Short-lived tokens with fine-grained permissions
- ✅ **Automatic Rotation**: Token refresh handled automatically
- ✅ **Enterprise Ready**: Built for organizational use

#### Prerequisites

1. **Install GitHub App in Nirmata Control Hub**: Follow the [GitHub App Integration guide](/docs/control-hub/settings/integrations/githubapp/) to install Nirmata's GitHub App
   - Navigate to Settings → Integrations in Nirmata Control Hub
   - Click **Connect** on the GitHub card
   - Authorize Nirmata to access your GitHub repositories
2. **Nirmata Control Hub Service Account Token**: Your Nirmata Control Hub Service Account token (`SERVICE_ACCOUNT_TOKEN`) or API token (`API_TOKEN`) must be configured in your cluster

#### Quick Setup

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: nirmata-github-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: nirmata-app  # That's all you need!
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "remediator: Fix policy violations in %s"
        commitMessageTemplate: "Auto-fix: Remediate policy violations in %s"
        systemLabels:
          - "clusterName"
          - "namespace"
        customLabels:
          - "security"
          - "compliance"
```text

#### How It Works

1. You install Nirmata's GitHub App once from the Control Hub UI
2. The agent retrieves GitHub App credentials from Nirmata Control Hub using your Nirmata API token
3. No additional secrets or configuration required in your Kubernetes cluster
4. Works for all GitOps operations including AI agents, policy management, and repository sync

---

### 2. Personal Access Token (PAT)

**Best for**: Development, testing, and proof-of-concept scenarios

Use PAT for quick testing or when GitHub App setup is not feasible.

#### Considerations

- ⚠️ **Long-lived Tokens**: Less secure than GitHub Apps
- ⚠️ **User-bound**: Tied to individual user accounts
- ⚠️ **Broader Permissions**: Less granular permission control
- ✅ **Simple Setup**: Quick to configure for testing

#### Quick Setup

**Step 1: Create PAT Secret**

```bash
kubectl create secret generic github-pat-token \
  --from-literal=token=YOUR_GITHUB_PAT \
  --namespace nirmata
```text

**Step 2: Create ToolConfig**

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: github-pat-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: pat
    pat:
      tokenSecretRef:
        name: github-pat-token
        namespace: nirmata
        key: token
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Remediation] Fix policy violations"
        commitMessageTemplate: "Auto-fix: Remediate policy violations"
```text

---

## Comparison Matrix

| Feature | Nirmata GitHub App | Personal Access Token |
|---------|-------------------|-----------------------|
| **Security** | ✅ Excellent | ⚠️ Moderate |
| **Token Lifetime** | 1 hour (auto-refresh) | Long-lived (requires manual rotation) |
| **Setup Complexity** | ✅ Simplest (one-click install) | ✅ Simple |
| **Secret Management** | ✅ Zero (managed by Nirmata Control Hub) | ⚠️ Manual (K8s secrets) |
| **Permission Control** | ✅ Fine-grained | ⚠️ Broad (user-level) |
| **Multi-Org Support** | ✅ Yes (install per org) | ⚠️ User-bound |
| **Audit Trail** | ✅ Excellent (GitHub + Nirmata Control Hub) | ⚠️ User-level only |
| **Enterprise Ready** | ✅ Yes | ⚠️ Limited |
| **Credential Rotation** | ✅ Automatic | ⚠️ Manual |
| **Works for All GitOps** | ✅ Yes | ⚠️ Requires separate PATs |
| **Best Use Case** | Production & all environments | Quick testing/POC only |

---

## Decision Guide

### Choose Nirmata GitHub App When:

- ✅ You're using Nirmata Control Hub (highly recommended)
- ✅ You want the simplest, most secure configuration
- ✅ You need enterprise-grade security without overhead
- ✅ You want zero secret management in Kubernetes
- ✅ You're deploying to production
- ✅ You want one integration for all GitOps operations
- ✅ You need automated credential rotation

**Recommendation**: Use Nirmata GitHub App for all environments (production, staging, development). It's designed specifically for GitOps workflows and requires minimal configuration.

### Choose Personal Access Token When:

- ⚠️ You're doing quick testing or proof-of-concept
- ⚠️ You have simple, single-repository scenarios
- ⚠️ You need to get started immediately without Nirmata Control Hub setup
- ⚠️ Your organization explicitly allows PAT usage for automation

**Note**: PATs are not recommended for production use due to security limitations and manual credential management requirements.

---

## Complete Configuration Example

### Step 1: Install GitHub App (One-Time Setup)

1. Navigate to Nirmata Control Hub → Settings → Integrations
2. Click **Connect** on the GitHub card
3. Authorize Nirmata to access your repositories
4. Done! The GitHub App is now available for all GitOps operations

### Step 2: Create ToolConfig for AI Agents

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: ToolConfig
metadata:
  name: nirmata-github-tool
  namespace: nirmata
spec:
  type: github
  credentials:
    method: nirmata-app  # Uses GitHub App from Nirmata Control Hub
  defaults:
    git:
      pullRequests:
        branchPrefix: "remediation-"
        titleTemplate: "[Auto-Fix] {{.PolicyName}}"
        commitMessageTemplate: "Remediate: {{.PolicyName}} violations"
        systemLabels:
          - "clusterName"
          - "namespace"
        customLabels:
          - "automated"
          - "security"
```text

### Step 3: Use with Remediator Agent

```yaml
apiVersion: serviceagents.nirmata.io/v1alpha1
kind: Remediator
metadata:
  name: remediator-prod
  namespace: nirmata
spec:
  environment:
    type: argoHub
  
  target:
    argoHubTarget:
      argoAppSelector:
        allApps: true
  
  remediation:
    llmConfigRef:
      name: nirmata-agent-llm
      namespace: nirmata
    gitCredentials:
      name: nirmata-github-tool  # References Nirmata GitHub App
      namespace: nirmata
    triggers:
      - schedule:
          crontab: "0 */6 * * *"
    eventPolling:
      enabled: true
      intervalMinutes: 5
    actions:
      - type: CreatePR
        toolRef:
          name: nirmata-github-tool
          namespace: nirmata
```text

### Complete Workflow

1. ✅ **Install GitHub App** in Nirmata Control Hub (one-time, works for all GitOps operations)
2. ✅ **Create ToolConfig** with `method: nirmata-app`
3. ✅ **Reference in Remediator** - agent automatically uses GitHub App credentials
4. ✅ **Zero secret management** - no GitHub secrets in Kubernetes
5. ✅ **Automatic token rotation** - handled by Nirmata Control Hub
6. ✅ **Audit trail** - tracked in both GitHub and Nirmata Control Hub

---

## Pull Request Configuration

### Understanding Labels

ToolConfig supports two types of PR labels:

#### System Labels (Dynamic)

Computed at runtime based on remediation context:
- `branch`: Git branch being remediated
- `clusterName`: Cluster where violations were found
- `appName`: ArgoCD application name (if applicable)
- `namespace`: Kubernetes namespace with violations

**Note**: System labels are only applied when data is available in the remediation context.

#### Custom Labels (Static)

Always added to every PR:
- `auto-remediation`
- `security`
- `compliance`
- Any custom labels you define

### Example with All Label Types

```yaml
defaults:
  git:
    pullRequests:
      branchPrefix: "fix/"
      titleTemplate: "[Auto-Fix] {{.PolicyName}}"
      commitMessageTemplate: "Remediate: {{.PolicyName}} violations"
      systemLabels:
        - "clusterName"
        - "namespace"
        - "appName"
      customLabels:
        - "automated"
        - "kyverno-policy"
        - "needs-review"
```text

---

## Best Practices

### Security

1. **Use Nirmata GitHub App** for production environments with Nirmata Control Hub
2. **Rotate credentials regularly** regardless of method
3. **Limit repository access** to only what's needed
4. **Use separate credentials** for different environments
5. **Monitor audit logs** for unexpected activity

### Configuration

1. **Use descriptive names** for ToolConfig resources
2. **Namespace isolation** - keep configs in appropriate namespaces
3. **Document your setup** - add comments explaining configuration choices
4. **Test in development** before deploying to production

### Maintenance

1. **Review permissions quarterly** - ensure they're still appropriate
2. **Update private keys** on a schedule (every 90 days recommended)
3. **Monitor token usage** through GitHub audit logs
4. **Keep documentation updated** when changing configurations

---

## Troubleshooting

### GitHub App Not Connected

**Symptom**: ToolConfig fails with "GitHub App not configured"

**Solutions**:
1. Verify GitHub App is installed in Nirmata Control Hub:
   - Go to Settings → Integrations
   - Check if GitHub shows "Connected" status
2. If not connected, click **Connect** and complete the authorization
3. Ensure you selected the repositories where you want Nirmata to work

### Authentication Failures

**Symptom**: Agent cannot authenticate with GitHub

**Common Causes & Solutions**:

**For Nirmata GitHub App**:
1. Verify GitHub App is connected in Nirmata Control Hub (Settings → Integrations)
2. Check your Nirmata Control Hub Service Account token (`SERVICE_ACCOUNT_TOKEN`) or API token (`API_TOKEN`) is valid:
   ```bash
   kubectl get secret nirmata-service-account-token -n nirmata -o yaml
   ```
3. Ensure the secret contains a valid, non-expired Nirmata Control Hub Service Account token
4. Verify network connectivity from cluster to Nirmata Control Hub
5. Check that the GitHub App has access to the target repository

**For PAT**:
1. Check token hasn't expired
2. Verify token has required scopes (`repo`, `write:discussion`)
3. Ensure token secret exists in correct namespace:
   ```bash
   kubectl get secret github-pat-token -n nirmata
   ```

### Cannot Create Pull Requests

**Symptom**: Authentication succeeds but PRs aren't created

**Solutions**:
1. Verify Nirmata GitHub App is installed on the target repository:
   - Go to GitHub → Settings → Applications → Installed GitHub Apps
   - Find "Nirmata" and check repository access
2. Ensure the repository is not archived or read-only
3. Check branch protection rules don't block the app
4. Review agent logs for detailed error messages:
   ```bash
   kubectl logs -n nirmata -l app.kubernetes.io/name=nirmata-agent --tail=100
   ```

### Missing Repository Access

**Symptom**: Nirmata cannot access a specific repository

**Solutions**:
1. In GitHub, go to Settings → Applications → Installed GitHub Apps
2. Click **Configure** next to Nirmata
3. Add the repository or select "All repositories"
4. Click **Save**
5. Wait a few minutes for permissions to propagate

### ToolConfig Method Not Working

**Symptom**: Error like "unsupported method: nirmata-app"

**Solutions**:
1. Verify you're using the latest Remediator Agent version
2. Check that your Helm chart is up to date
3. Ensure the agent has network access to Nirmata Control Hub
4. Review pod logs for version/compatibility information

---

## Related Documentation

- [GitHub App Integration in Nirmata Control Hub](/docs/control-hub/settings/integrations/githubapp/)
- [Remediator Agent Overview](/docs/control-hub/agent-hub/service-agents/)
- [ToolConfig Configuration](/docs/control-hub/agent-hub/service-agents/configuration/#toolconfig)
- [GitHub Apps Documentation](https://docs.github.com/en/developers/apps)

---

## Support

For additional help:

- **Nirmata Support**: support@nirmata.com
- **Documentation**: https://docs.nirmata.io
- **GitHub Apps**: https://docs.github.com/en/developers/apps