# OCI DevOps Remediation Pipelines

This demo uses one detection path and three independent remediation pipelines. The preferred remediation style is declarative Kubernetes manifests applied by OCI DevOps.

## Detection flow

Use the main deploy or diagnosis pipeline to invoke the kagent Function with:

```text
k8s/kagent-multi-error-payload.json
```

kagent can return one or more problems:

```text
BUSINESS_PROCESS_DEGRADATION
IMAGE_PULL_FAILURE
HPA_AUTOSCALING_FAILURE
```

OCI DevOps does not provide a simple native OR/branch based on the Function JSON. Use one of these patterns:

```text
Manual demo:
Invoke Function -> Approval -> operator starts the correct remediation pipeline.

Automated demo:
Function parses kagent response and calls OCI DevOps API to start the correct remediation pipeline.
```

## Pipeline 1: Business application remediation

Pipeline name:

```text
kagent-remediate-business-config
```

Purpose:

```text
Fix order-service business degradation caused by ConfigMap thresholds or stuck review mode.
```

Artifacts:

```text
Name: remediate-business-configmap
Type: Kubernetes manifest
Path: k8s/remediation/business-configmap.yaml

Name: remediate-order-rollout-restart
Type: Kubernetes manifest
Path: k8s/remediation/order-rollout-restart.yaml
```

Recommended parameters:

```text
APP_NAMESPACE=kagent-demo
CONFIGMAP_NAME=demo-config
ORDER_MANUAL_REVIEW_RATE_THRESHOLD=70
ORDER_REVENUE_AT_RISK_THRESHOLD=15000
ORDER_STUCK_REVIEW_MODE=false
PAYMENT_FORCE_TIMEOUT=false
INVENTORY_FORCE_OUTAGE=false
WAIT_TIMEOUT=300s
```

Stages:

```text
approval-remediate-business
  -> apply-business-configmap
  -> apply-order-rollout-restart
  -> wait-after-business-fix
  -> invoke-kagent-function-validate
```

Important:

```text
order-service reads ConfigMap values as environment variables.
After applying business-configmap.yaml, apply order-rollout-restart.yaml with a new <restart-token>.
```

## Pipeline 2: ImagePull remediation

Pipeline name:

```text
kagent-remediate-imagepull-secret
```

Purpose:

```text
Restore the valid OCIR imagePullSecret in any business deployment: order-service, payment-service, or inventory-service.
```

Artifacts:

```text
Name: remediate-imagepull-order
Type: Kubernetes manifest
Path: k8s/remediation/imagepull-order.yaml

Name: remediate-imagepull-payment
Type: Kubernetes manifest
Path: k8s/remediation/imagepull-payment.yaml

Name: remediate-imagepull-inventory
Type: Kubernetes manifest
Path: k8s/remediation/imagepull-inventory.yaml

Name: remediate-imagepull-all-services
Type: Kubernetes manifest
Path: k8s/remediation/imagepull-all-services.yaml
```

Manifest placeholders:

```text
<region-key>
<tenancy-namespace>
<image-tag>
<restart-token>
```

Allowed values:

```text
DEPLOYMENT_NAME=order-service
DEPLOYMENT_NAME=payment-service
DEPLOYMENT_NAME=inventory-service
```

When remediating another service, set `CONTAINER_NAME` to the same value as `DEPLOYMENT_NAME`.

Stages:

```text
approval-remediate-imagepull
  -> apply-imagepull-service-manifest
  -> wait-after-imagepull-fix
  -> invoke-kagent-function-validate
```

For exactly three remediation pipelines, use `imagepull-all-services.yaml`. It restores the imagePullSecret in all three services and avoids conditional branching.

Create one pipeline per target service only if you want narrower remediation:

```text
kagent-remediate-imagepull-order
kagent-remediate-imagepull-payment
kagent-remediate-imagepull-inventory
```

Or create one pipeline and manually select the matching manifest artifact based on kagent output.

## Pipeline 3: HPA capacity remediation

Pipeline name:

```text
kagent-remediate-order-hpa
```

Purpose:

```text
Increase HPA capacity when order-service reaches maxReplicas and demand still requires more pods.
```

Artifact:

```text
Name: remediate-order-hpa-capacity
Type: Kubernetes manifest
Path: k8s/remediation/order-hpa-capacity.yaml
```

Recommended parameters:

```text
APP_NAMESPACE=kagent-demo
HPA_NAME=order-service
MIN_REPLICAS=3
MAX_REPLICAS=8
CPU_TARGET=65
```

Stages:

```text
approval-remediate-hpa
  -> apply-order-hpa-capacity
  -> wait-after-hpa-fix
  -> invoke-kagent-function-validate
```

## Demo execution order

Use this sequence for the conference:

```text
1. Run kagent-deployment-pipeline to deploy the healthy application.
2. Run kagent-simulate-failures-pipeline to inject controlled failures.
3. Run invoke-kagent-function with kagent-multi-error-payload.json.
4. Review kagent output.
5. Run one or more remediation pipelines:
   - kagent-remediate-business-config
   - kagent-remediate-imagepull-order/payment/inventory
   - kagent-remediate-order-hpa
6. Run final kagent validation.
```

## Mapping from kagent response to remediation pipeline

```text
BUSINESS_PROCESS_DEGRADATION -> kagent-remediate-business-config
IMAGE_PULL_FAILURE           -> kagent-remediate-imagepull-secret
HPA_AUTOSCALING_FAILURE      -> kagent-remediate-order-hpa
NO_ACTIVE_FAILURE            -> no remediation
```