51 lines
1.9 KiB
Markdown
51 lines
1.9 KiB
Markdown
# OKE Remediator Job
|
|
|
|
This workload runs inside OKE as a `CronJob` and performs the same high-level flow as the OCI Function:
|
|
|
|
1. Queries the business observability API.
|
|
2. Sends a task to `kagent` through MCP.
|
|
3. Detects business degradation, image pull failures, or HPA capacity issues.
|
|
4. Triggers the corresponding OCI DevOps remediation deployment for approval.
|
|
5. Sends an OCI Notifications email with the diagnostic report and the approval link to the remediation deployment.
|
|
|
|
## Authentication
|
|
|
|
The workload is configured to use OKE Workload Identity first, and falls back to generic OCI resource principals only if workload identity is not available.
|
|
|
|
For OKE Workload Identity, the important pieces are:
|
|
|
|
- Enhanced OKE cluster.
|
|
- Kubernetes `ServiceAccount`.
|
|
- IAM policy scoped to:
|
|
- `request.principal.type = 'workload'`
|
|
- `request.principal.namespace = 'kagent-demo'`
|
|
- `request.principal.service_account = 'kagent-oke-remediator'`
|
|
- `request.principal.cluster_id = '<cluster-ocid>'`
|
|
|
|
No OCI config file mount is required for the manifest included here.
|
|
|
|
## Deployment
|
|
|
|
Apply:
|
|
|
|
```bash
|
|
kubectl apply -f k8s/oke-remediator/serviceaccount.yaml
|
|
kubectl apply -f k8s/oke-remediator/role.yaml
|
|
kubectl apply -f k8s/oke-remediator/rolebinding.yaml
|
|
kubectl apply -f k8s/oke-remediator/cronjob.yaml
|
|
```
|
|
|
|
Run one job immediately for validation:
|
|
|
|
```bash
|
|
kubectl create job --from=cronjob/kagent-oke-remediator kagent-oke-remediator-manual -n kagent-demo
|
|
kubectl logs job/kagent-oke-remediator-manual -n kagent-demo -f
|
|
```
|
|
|
|
## Notes
|
|
|
|
- The remediation pipelines must include an approval stage.
|
|
- The job triggers the remediation deployment and sends the email with the deployment execution link.
|
|
- The job stores the last active incident signature in the `kagent-oke-remediator-state` ConfigMap to suppress duplicate emails for the same active issue.
|
|
- The Python worker uses `oci.auth.signers.get_oke_workload_identity_resource_principal_signer()` before any fallback signer.
|