Installation Guide¶

This guide walks you through installing the LlamaStack Kubernetes Operator in your cluster.

Prerequisites¶

Before installing the operator, ensure you have:

Kubernetes cluster (version 1.25 or later)
kubectl configured to access your cluster
Cluster admin permissions to install CRDs and RBAC resources
Container runtime that supports pulling images from public registries

Installation Methods¶

Method 1: Quick Install (Recommended)¶

The fastest way to install the operator is using the pre-built manifests:

kubectl apply -f https://github.com/llamastack/llama-stack-k8s-operator/releases/latest/download/operator.yaml

This will: - Install the Custom Resource Definitions (CRDs) - Create the necessary RBAC resources - Deploy the operator in the llamastack-operator-system namespace

Method 2: Helm Chart¶

Install using Helm for more configuration options:

# Add the Helm repository
helm repo add llamastack https://llamastack.github.io/helm-charts
helm repo update

# Install the operator
helm install llamastack-operator llamastack/llama-stack-k8s-operator \
  --namespace llamastack-operator-system \
  --create-namespace

Method 3: Kustomize¶

For customized deployments, use Kustomize:

# Clone the repository
git clone https://github.com/llamastack/llama-stack-k8s-operator.git
cd llama-stack-k8s-operator

# Install using Kustomize
kubectl apply -k config/default

Verification¶

After installation, verify that the operator is running:

# Check operator deployment
kubectl get deployment -n llamastack-operator-system

# Check operator logs
kubectl logs -n llamastack-operator-system deployment/llamastack-operator-controller-manager

# Verify CRDs are installed
kubectl get crd llamastackdistributions.llamastack.io

Expected output:

NAME                                        CREATED AT
llamastackdistributions.llamastack.io      2024-01-15T10:30:00Z

Configuration¶

Resource Requirements¶

The operator has minimal resource requirements:

resources:
  limits:
    cpu: 500m
    memory: 128Mi
  requests:
    cpu: 10m
    memory: 64Mi

Environment Variables¶

Configure the operator behavior using environment variables:

Variable	Description	Default
`METRICS_BIND_ADDRESS`	Metrics server bind address	`:8080`
`HEALTH_PROBE_BIND_ADDRESS`	Health probe bind address	`:8081`
`LEADER_ELECT`	Enable leader election	`false`
`LOG_LEVEL`	Logging level	`info`

Custom Configuration¶

For custom configurations, create a kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/llamastack/llama-stack-k8s-operator/config/default

patchesStrategicMerge:
- manager_config_patch.yaml

images:
- name: quay.io/llamastack/llama-stack-k8s-operator
  newTag: v0.1.0

Namespace Configuration¶

Default Namespace¶

By default, the operator watches all namespaces. To restrict to specific namespaces:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: llamastack-operator-controller-manager
spec:
  template:
    spec:
      containers:
      - name: manager
        env:
        - name: WATCH_NAMESPACE
          value: "llamastack-system,production"

Multi-tenant Setup¶

For multi-tenant environments, install the operator with namespace restrictions:

# Install operator in tenant namespace
kubectl create namespace tenant-a
kubectl apply -f operator.yaml -n tenant-a

# Configure RBAC for tenant isolation
kubectl apply -f tenant-rbac.yaml

Security Configuration¶

RBAC¶

The operator requires the following permissions:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: llamastack-operator-manager-role
rules:
- apiGroups: ["llamastack.io"]
  resources: ["llamastackdistributions"]
  verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]
- apiGroups: [""]
  resources: ["services", "configmaps", "persistentvolumeclaims"]
  verbs: ["create", "delete", "get", "list", "patch", "update", "watch"]

Network Policies¶

Secure your deployment with network policies:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: llamastack-operator-netpol
  namespace: llamastack-operator-system
spec:
  podSelector:
    matchLabels:
      control-plane: controller-manager
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector: {}
    ports:
    - protocol: TCP
      port: 8080
    - protocol: TCP
      port: 8081

Troubleshooting¶

Common Issues¶

1. CRD Installation Failed

# Check if CRDs exist
kubectl get crd | grep llamastack

# Manually install CRDs
kubectl apply -f https://raw.githubusercontent.com/llamastack/llama-stack-k8s-operator/main/config/crd/bases/llamastack.io_llamastackdistributions.yaml

2. Operator Pod Not Starting

# Check pod status
kubectl get pods -n llamastack-operator-system

# Check events
kubectl describe pod -n llamastack-operator-system <pod-name>

# Check logs
kubectl logs -n llamastack-operator-system <pod-name>

3. Permission Denied Errors

# Check RBAC configuration
kubectl auth can-i create llamastackdistributions --as=system:serviceaccount:llamastack-operator-system:llamastack-operator-controller-manager

# Verify service account
kubectl get serviceaccount -n llamastack-operator-system

Debug Mode¶

Enable debug logging for troubleshooting:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: llamastack-operator-controller-manager
spec:
  template:
    spec:
      containers:
      - name: manager
        env:
        - name: LOG_LEVEL
          value: "debug"

Uninstallation¶

To remove the operator:

# Delete operator deployment
kubectl delete -f https://github.com/llamastack/llama-stack-k8s-operator/releases/latest/download/operator.yaml

# Clean up CRDs (this will delete all LlamaStackDistribution resources)
kubectl delete crd llamastackdistributions.llamastack.io

Data Loss Warning

Deleting the CRD will remove all LlamaStackDistribution resources and their associated data. Make sure to backup any important configurations before uninstalling.

Next Steps¶

After successful installation: