Deployment

Deploy Conduit in production - Docker, Kubernetes, on-premise, and air-gapped environments.

Deployment Guide

This guide covers deploying Conduit in various production environments.

Deployment Options

| Option | Best For | |--------|----------| | Docker Compose | Small deployments, evaluation | | Kubernetes | Cloud-native, scalable deployments | | Native Binary | Edge hardware, minimal overhead | | Air-Gapped | Secure, isolated environments |

Docker Compose Deployment

Basic Setup

# docker-compose.yaml
version: '3.8'

services:
  control-plane:
    image: conduit/control-plane:latest
    ports:
      - "8080:8080"
    environment:
      - DATABASE_URL=postgresql://conduit:password@postgres:5432/conduit
      - MQTT_BROKER=mqtt://mqtt:1883
      - JWT_SECRET=${JWT_SECRET}
    depends_on:
      - postgres
      - mqtt
    volumes:
      - ./config:/etc/conduit
      - conduit-data:/var/lib/conduit

  postgres:
    image: postgres:15
    environment:
      - POSTGRES_USER=conduit
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=conduit
    volumes:
      - postgres-data:/var/lib/postgresql/data

  mqtt:
    image: eclipse-mosquitto:2
    ports:
      - "1883:1883"
    volumes:
      - ./mosquitto.conf:/mosquitto/config/mosquitto.conf
      - mqtt-data:/mosquitto/data

volumes:
  conduit-data:
  postgres-data:
  mqtt-data:

Starting the Stack

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f control-plane

# Check status
docker-compose ps

Kubernetes Deployment

Helm Chart

# Add Conduit Helm repository
helm repo add conduit https://charts.conduit.io
helm repo update

# Install Conduit
helm install conduit conduit/conduit \
  --namespace conduit \
  --create-namespace \
  --set controlPlane.replicas=3 \
  --set database.host=postgres.database.svc \
  --set mqtt.broker=mqtt.messaging.svc

Custom Values

# values.yaml
controlPlane:
  replicas: 3
  image:
    repository: conduit/control-plane
    tag: latest

  resources:
    requests:
      cpu: 500m
      memory: 512Mi
    limits:
      cpu: 2000m
      memory: 2Gi

  ingress:
    enabled: true
    className: nginx
    hosts:
      - host: conduit.company.com
        paths:
          - path: /
            pathType: Prefix
    tls:
      - secretName: conduit-tls
        hosts:
          - conduit.company.com

database:
  host: postgres.database.svc
  port: 5432
  name: conduit
  username: conduit
  existingSecret: conduit-db-credentials

mqtt:
  broker: mqtt://mqtt.messaging.svc:1883
  existingSecret: conduit-mqtt-credentials

monitoring:
  serviceMonitor:
    enabled: true

Manual Kubernetes Manifests

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: conduit-control-plane
  namespace: conduit
spec:
  replicas: 3
  selector:
    matchLabels:
      app: conduit-control-plane
  template:
    metadata:
      labels:
        app: conduit-control-plane
    spec:
      containers:
        - name: control-plane
          image: conduit/control-plane:latest
          ports:
            - containerPort: 8080
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: conduit-secrets
                  key: database-url
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 30
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080

Edge Connector Deployment

Docker

docker run -d \
  --name conduit-edge-ignition \
  -e CONDUIT_CONTROL_PLANE=mqtt://control-plane:1883 \
  -e CONDUIT_API_KEY=${API_KEY} \
  -v /etc/conduit/adapters:/etc/conduit \
  conduit/adapter-ignition:latest

Kubernetes DaemonSet

For deploying edge connectors across nodes:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: conduit-edge
  namespace: conduit-edge
spec:
  selector:
    matchLabels:
      app: conduit-edge
  template:
    spec:
      containers:
        - name: edge-connector
          image: conduit/edge-connector:latest
          env:
            - name: CONDUIT_CONTROL_PLANE
              value: mqtt://control-plane.conduit:1883
          volumeMounts:
            - name: config
              mountPath: /etc/conduit
      volumes:
        - name: config
          configMap:
            name: conduit-edge-config

Native Binary

For resource-constrained edge hardware:

# Download binary
curl -LO https://releases.conduit.io/edge-connector/latest/conduit-edge-linux-arm64

# Install
chmod +x conduit-edge-linux-arm64
sudo mv conduit-edge-linux-arm64 /usr/local/bin/conduit-edge

# Create systemd service
sudo cat > /etc/systemd/system/conduit-edge.service << EOF
[Unit]
Description=Conduit Edge Connector
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/conduit-edge --config /etc/conduit/edge.yaml
Restart=always
User=conduit

[Install]
WantedBy=multi-user.target
EOF

# Start service
sudo systemctl enable conduit-edge
sudo systemctl start conduit-edge

Air-Gapped Deployment

For environments without internet access:

1. Export Images

# On internet-connected machine
docker pull conduit/control-plane:latest
docker pull conduit/edge-connector:latest
docker save conduit/control-plane:latest | gzip > conduit-control-plane.tar.gz
docker save conduit/edge-connector:latest | gzip > conduit-edge.tar.gz

2. Transfer to Air-Gapped Environment

# Transfer files via approved media

3. Import Images

# On air-gapped machine
docker load < conduit-control-plane.tar.gz
docker load < conduit-edge.tar.gz

4. Configure for Offline

# config/conduit.yaml
updates:
  autoCheck: false

telemetry:
  enabled: false

# All dependencies must be internal
database:
  host: internal-postgres.local

mqtt:
  broker: mqtt://internal-mqtt.local:1883

Production Checklist

Security

  • [ ] TLS enabled for all connections
  • [ ] Secrets managed via secret manager (Vault, AWS Secrets, etc.)
  • [ ] Network policies restrict traffic
  • [ ] RBAC configured with least privilege
  • [ ] Audit logging enabled

High Availability

  • [ ] Control Plane: 3+ replicas
  • [ ] Database: Replicated (primary + standby)
  • [ ] MQTT: Clustered broker
  • [ ] Load balancer configured
  • [ ] Health checks enabled

Monitoring

  • [ ] Metrics exported (Prometheus)
  • [ ] Dashboards created (Grafana)
  • [ ] Alerts configured
  • [ ] Log aggregation (ELK, Loki)
  • [ ] Tracing enabled (Jaeger, Zipkin)

Backup & Recovery

  • [ ] Database backups scheduled
  • [ ] Configuration versioned in Git
  • [ ] Disaster recovery plan documented
  • [ ] Recovery procedures tested

Performance

  • [ ] Resource limits set appropriately
  • [ ] Connection pools sized correctly
  • [ ] Query timeouts configured
  • [ ] Caching enabled

Scaling Considerations

Horizontal Scaling

| Component | Scale Strategy | |-----------|---------------| | Control Plane | Add replicas behind load balancer | | Edge Connectors | One per data source location | | Database | Read replicas for query scaling | | MQTT | Broker clustering |

Vertical Scaling

| Symptom | Solution | |---------|----------| | Slow queries | Increase Control Plane memory | | Connection timeouts | Increase pool sizes | | High latency | Add edge connectors closer to sources |

Troubleshooting

Health Checks

# Control Plane health
curl https://conduit.company.com/health

# Edge Connector status
curl http://localhost:9090/metrics | grep conduit_edge

Common Issues

Control Plane won't start

  • Check database connectivity
  • Verify MQTT broker is reachable
  • Review logs: docker logs conduit-control-plane

Edge Connector can't connect

  • Verify API key is valid
  • Check network path to Control Plane
  • Confirm MQTT port is open

Slow query performance

  • Check database indexes
  • Review query patterns
  • Consider adding caching

Next Steps