Architecture

Understand Conduit's federated architecture - Control Plane, Edge Connectors, and the flow of queries through the system.

Architecture Overview

Conduit is built on a federated architecture that brings queries to data rather than moving data to a central location. This design enables Conduit to work with data sovereignty requirements while providing unified access across all your industrial data sources.

Core Principles

1. Data Stays Where It Is

Unlike traditional data platforms that require ETL pipelines and data lakes, Conduit queries data at the source. Your historian data stays in your historian. Your SCADA data stays in your SCADA system. Conduit simply provides a unified query interface.

2. Queries Travel, Data Doesn't

When you ask Conduit a question, the query is compiled and routed to the appropriate edge connectors. Each connector translates the query to the native format of its data source, executes it locally, and returns only the results.

3. Context is Centralized, Data is Distributed

While data remains distributed, Conduit maintains a centralized Context Store that holds metadata about your tags, their relationships, semantic meaning, and access patterns. This context enables natural language queries without centralizing the actual data.

System Components

┌─────────────────────────────────────────────────────────────┐
│                      CONTROL PLANE                          │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │
│  │ Context     │  │ AI Query    │  │ Query               │ │
│  │ Store       │  │ Interpreter │  │ Federation          │ │
│  └─────────────┘  └─────────────┘  └─────────────────────┘ │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐ │
│  │ Tag         │  │ Pattern     │  │ User                │ │
│  │ Catalog     │  │ Cache       │  │ Management          │ │
│  └─────────────┘  └─────────────┘  └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
                              │
                    MQTT / gRPC / REST
                              │
     ┌────────────────────────┼────────────────────────┐
     │                        │                        │
     ▼                        ▼                        ▼
┌─────────────┐        ┌─────────────┐        ┌─────────────┐
│ Edge        │        │ Edge        │        │ Edge        │
│ Connector   │        │ Connector   │        │ Connector   │
│ (Ignition)  │        │ (PI)        │        │ (OPC-UA)    │
└─────────────┘        └─────────────┘        └─────────────┘
     │                        │                        │
     ▼                        ▼                        ▼
┌─────────────┐        ┌─────────────┐        ┌─────────────┐
│ Ignition    │        │ PI Data     │        │ OPC-UA      │
│ Historian   │        │ Archive     │        │ Server      │
└─────────────┘        └─────────────┘        └─────────────┘

Control Plane

The Control Plane is the brain of Conduit. It handles query compilation, routing, and context management. The Control Plane can be deployed in the cloud, on-premise, or in a hybrid configuration.

Key Components

Context Store

A PostgreSQL database that stores:

  • Tag Catalog: Metadata about all discovered tags across all sources
  • Semantic Context: Descriptions, units, relationships, and ownership
  • Vector Embeddings: For semantic similarity search
  • Access Patterns: Query history and usage statistics
  • UNS Topics: Dynamic Unified Namespace structure

NQE Parser & Query Interpreter

The Natural Query Engine processes natural language queries through several stages:

  1. AI Interpretation: Understand user intent and extract entities
  2. Query Generation: Create structured query for user review
  3. Validation: Check against the tag catalog
  4. User Review: Present query for user confirmation or adjustment
  5. Federation Planning: Determine which connectors to query

Query Federation

The federation layer:

  • Routes query fragments to appropriate edge connectors
  • Manages parallel execution across multiple sources
  • Handles partial failures gracefully
  • Aggregates and correlates results
  • Applies time-window alignment

Pattern Cache

Caches frequently-accessed query patterns and results:

  • Reduces load on edge connectors
  • Enables trend detection
  • Powers "similar queries" suggestions

Edge Connectors

Edge Connectors are lightweight agents deployed near your data sources. They handle the translation between Conduit's query language and the native protocols of each system.

Responsibilities

  1. Protocol Translation: Convert queries to native formats (SQL, PI AF, OPC-UA reads)
  2. Connection Management: Maintain secure connections to data sources
  3. Local Caching: Local cache for offline resilience
  4. Tag Discovery: Automatically discover and report available tags
  5. Health Reporting: Monitor connection status and performance

Deployment Options

Edge connectors can be deployed as:

  • Docker containers: Recommended for most deployments
  • Native binaries: For resource-constrained edge hardware
  • Kubernetes pods: For orchestrated environments

Supported Adapters

| Adapter | Protocol | Use Case | |---------|----------|----------| | Ignition | OPC-UA + SQL | Ignition SCADA/historian | | OSIsoft PI | PI Web API | PI Data Archive + AF | | OPC-UA | OPC-UA | Generic OPC-UA servers | | Modbus | Modbus TCP/RTU | PLCs and field devices | | MQTT | MQTT 3.1.1/5.0 | Broker subscriptions | | REST | HTTP/HTTPS | Generic REST APIs |

Communication Protocols

Control Plane ↔ Edge Connector

Communication between the Control Plane and Edge Connectors uses:

  • MQTT: Primary channel for commands, queries, and results
  • gRPC: Optional high-performance channel for streaming
  • REST: Fallback for environments where MQTT isn't available

Message Flow

1. User submits natural language query
2. AI interprets intent and generates structured query
3. User reviews and confirms the query
4. Federation layer identifies target connectors
5. Query dispatched to connector topics
6. Connectors execute against local sources
7. Results normalized and returned
8. Control Plane correlates and returns to user

Security

All communication is secured with:

  • TLS 1.3 encryption in transit
  • JWT tokens for authentication
  • API keys for edge connector identity
  • mTLS available for high-security deployments

Data Flow Example

Let's trace a query through the system:

Query: "Show average temperature for Tank 1 over the last hour"

1. AI Query Interpreter
   Input: "Show average temperature for Tank 1 over the last hour"
   Output: Structured NQE query + confidence score

2. User Review
   User sees: "Show average temperature by tank during last 1 hour,
              where tank is Tank1"
   User confirms or adjusts the query

3. Federation Planner
   Decision: Route to Ignition connector (owns Tank1_Temperature)

4. Edge Connector (Ignition)
   Translation: Native query against historian
   Execution: Local query, results normalized
   Result: 60 data points (1 per minute)

5. Control Plane
   Combine results, apply formatting
   Return to user with metadata

Scaling

Horizontal Scaling

  • Control Plane: Scale behind a load balancer for high query volumes
  • Edge Connectors: Deploy multiple connectors per source for redundancy
  • Context Store: PostgreSQL replication for read scaling

Vertical Scaling

  • Pattern Cache: Increase memory for larger cache sizes
  • Edge Connectors: Tune connection pools for high-throughput sources

Next Steps