🔌 Integration

How systems communicate, share data, and coordinate. This is where most enterprise architecture complexity lives — connecting heterogeneous systems, managing data flow, and choosing between synchronous and asynchronous patterns.

🔌

API Styles

Contracts for system-to-system communication

REST

RESTful APIs, HTTP/JSON, OpenAPI/Swagger

The most common API style. Uses HTTP verbs (GET, POST, PUT, DELETE) on resource URLs. Stateless, cacheable, and human-readable. OpenAPI (Swagger) provides machine-readable API specifications.

🏛️ Context: REST is the safe default for most integrations. Enforce OpenAPI specs as contracts. Use API-first design — spec before code. Versioning strategy (URL path vs. header) should be decided once and applied consistently.

GraphQL

GraphQL, Apollo, Schema-first API

Query language for APIs where clients specify exactly what data they need. Single endpoint, strongly typed schema. Eliminates over-fetching (getting too much) and under-fetching (needing multiple calls).

🏛️ Context: Excellent for frontend-driven development with diverse client needs. Federation (Apollo Federation) enables composing a single graph from multiple team-owned subgraphs. Watch for N+1 query problems on the server side.

gRPC

gRPC, Protocol Buffers, Protobuf, HTTP/2

High-performance RPC framework using Protocol Buffers for binary serialisation over HTTP/2. Supports streaming (unary, server, client, bidirectional). Significantly faster than REST/JSON for service-to-service calls.

🏛️ Context: Preferred for internal microservice communication where performance matters. Not browser-native — use gRPC-Web or a REST gateway for external consumers. Proto files serve as strict contracts.

WebSocket

WebSocket, Socket.IO, SSE

Full-duplex persistent connection enabling real-time bidirectional communication. Used for chat, live dashboards, collaborative editing, and gaming. Server-Sent Events (SSE) is the simpler one-way alternative.

🏛️ Context: WebSockets break the request-response model — infrastructure (load balancers, firewalls) must support persistent connections. Consider SSE for one-way streaming; it's simpler and works with standard HTTP.

AsyncAPI

AsyncAPI, Event-driven API specs

A specification format (like OpenAPI but for async) that documents event-driven APIs — message brokers, channels, message schemas, and bindings. Enables machine-readable documentation of asynchronous integrations.

🏛️ Context: As organisations adopt event-driven architectures, AsyncAPI brings the same governance to events that OpenAPI brought to REST. Essential for discoverability in event-heavy ecosystems.

📨

Messaging & Events

Asynchronous communication and event-driven patterns

Message Queue

RabbitMQ, SQS, Azure Service Bus

Point-to-point messaging: a producer sends a message to a queue, and one consumer processes it. Guarantees delivery, enables work distribution, and decouples sender from receiver. Messages are consumed (deleted) after processing.

🏛️ Context: Ideal for task distribution and workload levelling. Implement dead-letter queues for failed messages. Design consumers to be idempotent — messages may be delivered more than once.

Event Streaming

Apache Kafka, Confluent, Amazon Kinesis, Pulsar

Publish-subscribe log where events are appended to partitioned topics and retained. Multiple consumers read independently at their own pace. The backbone of event-driven and event-sourced architectures.

🏛️ Context: Kafka is the de facto standard for event streaming. Key decisions: partition strategy (affects parallelism), retention period, compaction, and schema registry (Avro/Protobuf) for contract enforcement.

Event Bus / Broker

EventBridge, Cloud Events, SNS

Lightweight pub-sub routing that matches events to subscribers based on rules or patterns. Cloud-managed event buses (EventBridge) handle routing, filtering, and delivery without managing infrastructure.

🏛️ Context: EventBridge is excellent for serverless event-driven architectures within AWS. For cross-cloud or hybrid, evaluate CloudEvents as a portable event format standard.

Webhook

HTTP callback, Event notification

Push-based notification: when an event occurs, the source POSTs to a registered URL. The simplest integration pattern — no message broker needed. Widely used by SaaS platforms (Stripe, GitHub, Slack).

🏛️ Context: Simple but fragile at scale. Require: HMAC signature verification, idempotency keys, exponential backoff retries, dead-letter persistence, and timeout handling on the receiving end.

🔗

Integration Platforms

Middleware, iPaaS, and orchestration engines

Enterprise Service Bus (ESB)

MuleSoft, IBM MQ, TIBCO, BizTalk

Centralised integration backbone that routes, transforms, and orchestrates messages between systems. Legacy ESBs are heavy and monolithic; they're being replaced by lighter alternatives but persist in many enterprises.

🏛️ Context: ESBs centralise integration logic (and risk). Modern approach: decompose ESB into domain-specific integration services. Migrate incrementally — strangler fig pattern works well here.

iPaaS

MuleSoft Anypoint, Boomi, Workato, Celigo

Integration Platform as a Service — cloud-native middleware with pre-built connectors to SaaS and on-prem systems. Low-code/no-code interfaces for building integrations. Managed infrastructure and monitoring.

🏛️ Context: iPaaS accelerates SaaS-to-SaaS integration dramatically. Evaluate connector quality for your specific systems. Vendor lock-in risk is high — ensure integration logic can be exported or recreated.

Workflow / Orchestration Engine

Temporal, Step Functions, Airflow, Camunda

Engines that coordinate multi-step processes across services. Define workflows as code or BPMN diagrams. Handle retries, compensation, timeouts, and long-running transactions that span multiple systems.

🏛️ Context: Temporal is emerging as the standard for durable execution. Step Functions for serverless AWS workflows. Distinguish between orchestration (central coordinator) and choreography (services react to events).

Service Mesh

Istio, Linkerd, Consul Connect

Infrastructure layer handling service-to-service communication. Sidecar proxies (Envoy) manage mTLS, load balancing, retries, circuit breaking, and observability without changing application code.

🏛️ Context: Powerful but operationally demanding. Justified at 20+ microservices where cross-cutting concerns become unmanageable. Linkerd is simpler; Istio is more feature-rich. Evaluate ambient mesh (no sidecars).

📡

Data Integration

Moving and transforming data between systems

ETL / ELT Pipelines

dbt, Airflow, Fivetran, Informatica, Talend

ETL: Extract → Transform → Load (transform before storing). ELT: Extract → Load → Transform (store raw, transform in-place). Modern cloud warehouses favour ELT because compute scales elastically.

🏛️ Context: dbt has become the standard for the Transform step in ELT. Fivetran/Airbyte for Extract+Load. Design pipelines for idempotency, lineage tracking, and schema drift handling. Monitor data freshness SLAs.

Change Data Capture (CDC)

Debezium, DMS, Golden Gate, Log-based CDC

Captures real-time changes from database transaction logs and streams them as events. Enables near-real-time data synchronisation without polling or batch jobs. The bridge between operational and analytical systems.

🏛️ Context: CDC + Kafka is the modern alternative to batch ETL for operational data. Debezium (open-source) is the standard. Handles schema evolution, but requires careful handling of deletes and schema changes.

Data Virtualisation

Denodo, Dremio, Trino, Presto

Query data where it lives without moving it. A virtual layer presents a unified view across databases, data lakes, APIs, and files. Reduces data duplication and enables real-time access to distributed data.

🏛️ Context: Reduces data movement and duplication but adds a query performance dependency. Best for ad-hoc analytics and federated queries. Not a replacement for a properly designed data warehouse for heavy reporting.

File Transfer / SFTP

SFTP, MFT, AS2, EDI

The oldest integration pattern — exchanging files between systems. Still prevalent in B2B (EDI), finance (SWIFT), healthcare (HL7), and government. Managed File Transfer (MFT) adds security, auditing, and automation.

🏛️ Context: File-based integration isn't glamorous but it's everywhere. Modernise with managed MFT, automate with event-driven triggers, and ensure encryption in transit and at rest. Plan for format evolution.

Key Integration Patterns

Request-Response (Synchronous)

Caller waits for a response. Simple but creates tight coupling. If the downstream service is slow or down, the caller is blocked. Use for real-time queries where the user is waiting.

Fire-and-Forget (Async)

Producer sends a message and moves on without waiting. Queue/broker guarantees eventual delivery. Decouples systems in time. Use for commands where immediate confirmation isn't needed.

Publish-Subscribe

Publisher emits events; multiple subscribers receive independently. Neither knows about the other. Enables extensibility — new subscribers can be added without changing the publisher.

Saga (Distributed Transactions)

A sequence of local transactions across services with compensating actions for rollback. Replaces distributed ACID transactions. Two styles: orchestration (central coordinator) or choreography (event-driven).

Strangler Fig

Incrementally migrate from a legacy system by routing traffic through a facade. New features go to the new system; old features are migrated one by one. The legacy system gradually "withers."

Anti-Corruption Layer

A translation layer between your domain and an external system's model. Prevents a legacy or third-party system's concepts from leaking into your clean domain model.

How Integration Connects

⬆️

Integration → Application (Layer 7): APIs are how frontends talk to backends and how applications expose functionality. Integration patterns shape application architecture.

⬇️

Data (Layer 3) → Integration: ETL/CDC moves data between stores. APIs query databases. Event streaming connects operational and analytical worlds.

🛡️

Security (Layer 5) ↔ Integration: API authentication (OAuth), mTLS between services, message encryption, and data masking in transit are all security-integration intersections.

🌐

Network (Layer 1) ↔ Integration: Network latency, bandwidth, and reliability directly constrain integration patterns. High-latency links favour async; low-latency enables sync.