Senior Middleware Engineer

We are seeking a Senior Middleware Engineer with deep expertise in enterprise messaging, event-driven architecture, and distributed systems integration. In this senior-level role you will be the technical authority for all middleware platforms across the organisation, owning the design, deployment, performance, and reliability of messaging infrastructure that underpins our mission-critical applications.

You will lead integration strategy across RabbitMQ, Redis, and Azure Service Bus, working alongside development, DevOps, and cloud architecture teams to deliver scalable, resilient, and observable messaging solutions. You will also serve as the primary escalation point for integration incidents and provide technical leadership and mentoring to the wider engineering team.

KEY RESPONSIBILITIES

• Architect, deploy, and operate enterprise messaging and middleware platforms including RabbitMQ, Redis, and Azure Service Bus across development, staging, and production environments.

• Design and govern event-driven and message-driven integration patterns: pub/sub, competing consumers, dead-letter handling, saga/choreography, and request-reply.

• Lead performance engineering for middleware: tune RabbitMQ broker configuration, channel and prefetch settings, Redis eviction policies, cluster topology, and Azure Service Bus namespace tiers and partitioning.

• Define and enforce standards for message schema design, versioning, serialisation (JSON, Avro, Protobuf), and backward/forward compatibility.

• Implement and maintain High Availability and Disaster Recovery for middleware: RabbitMQ mirrored/quorum queues, Redis Cluster and Sentinel, Azure Service Bus geo-redundancy and paired namespaces.

• Establish end-to-end observability: distributed tracing (OpenTelemetry), metrics (Prometheus/Grafana), and alerting for queue depth, consumer lag, connection pool saturation, and error rates.

• Collaborate with application development teams to review integration designs, message flow architectures, and service mesh configurations before production release.

• Own middleware security posture: TLS configuration, certificate management, RBAC/ACL policies on brokers, Managed Identity authentication for Azure Service Bus, and secrets management.

• Drive Infrastructure-as-Code adoption for middleware provisioning using Terraform, Bicep, or Ansible.

• Support CI/CD pipeline integration for automated deployment, configuration, and smoke-testing of middleware components.

• Lead incident response and post-incident reviews for Severity 1 middleware outages; produce actionable remediation plans.

• Mentor mid-level and junior engineers; produce architecture decision records (ADRs), runbooks, and operational documentation.

CORE TECHNOLOGY EXPERTISE

RabbitMQ

• Expert knowledge of AMQP 0-9-1, exchanges, bindings, queues, and routing topologies (direct, topic, fanout, headers).

• Experience with Quorum Queues, Lazy Queues, and stream queues for durability and high-throughput scenarios.

• RabbitMQ cluster formation, quorum-based leader election, and cross-datacenter federation/shovels.

• Management API, Prometheus plugin, and Grafana dashboards for broker observability.

• Experience with MassTransit or NServiceBus on top of RabbitMQ for .NET service bus abstractions.

Redis

• Deep expertise in Redis data structures: Strings, Hashes, Lists, Sets, Sorted Sets, Streams, and HyperLogLog.

• Redis as a message broker: Pub/Sub, Redis Streams with consumer groups, and XREAD/XACK patterns for reliable processing.

• Redis as a caching layer: cache-aside, write-through, write-behind, TTL strategies, and cache stampede prevention.

• Redis Cluster (hash slots, resharding) and Redis Sentinel for HA; Azure Cache for Redis Premium tier configuration.

• Performance tuning: memory optimisation, eviction policies (LRU, LFU, allkeys), persistence (RDB/AOF) trade-offs.

Azure Service Bus

• Expert-level experience with Azure Service Bus queues, topics, subscriptions, and forwarding rules.

• Message sessions, scheduled messages, deferred messages, dead-letter queues (DLQ), and message lock renewal patterns.

• Premium namespace configuration: partitioning, geo-disaster recovery (paired namespaces), and availability zones.

• Authentication via Managed Identity (RBAC), Shared Access Signatures (SAS), and Private Endpoints for network isolation.

• Integration with Azure Event Grid, Azure Logic Apps, Azure Functions, and APIM for end-to-end integration workflows.

• Cost optimisation: right-sizing tiers, auto-delete on idle, and message batching strategies.

REQUIRED SKILLS & EXPERIENCE

• 7+ years of software engineering or platform engineering experience, with 4+ years focused on middleware and integration platforms.

• Demonstrable senior-level expertise across RabbitMQ, Redis, and Azure Service Bus in production enterprise environments.

• Strong understanding of distributed systems fundamentals: eventual consistency, idempotency, at-least-once vs exactly-once delivery, and the two-generals problem.

• Experience designing and implementing event-driven microservices architectures and service integration patterns (EIP).

• Proficiency in at least one primary programming language commonly used for integration workloads: C# (.NET), Java/Kotlin, Python, or Go.

• Hands-on experience with Kubernetes or container orchestration for middleware workloads (Helm charts, operators).

• Infrastructure-as-Code experience with Terraform, Bicep, or Ansible for middleware provisioning.

• Strong observability skills: distributed tracing with OpenTelemetry, metrics with Prometheus/Grafana, and structured logging.

• Experience with secure middleware configuration: TLS/mTLS, certificate rotation, secrets management (Azure Key Vault, HashiCorp Vault).

• Proven ability to lead technical initiatives, produce architectural documentation, and mentor engineers.

DESIRABLE / ADVANTAGEOUS

• Experience with Apache Kafka or Azure Event Hubs for high-throughput event streaming use cases.

• Knowledge of MassTransit, NServiceBus, or Rebus service bus frameworks for .NET.

• Familiarity with Apache Camel, MuleSoft, or Azure Integration Services for enterprise integration patterns.

• Microsoft certification: AZ-204 (Azure Developer) or AZ-305 (Azure Solutions Architect).

• Experience with service mesh technologies: Istio, Linkerd, or Azure Service Mesh for mTLS and traffic management.

• Exposure to CQRS, Event Sourcing, and Outbox Pattern implementations in distributed systems.

• Familiarity with NATS, ActiveMQ, or IBM MQ from prior enterprise middleware environments.

• Knowledge of FinOps practices for cloud messaging cost governance.