Building a Scalable Middleware Architecture with Salesforce & AWS

INTRODUCTION

Integrating Salesforce with multiple external applications can quickly become complex, especially when data volumes grow. A modern, scalable solution is to build a Node.js middleware hosted on AWS App Runner, orchestrated via AWS Lambda and an event-driven architecture.

This article describes a reliable, scalable, and maintainable design for integrating Salesforce with external systems.

ARCHITECTURE OVERVIEW

Component Responsibilities

WORKFLOW DESCRIPTION

1. Salesforce emits an event (platform event or CDC).
2. Event Relay receives it and forwards it to Lambda.
3. Lambda validates and transforms the event, then calls App Runner via HTTPS.
4. App Runner middleware orchestrates the workflow:
   • Calls one or multiple external APIs
   • Handles retries, throttling, and response transformations
   • Logs requests and responses centrally
5. External applications receive processed data reliably.

ADVANTAGES OF THIS ARCHITECTURE

• Event-Driven & Decoupled: Each layer is independent, reducing impact of failures.
• Scalable: Lambda handles bursts; App Runner scales for sustained API calls.
• Long-Running Operations: App Runner supports multi-step API calls without Lambda time limits.
• Centralized Security & Logging: Middleware layer centralizes secrets, tokens, and logging.
• Future-Proof: Can migrate App Runner workloads to ECS/Fargate for millions of requests/day.

BEST PRACTICES

• Secrets Management: Use AWS Secrets Manager or Parameter Store for API keys.
• Rate Limiting: Prevent overloading external APIs with express-rate-limit or similar middleware.
• Health Monitoring: Add /health endpoints and CloudWatch alarms.
• Retries & Error Handling: Implement exponential backoff and circuit breakers.
• Concurrency Tuning: Configure App Runner concurrency to match expected traffic patterns.
• Prefer Salesforce Platform Events over CDC when possible.

SCALING STRATEGY

• Startup / MVP: App Runner + Lambda handles hundreds to thousands of events/day.
• Growth: Tune concurrency and scale App Runner min/max instances for tens of thousands of events/day.
• Enterprise: Migrate App Runner workloads to ECS or Fargate for millions of events/day, keeping the same Docker images.