Skip to main content
The safe production architecture is Amazon MSK for Kafka brokers and Springwinter for producers, consumers, and stream-processing applications. Current Springwinter resources do not provision a Kafka cluster or durable broker volumes. Updated October 9, 2026. Do not run a production Kafka broker as an ordinary Springwinter Worker. Use MSK and deploy Kafka clients on Springwinter.

​
Why brokers and clients need different infrastructure

Kafka brokers require stable identities, durable replicated storage, controlled rolling maintenance, partition rebalancing, and private multi-broker networking. A long-running application container is not a substitute for that stateful operating model. Springwinter web servers and workers are a strong fit for Kafka clients because clients are replaceable compute. Producers publish events. Consumer groups divide partitions among replicas. Stream processors rebuild local state from Kafka topics and checkpoints.
  1. Create an Amazon MSK provisioned or serverless cluster in the same AWS Region and VPC used by the applications.
  2. Keep broker endpoints private.
  3. Permit network traffic from application task security groups to the MSK security group on the configured broker ports.
  4. Choose IAM, SASL/SCRAM, or mTLS authentication.
  5. Deploy HTTP producers as Springwinter web servers.
  6. Deploy consumers and stream processors as Springwinter workers.
  7. Monitor consumer lag, rejected connections, broker storage, and processing errors.

​
IAM authentication

MSK supports IAM authentication for Java and non-Java clients. ECS applications should receive AWS credentials through a task IAM role rather than static access keys. A Java client commonly uses:
Grant only the cluster, topic, group, and transactional actions that the application requires. Springwinter’s current public configuration does not expose arbitrary per-resource task-role authoring, so verify that the deployed task role supports your chosen MSK authentication before committing to this design.

​
Deploy the application clients

1

Create the MSK cluster

Provision MSK separately and record bootstrap broker endpoints, authentication mode, and security group.
2

Build the producer or consumer

Package the application in a Docker image. Read broker endpoints, topic names, group IDs, and authentication settings from environment variables.
3

Deploy on Springwinter

Use a Web server for HTTP-facing producers and a Worker for continuously polling consumers.
4

Validate failure behavior

Restart a consumer, revoke network access in a test environment, and confirm retries, lag alerts, and idempotent processing.

​
Production checklist

  • Use at least three Availability Zones when the selected MSK mode and Region support it.
  • Set replication and minimum in-sync replica policies deliberately.
  • Avoid automatic topic creation in production.
  • Use idempotent producers where ordering and duplicate resistance matter.
  • Track consumer lag by group and topic partition.
  • Cap retries and route poison messages to a recovery topic.
  • Test client compatibility before broker-version upgrades.

​
Frequently asked questions

Not as a first-class durable resource. Springwinter Workers are replaceable ECS services and do not provide Kafka’s required broker identity, replicated disk topology, or cluster lifecycle management.
Serverless reduces broker capacity management for compatible workloads. Provisioned MSK provides more explicit broker, storage, configuration, and scaling control. Compare protocol requirements, throughput shape, networking, and regional pricing.
Yes. Producers and consumers are ordinary containerized applications. They need network reachability, authentication, sufficient CPU and memory, and a controlled restart strategy.

​
Sources and further reading

Last modified on October 8, 2026