Kafka topic architecture¶
Every event type has its own topic, and the topic name is the EventType value itself, with no prefix: ExecutionRequestedEvent goes to execution_requested. Since EventType is a StringEnum, the member already is that string, so neither side needs a lookup table or .value.
That line in UnifiedProducer.produce is the whole routing layer. Subscribers name the same enum member:
async def schedule_execution(body: ExecutionRequestedEvent, scheduler: FromDishka[ExecutionScheduler]) -> None:
await scheduler.handle_execution_requested(body)
router.subscriber(EventType.EXECUTION_REQUESTED, group_id="saga-orchestrator")(schedule_execution)
Each handler gets exactly one typed model. Nobody decodes a message only to drop it because the type doesn't match.
Topic configuration¶
No code creates topics. The broker has KAFKA_AUTO_CREATE_TOPICS_ENABLE on, so a topic appears the first time a client uses it, with the broker defaults from docker-compose.yaml: 3 partitions (KAFKA_NUM_PARTITIONS) and 7 days of retention (KAFKA_LOG_RETENTION_HOURS: 168). Every topic gets the same settings.
EventType has 47 members, but the code constructs fewer than half of them, so most of those topics never come into existence. app/infrastructure/kafka/topics.py groups event types into the sets the code needs, such as the terminal events that end an execution's SSE stream. They have nothing to do with partitions or retention.
When the producer first publishes them, execution, pod and result events are keyed by execution ID. User events use the username, notifications the user ID and settings events user_settings_{user_id}. Within a topic, Kafka sends every message with the same key to the same partition, so they stay in order. Resends don't keep those keys: a DLQ retry keys the message by event_id, and a replay by aggregate_id, falling back to event_id. So a retried or replayed event can land on another partition and arrive out of order with the rest of its execution. Across topics there is no ordering at all, so a group that reads several topics, like the SSE bridge, can see one execution's events out of order. With 3 partitions, up to three consumers in one group can share a topic.
Consumer groups¶
Group IDs are string literals in app/events/handlers.py.
| Group | Runs in | Topics |
|---|---|---|
saga-orchestrator |
worker | execution_requested, execution_cancelled |
notification-service |
API | execution_completed, execution_failed, execution_timeout |
sse-bridge-pool |
API | 17 types: every execution_* event except execution_accepted, all seven pod_* events, result_stored and result_failed |
The worker's group is the execution scheduler. It kept the name of the saga orchestrator it replaced, so the offsets that group committed carried over. Kafka delivers each message to every group that subscribes to the topic, so one execution_completed reaches the notification service and the SSE bridge. Inside the worker the parts call each other directly. The pod monitor, the result processor and the DLQ job only publish.
Most subscribers use AckPolicy.ACK, which commits the offset after the handler returns, even when it raised. The SSE bridge uses ACK_FIRST and starts from the latest offset, since a browser only cares about events from now on.
Request flow¶
The execution service publishes ExecutionRequestedEvent to execution_requested. Two groups receive it: the execution scheduler queues it, and the SSE bridge forwards it to the browser. When a slot is free, the scheduler creates the pod directly. From there the pod, execution and result events flow back through their own topics, as described in the overview.
If the worker is down, execution_requested and execution_cancelled wait in Kafka for its group, and the API keeps answering.
Replay¶
graph LR
Admin[Admin API] -->|scheduled session| ReplayRepo[(ReplayRepository
MongoDB)]
ReplayRepo --> ReplayService[Worker: EventReplayService]
ReplayService --> Producer[UnifiedProducer]
Producer --> Kafka[(Kafka)]
EventReplayService reads stored events that match a filter from MongoDB in batches and publishes them again through UnifiedProducer, so each one goes back to the topic of its own type. The event store ignores the duplicate event_id, so a replay doesn't double the stored history. The admin API only stores the session, and the worker runs it, so its state lives in MongoDB and any API process can report it. See Event Replay.
Dead letters¶
The DLQ is a MongoDB collection, dlq_messages, not a Kafka topic. DLQManager.handle_message stores a failed message there and schedules a retry by the event type's category: execution events get 5 retries with exponential backoff from 30 seconds, pod events 3 retries from 60 seconds, result events 3 immediate retries, and everything else 4 retries from 60 seconds. The worker's DLQ job republishes due messages to their original topic every 10 seconds and publishes dlq_message_retried.
DeadLetterMiddleware on the execution scheduler and notification routers calls handle_message when a handler raises. A retry goes back to the original topic with a header naming the failed consumer group, and every other group skips it. See Dead Letter Queue.
Key files¶
app/domain/enums/events.pyholdsEventType, which doubles as the topic list.app/domain/events/typed.pyholds the Pydantic event models and theDomainEventunion.app/events/core/producer.pyandtransport.pyholdUnifiedProducerandKafkaEventTransport.app/events/handlers.pydeclares every subscriber, on oneKafkaRouterper consumer.app/infrastructure/kafka/topics.pyhas the event type category sets.