Event system design¶
Each event is defined once, as a Pydantic model in app/domain/events/typed.py. The same model is dumped into an EventDocument in MongoDB, published to Kafka and handed to subscribers, so there is no separate Kafka schema and no per-event mapping to keep in sync.
graph LR
ET[EventType enum] --> DE["Event models
typed.py"]
DE --> Kafka[(Kafka topics)]
DE --> MongoDB[(MongoDB events)]
EventType in app/domain/enums/events.py lists every event type as a string, 57 of them. Each value is also the Kafka topic name, with no prefix. FastStream serializes the model to JSON on publish and validates it back into the model on consume.
BaseEvent¶
Every event extends BaseEvent:
class BaseEvent(BaseModel):
"""Base fields for all domain events."""
model_config = ConfigDict(
from_attributes=True,
json_schema_extra={"required": ["event_id", "event_type", "event_version", "timestamp", "metadata"]},
)
event_id: str = Field(default_factory=lambda: str(uuid4()))
event_type: EventType
event_version: str = "1.0"
timestamp: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
aggregate_id: str | None = None
metadata: EventMetadata
EventMetadata is a stdlib dataclass with service_name, service_version, user_id and environment. The json_schema_extra line marks the defaulted fields as required in the OpenAPI schema, so the generated TypeScript types don't make them optional. Storage fields such as stored_at and ttl_expires_at live on EventDocument, not on the event. See Event Storage.
The DomainEvent union¶
Each event class pins its type with a Literal:
class ExecutionCompletedEvent(BaseEvent):
event_type: Literal[EventType.EXECUTION_COMPLETED] = EventType.EXECUTION_COMPLETED
execution_id: str
exit_code: int
resource_usage: ResourceUsageDomain | None = None
stdout: str = ""
stderr: str = ""
DomainEvent is an Annotated union of all 57 classes with Discriminator("event_type"), and DomainEventAdapter is a TypeAdapter over it. Pydantic reads event_type first and validates against that one class, instead of trying every member until one fits.
sequenceDiagram
participant DB as MongoDB
participant Repo as EventRepository
participant TA as DomainEventAdapter
participant Event as ExecutionCompletedEvent
DB->>Repo: {event_type: "execution_completed", ...}
Repo->>TA: validate_python(doc)
TA->>Event: picks the class by event_type
Event->>Repo: typed event
EventRepository reads stored events this way. The SSE bridge does the same on the Kafka side: it subscribes to 17 topics with a single body: DomainEvent parameter and gets the right class for each message.
Publishing¶
All publishing goes through UnifiedProducer in app/events/core/producer.py. It writes the event to MongoDB, then hands it to KafkaEventTransport, which calls broker.publish with the event type as the topic.
async def produce(self, event_to_produce: DomainEvent, key: str) -> None:
await self._event_repository.store_event(event_to_produce)
await self._transport.publish(event_to_produce, event_to_produce.event_type, key)
store_event ignores a duplicate event_id, so storing the same event twice is a no-op. If Kafka fails, produce raises to the caller, which decides: the API marks a new execution error and answers 500, and a cancel answers 500. produce_or_log logs the Kafka error instead, for events sent after their work is done: auth and settings events, pod progress and the result processor's terminal event and result_stored. Nothing re-sends an event that didn't go out; a replay session with the Kafka target can.
graph TB
subgraph Producers
API[API services]
Workers[Worker: k8s worker,
pod monitor, result processor]
end
API --> Producer[UnifiedProducer]
Workers --> Producer
Producer --> MongoDB[(MongoDB events)]
Producer --> Transport[KafkaEventTransport] --> Kafka[(Kafka)]
subgraph Consumers
Sched[Worker: execution scheduler]
APIc["API: sse-bridge-pool,
notification-service"]
end
Kafka --> Sched
Kafka --> APIc
The worker consumes only execution_requested and execution_cancelled; inside it, the parts call each other directly. The DLQ manager is the one place that skips the producer: it calls broker.publish directly for retries and its dlq_message_* status events. See Kafka Topics for the consumer groups.
Adding an event type¶
- Add the value to
EventType. - Add the class to
typed.pywith the matchingLiteraldefault. - Add it to the
DomainEventunion.
The topic needs no setup, since Kafka creates it on first use. tests/unit/domain/events/test_event_schema_coverage.py fails if an EventType has no class in the union, if a BaseEvent subclass has no EventType default for event_type, or if DomainEventAdapter doesn't recognize a type.