Requirements
12+ years in software/data engineering, incl. 5+ years owning solution/platform architecture for large-scale distributed systems.End-to-end ownership of a streaming data platform on self-managed infra (on-prem/private cloud): topology, state placement, failure domains, cutover, capacity.Deep hands-on Apache Flink architecture: DataStream/SQL, keyed state on RocksDB, broadcast state, Async I/O, checkpoint/savepoint discipline, failure isolation (core requirement).Two-tier latency design: ms-level hot path vs. minute-level cold path, with explicit latency budgets.Kafka topology for multi-tenant platforms: topic taxonomy, partition-key/sizing, retention classes, naming conventions.Schema governance (Avro + registry), compatibility modes, breaking-change coordination.Lakehouse formats (Paimon, Iceberg, Delta, or Hudi) on S3-compatible storage: PK/LSM design, partitioning, compaction, snapshot expiry.Governed single-writer patterns with per-stream config and exactly-once-effective semantics.Distributed in-memory caches (Ignite, Hazelcast, Redis): partitioning, affinity colocation, invalidation.State-placement decision framework (broadcast vs. keyed vs. cache vs. first-seen load).Aggregation architecture for rolling windows with sub-ms reads and bounded state (tiles, sketches — HLL, t-digest, count-min).Identity resolution at scale: deterministic/probabilistic matching, identity graphs, merge/split, anonymous-to-known stitching.Entity lifecycle state machines with auditable transitions.Consent enforcement/data governance without synchronous per-event calls.Tokenization/encryption for sensitive identifiers: key versioning (Vault Transit), audited detokenization.End-to-end lineage/impact analysis: graph-modeled dependencies, orphan prevention.Versioned artifact cutover: grace windows, late-event handling, rollback.State reconciliation on definition changes (recompute, parallel versions, snapshot seeding).Multi-tenant isolation on Kubernetes: namespace/resource topology, per-tenant secrets/storage.AuthN/AuthZ/network security: OIDC/Keycloak, ABAC, K8s NetworkPolicies, mTLS.Failure domains, SLOs, RPO/RTO; replay/backfill without re-triggering external actions.Full-platform capacity sizing validated against measured behavior.Judgment on wrapping OSS components behind stable interfaces vs. direct dependency.Semantic model discipline; TM Forum SID/TMF620 telco experience a plus.ML integration: online/batch scoring, training-serving consistency, hot-path discipline.Spring Boot/Java control-plane services, PostgreSQL, workflow orchestration (Temporal/Argo) with BPMN (Flowable/Camunda).Observability (OpenTelemetry, Prometheus, Grafana, tracing).Architecture documentation ownership (blueprint + annexes), defended in review.AI tooling familiarity (Claude, Cursor, Codex).
Description
Owning the end-to-end solution architecture of the Daitics AI CDP: a sovereign, on-prem, telco-native Customer Data Platform built on a streaming architecture, deployed on infrastructure we run ourselves (Kubernetes, Helm), not managed cloud services. The platform processes hundreds of thousands of events per second across thirty or more source systems and delivers unified customer profiles for B2C persons and B2B organizations, accounts, sites, lines, devices and contacts. You will own the architecture across the five planes (Authoring, Control, Data, Activation, Observability) and the five layers (atomic events, tile primitives, trait values, signal filters, signal events), hold the line on architectural principles across engine boundaries, and translate them into designs engineering.