Data Engineer - Onboarding

Responsibilities

• Own the data ingestion layer that brings device telemetry, transaction events, KYC/identity signals, and third-party enrichment into the platform — designing streaming pipelines (Pub/Sub, Apache Beam on Dataflow, Flink) and batch pipelines (Python, Airflow on Cloud Composer, Spark on Dataproc) that are correct,…
• Build and evolve our feature platform, where the same Chronon feature definitions are computed by Flink for streaming and Spark for batch, with aggregation windows from one hour to 300 days, served to the rules engine and to models under a sub-second budget.
• Establish feature correctness as an engineering discipline: streaming-versus-batch reconciliation, recomputation tests against the warehouse, train/serve parity checks, and drift monitoring that catches a broken feature before an analyst does.
• Productionize fraud and identity ML models — training pipelines on Vertex AI and Kubeflow, gradient-boosted and tree-based models (XGBoost, LightGBM, CatBoost, scikit-learn), hyperparameter search, SHAP-based explanations, and score normalization — and build the automated retraining, champion/challenger promotion, and…
• Engineer KYC, AML, and identity risk signals: document verification and doc-KYC outcomes, sanctions/PEP/adverse-media screening results, email and phone risk, synthetic identity indicators, bank and account verification, and periodic customer due diligence — turning noisy, multi-vendor, multi-jurisdiction data into…
• Integrate and harden new data sources, including 30+ third-party enrichment providers called in parallel on the request path, plus our cross-client consortium network — owning failover behavior, timeout budgets, graceful degradation, caching, and cost.
• Own the warehouse and modeling layer in BigQuery — partitioning strategy, the staging-to-mart layer cake, training datasets, and the in-flight migration off dbt onto scheduled SQL and Python pipelines.
• Design the entity resolution and graph data that link customers, devices, emails, phones, cards, bank accounts, and crypto addresses across clients, including large-scale connected-components work.
• Make the platform safe by construction: field-level encryption for sensitive identifiers, regional data residency enforced in the pipeline definitions, PII handling and deletion paths, and feature-level gating so a bad signal can be turned off without a deploy.
• Set technical direction and raise the team's ceiling — write the design docs, run the reviews, mentor engineers and data scientists, and decide what we build versus buy.

Languages