1178 - Sr. Data Architect

Responsibilities

• Design and own the end-to-end lakehouse architecture on Databricks, using Medallion (bronze/silver/gold) patterns.
• Build and optimize scalable batch and streaming pipelines in PySpark, Spark SQL and Python.
• Lead platform migrations (e.g., legacy data warehouse or Hadoop/EMR to Databricks) with measurable gains in cost and run time.
• Define data governance: Unity Catalog, fine-grained access control, auditing, lineage and compliance (GDPR and similar).
• Model data for analytics and operations (OLTP/OLAP, dimensional, SQL and NoSQL).
• Set engineering standards: CI/CD, unit and integration tests, code review, reusable Python packages, Databricks Asset Bundles.
• Manage infrastructure as code with Terraform and drive FinOps practices to control cloud spend.
• Evaluate and prototype AI use cases on the platform (LLMs, vector databases, agents, MLflow).
• Advise other teams on architecture and mentor junior and mid-level engineers.

Languages