Senior Data Integration Engineer

Responsibilities

• Pipeline Architecture & Development
• Design, build, and optimize scalable, reliable batch and near-real-time ETL/ELT pipelines using Python, PySpark, SQL, and modern cloud integration engines
• Orchestration & Automation
• Develop and manage complex workflow orchestrations (using Apache Airflow or cloud native schedulers) and automate ingestion routines to minimize manual operations
• Data Modeling & Warehousing
• Design and implement modern data warehouse/lakehouse layers (using Snowflake, ClickHouse, Azure Synapse, or Redshift), establishing optimal partitioning, indexing, and Slowly Changing Dimension (SCD Type 2) patterns
• Data Quality & Testing Integration: Establish rigorous data quality checks and validation frameworks utilizing tools like dbt (data build tool), Soda, or customized PySpark testing suites
• Collaboration &
• Design: Work closely with product owners, business analysts, and systems architects to define data requirements, analyze technical constraints, design Source-to-Target Mappings (STTM), and make critical architectural decisions
• Code Quality & DevOps

Languages