Senior Data Engineer/ Databricks, PySpark, Python

Responsibilities

• Monitor the health, stability, and performance of the production data platform
• Investigate and resolve performance issues and production incidents
• Analyze Databricks workloads and identify performance bottlenecks
• Optimize PySpark workloads and Databricks jobs for performance, reliability, and cost efficiency
• Support and improve approximately 200 existing Databricks jobs and associated data pipelines
• Prepare the platform for increasing data volumes and workloads as additional vehicle platforms are onboarded
• Design and implement scaling strategies with a strong focus on predictable infrastructure and operational costs
• Track and optimize Databricks and Azure resource consumption
• Improve platform observability, monitoring, alerting, and operational processes
• Ensure consistency across multiple regional deployments in Europe, North America, and China

Languages

Required

• Expertise in Azure Databricks
• Advanced skills in Python development
• Understanding of performance optimization for large-scale Spark workloads
• Background in operating and troubleshooting production data platforms
• Knowledge of monitoring, observability, and capacity planning combined with cloud cost optimization

Nice to have

• English proficiency at an Upper-Intermediate level (B2) or higher
• Familiarity with Microsoft Azure services and cloud infrastructure
• Experience building and maintaining CI/CD pipelines using GitHub Actions

Other open roles here

• Senior/Lead Power BI Software EngineerRemote · Ukraine
• Delivery Manager (AI Native)Remote · Ukraine
• Senior Financial Manager (m/f/d)Remote · Germany
• Principal, Business ConsultingRemote · United States
• Senior Delivery Manager - .NET/JavaIndia · On-site · relocation