All vacancies
Senior Data Engineer/ Databricks, PySpark, Python
EPAM Systems
Remote · ArmeniaSalary not disclosedfull-timeVerified recentlyEPAM Systems Careers
We are looking for an experienced Senior Data Engineer / Databricks Engineer to join a project focused on collecting, processing, and analyzing vehicle telemetry data for a major European automotive manufacturer. The platform operates at a significant scale, processing tens of terabytes of telemetry data in Azure Databricks and running approximately 200 Databricks jobs.
Responsibilities
- Monitor the health, stability, and performance of the production data platform
- Investigate and resolve performance issues and production incidents
- Analyze Databricks workloads and identify performance bottlenecks
- Optimize PySpark workloads and Databricks jobs for performance, reliability, and cost efficiency
- Support and improve approximately 200 existing Databricks jobs and associated data pipelines
- Prepare the platform for increasing data volumes and workloads as additional vehicle platforms are onboarded
- Design and implement scaling strategies with a strong focus on predictable infrastructure and operational costs
- Track and optimize Databricks and Azure resource consumption
- Improve platform observability, monitoring, alerting, and operational processes
- Ensure consistency across multiple regional deployments in Europe, North America, and China
Languages
- Work format
- Remote
- Seniority
- Senior
- Posted
- 22 Sept 2026 (today)
- Last verified
- 22 Sept 2026