Senior Data Engineer / Databricks Expert

Responsibilities

• Stermedia.ai is looking for an experienced Senior Data Engineer with strong Databricks expertise to join a data platform team building a global analytics environment for a leading pharmaceutical organization.
• The platform integrates data from more than 80 facilities worldwide, including laboratories, manufacturing systems, logistics platforms, and operational environments. Its goal is to transform distributed datasets into reliable, analytics-ready data that supports reporting, operational monitoring, and data-driven decision-making.
• You’ll contribute to the full delivery lifecycle — from direct client discussions and business requirements analysis to solution design, implementation, and production deployment. Working alongside international data engineering, analytics, and machine learning teams, you’ll focus on building scalable data pipelines and transformation layers using Databricks, Apache Spark, Python, SQL, and Delta Lake.
• The project focuses on building a scalable data platform for operational data from pharmaceutical laboratories, production systems, and global supply chains.
• As a Databricks expert, you’ll design and maintain data pipelines, optimize distributed processing, and support data quality, governance, and production reliability. DBT is a complementary skill for SQL-based transformations and analytics engineering workflows.
• Design, develop, and maintain data pipelines in Databricks using Python, PySpark, and SQL
• Build and maintain Delta Lake tables and transformation layers following bronze, silver, and gold architecture
• Implement incremental processing, change data capture, and schema evolution where required
• Develop and orchestrate production workflows, including dependencies, scheduling, retries, and monitoring
• Optimize Spark workloads, SQL queries, and compute usage to improve performance and manage costs

Languages