Skip to content
ITA Jobs
All vacancies

Data Platform Engineer (102-08SENG-01)

OpsBrasil Serviços Cloud LTDA

Remote · worldwide, UTC+4 acceptedSalary not disclosedcontractHimalayas
Apply on company siteMatch my CV

This role converts a large Azure Data Factory estate into Databricks workflows on AWS. The scope for year one: 2,094 ADF pipelines to migrate — built as reusable templates rather than one-by-one — 9,859 pipeline activities to translate (some map directly, others need rewriting as Lambda or Step Functions), 471 Spark dataflows to move onto Databricks on AWS, and 4 Databricks workspaces (Dev, QA, Pre-prod, Prod) to rehost, including notebook paths and Unity Catalog rewiring.

Responsibilities

  • Convert Azure Data Factory pipelines into Databricks workflows on AWS, building reusable templates rather than migrating one at a time.
  • Rehost Databricks workspaces onto AWS and migrate ADLS Gen2 storage to S3.
  • Rewrite ADF Web Activities as Lambda functions or Step Functions tasks, and replace ADF-specific scaling with native Databricks mechanisms.
  • Build and tune PySpark transformations for production data volumes.
  • Replace Azure Synapse Serverless with Databricks SQL Warehouse.
  • Reconcile migrated data against source systems as part of the definition of done.
  • Production experience with Databricks: workspaces, jobs and workflows. The central skill for this role.
  • Strong Spark and PySpark experience for real data volumes, including tuning.
  • Production-grade Python.
  • Experience building or migrating Azure Data Factory pipelines, with a solid understanding of the ADF activity model.

Languages

    Work format
    Remote
    Seniority
    Senior
    Posted
    21 Oct 2026
    Last verified
    25 Oct 2026
    Apply by
    19 Nov 2026

    Similar roles

    Keep looking

    What DevOps Engineer postings ask for, and what they payMore Software jobs in Armenia and remoteRemote jobs, open worldwideAll IT vacanciesInternships and entry level