Senior AI Solution Engineer with MS Foundry, MCP

EPAM Systems

Remote · ArmeniaSalary not disclosedfull-timeEPAM Systems Careers

We are seeking a Senior AI Solution Engineer with expertise in MS Foundry and MCP to build automated data ingestion pipelines and data scraping mechanisms on Azure Databricks that fuel the enterprise AI platform and semantic layer. This role focuses on gathering structured and unstructured data, automating workflows via Azure DevOps CI/CD pipelines, and exposing clean data models formatted via Open Knowledge Format directly to Databricks semantic layers, AI Search, and AI agent runtimes.

Responsibilities

Build high-throughput data ingestion pipelines and data scraping engines using Python, PySpark, and Databricks Auto Loader to collect unstructured document content, external registry feeds, and enterprise data
Structure and expose ingested data within the Databricks semantic layer, organize datasets using Open Knowledge Format standards for seamless consumption by AI Search, Databricks Genie Spaces, and LLM reasoning engines
Automate ingestion pipeline deployments, scraping job schedules, and code releases using Azure DevOps and Databricks Asset Bundles (DABs)
Scrape and ingest healthcare and insurance data sources, including regulatory databases, public registries, and unstructured medical/claims files, to support AI agent workflows
Implement schema enforcement, automated ingestion failure alerts, and job telemetry within Databricks Workflows and Azure DevOps

Languages

Work format: Remote
Seniority: Senior
Posted: 29 Oct 2026
Last verified: In the last 3 days

Similar roles

Keep looking

More Software jobs in Armenia and remote All IT vacancies Internships and entry level