Senior AI Engineer - Conversational AI & Semantic Search (Databricks / LangChain)

Responsibilities

• Architect and develop a conversational AI/agentic application leveraging LangChain and/or LangGraph, hosted on Databricks
• Build semantic search and retrieval-augmented generation (RAG) pipelines using vector embeddings paired with a vector database (such as Databricks Vector Search, Chroma, Pinecone, or FAISS)
• Establish and refine similarity-matching logic, including embedding model selection, distance metrics, thresholds, and ranking of "% match" results
• Connect to various data sources (Databricks Unity Catalog tables, Delta Lake, APIs) to build and prepare the knowledge base/corpus
• Manage the complete ML/LLMOps lifecycle, covering experimentation, evaluation, versioning, deployment, and production monitoring of the chatbot
• Partner with business stakeholders to convert new use cases into clear technical specifications
• Produce clean, well-documented, maintainable Python code while setting up testing and CI/CD workflows for the application
• Guarantee proper data privacy practices, access control, and cost oversight (covering token usage and compute resources)

Languages