Senior AI Engineer

Responsibilities

• Design and build distributed data pipelines (Apache Beam, Python) for large-scale document and model processing
• Implement horizontally scalable LLM-powered extraction workflows across thousands of entities
• Build runner-agnostic pipelines that run across multiple environments (local, cloud, on-prem)
• Design and implement entity resolution and cross-referencing systems with confidence scoring
• Architect and build graph-based Knowledge Bases (Dgraph or similar), including schema design and query optimization
• Develop pluggable LLM extractor frameworks (classification, relationships, completeness, document linking)
• Build abstraction layers for LLM providers (Anthropic Claude, Google Gemini, local models via Ollama/vLLM)
• Optimize LLM usage for cost, latency, and reliability in production environments
• Design and implement backend service layers (Go-based APIs) connecting AI pipelines with agent systems
• Ensure observability, debugging, and performance monitoring across distributed AI systems

Languages