Software Engineer, Performance Instrumentation

Responsibilities

• Define and maintain the performance data requirements from various sources including profiling libraries, annotation APIs, frameworks, runtimes, communication libraries, schedulers, configurations, and infrastructure telemetry.
• Identify and prove visibility gaps with concrete evidence, demonstrating which diagnoses are not possible with current data.
• Collaborate with relevant teams to address these gaps or build solutions where necessary.
• Inform the build of low-overhead instrumentation and annotation APIs from the perspective of the consumer, ensuring the necessary information is conveyed efficiently and effectively.
• Ensure data quality at ingestion by implementing validation, sanitization, checks, and alerting mechanisms, so that any gaps or errors are detected before impacting users or automated analysis.
• Measure and manage the overhead and observer effects of instrumentation on real workloads, ensuring they remain within acceptable limits.
• Collaborate with multiple partner teams simultaneously, managing commitments and ensuring continuous progress.
• What we need to see
• Bachelor’s degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• 8+ years of relevant software engineering experience, including hands-on experience with instrumentation or tracing in production machine learning or systems software.

Languages