Software Engineer, Data Systems

Responsibilities

• Develop and construct the store, data model, and query path for engineering performance data. This includes ingesting, joining, storing, and delivering it at the scale of a GPU data center with low latency.
• Join profiling data with context from the rest of the system, such as hosts, schedulers, configuration, logs, and storage, ensuring each record carries the necessary identity and dimensions for comparison across jobs, machines, versions, and time.
• Take responsibility for the data contract with the teams that generate the data: establish it, version it, and transition consumers through changes without disrupting them.
• Make deliberate trade-offs among data quality, query latency, storage cost, and retention, and verify improvements in efficiency, latency, and resource use with measurements.
• Handle delayed, duplicated, incomplete, or inconsistent data, and make its limitations visible to downstream users.
• Develop the data API along with the generated evidence views for people and automated analysis to access results.
• Implement access controls, resource limits, and maintain stable behavior during high demand or failures.
• What we need to see
• Bachelor’s degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• 12+ years of relevant software engineering experience, including delivery and operation of data pipelines, storage systems, analytical services, or comparable software used by others.

Languages