Job Description
Engineers develop novel metrics and sampling techniques to measure trajectories from ML models simulating driving. They employ creative simulation strategies to assess generative AI performance, identifying edge cases for reliable insights. Roles build data pipelines for signal discovery, feature extraction, metric computation in large-scale simulations. Engineers conduct data analysis to diagnose regressions in ML models, informing development.
Waymo develops autonomous driving technology since 2009 from Google project origins. The company focuses on Waymo Driver for safety, cutting crash deaths by thousands. Driver powers ride-hail services, vehicle platforms, with over 10 million rider-only trips. Operations include over 100 million miles on public roads, tens of billions in simulation across 15 states. Large Model Evaluation team assesses AI systems like LLMs, VLMs handling complex driving.
Engineers collaborate with engineering, research teams on large-scale ML models development. They navigate technical landscapes, define strategies, create roadmaps. Proficiency in Python or C++ required for programming fundamentals, software design, testing, version control. Experience with pipelines for data processing, system evaluation, metric computation. ML knowledge includes AI fundamentals, transformer architectures, distillation, evaluating model quality.
The base salary range is $204,000 to $259,000 USD. Compensation includes discretionary annual bonus, equity incentive plan, generous benefits. Full-time across US locations based on work needs. Benefits cover medical, dental, vision, paid time off. Professional development involves ML frameworks like JAX, TensorFlow, robotics, autonomous vehicles. Environment emphasizes quantitative skills in simulation systems.
Responsibilities
- Develop novel metrics and sampling techniques to measure driving trajectories from ML models
- Employ creative simulation strategies to measure driving performance of generative AI models
- Identify potential edge cases, provide reliable performance insights
- Build data pipelines for signal discovery, data labeling, feature extraction and metric computation
- Conduct data analysis to diagnose regressions in ML models
- Collaborate with engineering and research teams developing large-scale ML models
Requirements
- 5+ years of relevant industry experience in a heavily quantitative software engineering area
- Experience navigating complex technical and product landscapes, defining technical strategy, and creating roadmaps
- Proficiency in programming in Python or C++
- Experience with software design principles, coding best practices, testing methodologies, and version control
- Experience building software pipelines for data processing, system evaluation, or metric computation
- Knowledge of AI fundamentals, such as transformer architectures, distillation techniques, etc.
- Experience evaluating the quality of ML models
- Demonstrated experience taking quantitative findings through to productionized tools
- Experience with simulation systems, robotics, or autonomous vehicles preferred
- Familiarity with modern deep learning frameworks (e.g. JAX, Tensorflow) preferred