Software Engineer, Evaluation Engineering

Apple's Siri organization is hiring a Software Engineer to build the on-device execution harness and runtime infrastructure for agentic evaluation of Apple Intelligence. The role involves designing low-level harnesses, environment control, and trace collection pipelines to validate agentic reasoning, tool use, and on-screen awareness across Apple devices. This is a systems engineering position focused on deterministic multi-step agent execution and high-fidelity trace generation.

• Architect and build high-reliability execution harnesses for dynamic agentic trajectories on Apple hardware (iOS, macOS, watchOS, visionOS).
• Implement lightweight event-capture layers to record high-fidelity execution traces (tool inputs/outputs, state mutations, runtime events).
• Build CLI tools, local automation scripts, and integration hooks for ML teams to run on-device agent harnesses.
• Optimize harness performance and resource overhead without interfering with on-device AI runtime behavior or memory profiles.

• Strong programming proficiency in Swift, C++, or Objective-C.
• Scripting and automation skills in Python.
• Solid computer science and systems fundamentals, including process management and local runtimes.
• B.S. or M.S. in Computer Science, Machine Learning, or equivalent experience.