Search
Go
On-Device AI Inference Engineer
Summary by AIEngineer.careers
On-Device AI Inference Engineer is a AI Infrastructure Engineer role (full-time). with Hark. in SAN JOSE, US. Compensation shown: $200K–$450K. Imported listing (source: zerogtalent.com). Apply on the employer's site (zerogtalent.com).
Imported listing (source: zerogtalent.com) · Apply on zerogtalent.com
SAN JOSE · US
$200K–$450K
Full-time
SENIOR
The sections below reproduce the third-party job description for reference. AIEngineer.careers does not write or control this text.
Imported job description
Sourced from zerogtalent.com
Hark is seeking a Senior On-Device AI Inference Engineer to optimize transformer workloads for next-generation hardware. You will write low-level kernels and runtime paths to ensure models run efficiently within strict latency, power, and memory budgets on constrained silicon like DSPs and NPUs. This role bridges the gap between model architecture and physical device constraints.
Responsibilities
- Write and optimize low-level kernels and runtime paths for transformer workloads on target silicon.
- Manage memory residency, scheduling, and swap behavior for concurrent multi-model execution.
- Profile models on real hardware to identify bottlenecks and close performance gaps.
- Implement INT8/INT4 quantization to meet size and latency budgets.
- Collaborate with hardware teams to support new accelerators and feed deployment constraints back to model teams.
Requirements
- 4-8+ years of experience writing performance-critical software with optimization on GPUs, NPUs, or DSPs.
- Strong proficiency in C/C++, SIMD, custom kernels, and memory layout optimization.
- Deep understanding of attention mechanisms, KV-cache behavior, and transformer inference bottlenecks.
- Experience reasoning about compute, memory, and power budgets in constrained environments.
Hark
San Jose · US · 70+ employees
Hark is an artificial intelligence company focused on building proactive, multimodal personal intelligence systems that integrate foundation models with bespoke, AI-native hardware. Founded by serial entrepreneur Brett Adcock, the company aims to create a universal, seamless interface between humans and machines that can see, listen, and reason with persistent memory.
