Principal or Staff AI / ML Engineer

1 day ago Be among the first 25 applicants

Want to work on the next wave of AI? Join Wallaroo.AI and help us build the next generation of enterprise AI inference software!

Wallaroo.AI is on the hunt for a Principal or Staff AI / ML Engineer to lead the charge in pushing the boundaries of state‑of‑the‑art inference capabilities on the latest hardware. We’re looking for an expert with a passion for LLM deployment, inference, and lifecycle, coupled with a deep understanding of LLM architectures and cutting‑edge generative AI serving runtimes. Our mission at Wallaroo.AI is to enable inferencing on "any hardware," so if you thrive on mastering new technologies, diving deep, and extracting maximum performance, you’ll feel right at home.

Wallaroo.AI is a Microsoft‑backed startup at the forefront of AI, developing a powerful software platform that empowers AI teams to deploy AI inference at scale to any cloud, edge or on‑prem environment with unparalleled speed and efficiency. We have marquee enterprise customers and technology partners, including some of the most globally recognized brands. We’re poised for explosive growth in 2026 fueled by our go‑to‑market with leading AI hardware providers. This is your chance to be part of something truly transformative.

We’ve been fully distributed and remote‑first since before it was cool! This means we seek exceptional communicators who are self‑motivated and adept at both independent and collaborative work. We also need individuals who thrive in dynamic, fast‑paced environments. Successful Wallaroo.AI team members are comfortable experimenting with fresh ideas, taking initiative without constant oversight, adapting swiftly as priorities evolve, and bringing clarity to complex, emerging challenges. Prior startup experience or a similar high‑velocity environment is a significant advantage.

Responsibilities

At Wallaroo.AI, building a great product is a team endeavor. We expect active participation in discussions, proactive communication, and continuous feedback. You’ll partner closely with customers and hardware teams to solve problems and build capabilities. As an experienced engineer, you’ll negotiate tradeoffs with product managers, break down feature requests into incremental, high‑quality deliverables, and mentor less experienced team members. You’ll also propose process and workflow improvements, and confidently advocate for your perspective while thoughtfully listening to others.

Required Skills / Abilities (at least 4)

• Excellent verbal and written communication skills
• Expert Python, including extensive packaging experience using Pip, uv, etc
• Strong experience with vLLM, SGLang, and other modern serving frameworks
• Experience developing high‑scale production ML pipelines
• A strong foundation in deep learning architectures, especially the Transformer architecture, including experience with NLP tasks like text generation, summarization, and sentiment analysis using commonly available LLM APIs and SDKs (ie, OpenAI, Anthropic, Mistral, etc)

Desired Skills / Abilities

Education and Experience

Interview Process

Candidates are encouraged to interview us at each stage as well, to ensure that they have as much insight into the role and company culture as possible and allow them to make an informed decision. Every interview will include at least 10 minutes for the candidate to ask questions of the interviewer.

Benefits and Perks for US Employees

Referrals increase your chances of interviewing at Wallaroo.AI by 2x

Seniority Level

Employment Type

Job Function

Industries

#J-18808-Ljbffr