
Job Description
We are helping our client find exceptional Software Engineer - Technical Advisors who want to spend a focused period doing something most engineers never get to do: find out exactly where frontier models break on real engineering work.
You will take on hard problems across production-grade codebases, then work out precisely where and why model-generated solutions fall short. You will also build the hard problems those models are tested on. Your judgment about what separates good engineering from plausible engineering is the entire point of the role.
What you'll do
- Evaluate Agent Sessions: Analyze how AI coding agents behave over entire coding sessions—tracking what the agent investigates, verifies, assumes, and leaves undone.
- Review Model-Written Pull Requests: Audit model-generated pull requests against real production repositories, documenting every identified issue alongside its severity and detailed technical rationale.
- Build Evaluation Benchmarks: Design and construct hard, container-based problems that serve as rigorous test benchmarks for frontier models.
- Collaborate with AI Researchers: Work directly alongside AI researchers on frontier problems, producing clear written analyses that explain root-cause failures and boundary-condition gaps.
- Maintain High Written Rationale Standards: Author original, clear technical rationale for all evaluations (all written deliverables must be independently authored without AI text generation, though AI tools are welcomed for codebase exploration and running test suites).
Requirements
Requirements:
- 8+ years of production engineering experience preferred (exceptions only for clearly exceptional profiles).
- Backgrounds including Senior, Staff, or Principal Software Engineers, Tech Leads, or Open-Source Maintainers.
- Experience working in production codebases with a strict code review culture (startups, big tech, or open source are all acceptable).
- Polyglot adaptability: Heavy experience with Python and TypeScript is common, but must be comfortable dropping into unfamiliar languages weekly.
- Core Tooling: Demonstrated comfort using Docker, git, and the Command Line Interface (CLI) to reproduce, isolate, and debug results locally.
- Cross-Layer Fluency: Comfort working across backend, frontend, APIs, data, testing, or developer tooling.
- Strong written communication skills—able to articulate why code fails, not just how to fix it quickly.
Originally posted on Himalayas
Key Skills
Software-EngineerTechnical-AdvisorAI-EngineerSoftware-Dev.-Sr.-Specialist-AdvisorSenior-Technical-AdvisorSoftware-Engineering-Advisory
More Related Jobs
Marketing Engineer I
Ext
GOAT Group
FULL TIMEUSA · Remote
Software Engineering
2h agoApply Now
Staff Security Operations Engineer
Ext
Pleo
FULL TIMEDenmark, Germany, Netherlands, Portugal, Spain, Sweden, UK · Remote
Cybersecurity
2h agoApply Now
Account Executive 4
Ext
Twilio
FULL TIMEUSA · Remote
Sales
2h agoApply Now
District of Columbia (DC) Remote Mental Health Licensed Therapist, 1099 Contractor
Ext
Headspace
FULL TIMEUSA · Remote
Healthcare & Medical
2h agoApply Now