AI Model Benchmarking Intern (Remote, Iowa-Based) Job at Braintrust, San Francisco, CA

VHVEdjB3eFBHR3c2RHk1aHhTblNITkJHV0E9PQ==
  • Braintrust
  • San Francisco, CA

Job Description

  • Rate: $22 – $28/hr
  • Hours: 20 hours / week
  • Experience: 1 - 3 years
  • Location: United States | Canada + 23 more

AI Model Benchmarking Intern (Remote, Iowa-Based)

Part-time, 15–20 hours/week · Remote, Iowa-based · 6-month internship · $22–$28/hour

About SureHost

SureHost is an early-stage hospitality technology company based in Des Moines, Iowa. We help growing short-term rental operators confirm each property is guest-ready after a cleaning turn, using documented property standards, photo and video evidence, AI-assisted review, and human review where it matters.

We are working with an initial paying customer and preparing for broader growth in the U.S. market. Our customer and product work happens in Des Moines, and our technical team is based in Pune, India. Our stack is Python and Flutter, deployed on GCP.

The role

SureHost uses AI narrowly, where it can be measured and trusted. Language and vision models change quickly, and a model that looks strong in a demo can still fail on real property footage, cost too much to run, or make decisions an operator cannot follow.

In this role, you will help us tell the difference. You will build repeatable benchmarks that compare LLMs and VLMs on SureHost use cases, and your findings will inform model choices and the product roadmap. You will work with our AI Chief Architect.

What you will do

  • Research and compare evolving LLM and VLM models that may support SureHost's product use cases.
  • Create representative evaluation datasets, test cases, scoring criteria, and repeatable benchmark workflows.
  • Measure model quality across accuracy, reliability, latency, cost, and other product-relevant factors.
  • Analyze failure patterns and trade-offs, including false positives and false negatives on issues operators care about.
  • Present clear findings and recommendations to inform model strategy and the product roadmap.
  • Maintain benchmark results and documentation as models, APIs, and requirements change.

What you bring

  • Interest in applied AI and familiarity with LLMs, VLMs, generative AI tools, or model APIs through coursework or projects.
  • Logical, evidence-based thinking and comfort working with structured data and evaluation criteria.
  • Basic Python or data-analysis skills.
  • The ability to document methods and results clearly.
  • Patience to run repeatable experiments, investigate edge cases, and separate a compelling demo from dependable performance.
  • Clear communication, willingness to ask questions early, and responsible handling of customer, property, and company information.

Required

  • Based in Iowa.

Details

  • Pay: $22–$28/hour, based on experience and how independently you can work.
  • Hours: 15–20 hours per week, with flexible scheduling.
  • Location: Remote, for Iowa-based candidates.
  • Term: 6-month internship.
  • Start date: Flexible. We are onboarding now and can work around school schedules, finals, and the holidays.
  • Works with: AI Chief Architect.
  • Team time: A monthly in-person check-in over coffee in Des Moines.

What you get

  • Real responsibility on a live hospitality technology product, with few layers between customer needs and product decisions.
  • Direct mentorship from the AI Chief Architect on applied model evaluation.
  • Work you can point to: benchmark methods, results, and recommendations to reference in future roles.

Company: SureHost

Job Tags

Contract work, Temporary work, Part time, Internship, Remote work, Flexible hours

Similar Jobs

Dan Cummins Chrysler Dodge Jeep RAM of Paris

Automotive Service Technician Job at Dan Cummins Chrysler Dodge Jeep RAM of Paris

Dan Cummins CDJR in Paris Ky...We are looking to hire 2 certified Chrysler technicians. **These are flat-rate pay positions based on experience, 100% Chrysler certified technicians that work on transmissions and diesels can earn 40.00+ per flat rate hour**Dealer will... 

Advantis Medical

Pediatrics Travel Nurse - Asheville, NC Job at Advantis Medical

 ...This Pediatrics travel nurse position in Asheville, North Carolina, with Advantis Medical is a 8-week Travel Pediatrics Registered Nurse (RN) contract at Mission Hospital . This role pays $2,115 per week and follows a Days shift with a 3x12 schedule. In this Pediatrics... 

The World

IT Technician - Immediately Job at The World

The World, Residences at Sea, is the largest privately owned residential yacht on earth.Employees of The World strive for excellence in everything we do.We live and work on a one-of-a-kind luxury yacht that sails continuously around the world.Every few days you will wake... 

Replit

Senior Site Reliability Engineer Job at Replit

 ...is democratizing software development by removing traditional barriers to application creation. ABOUT THE ROLE: Join our Site Reliability Engineering team and help ensure the reliability, scalability, and performance of Replit's infrastructure that serves millions of... 

Nebraska State Jobs

G&P Fish & Wildlife Biologist I Job at Nebraska State Jobs

 ...Parks Commission - Agency 33 Location: Valentine, NE Hiring Rate: $22.123 Job Posting: JR2026-00030215 G&P Fish & Wildlife Biologist I (Open) Applications No Longer Accepted On (If no date is displayed, job is posted as open until closed): 11-24-202...