AI Operations Specialist at Bretton AI in San Francisco, CA

Summary: Bretton AI is the leading AI agent platform for financial services, trusted by companies like Robinhood and Gusto to automate critical work. The AI Operations Specialist will ensure AI agents are effective for banks and fintechs, focusing on quality monitoring, testing new features, and collaborating with engineering and customers. Responsibilities: - Conduct quality reviews of AI agents to ensure they meet the highest standards for compliance teams at banks and fintechs - Test new features and custom logic before they go live for customers - Analyze case data to identify patterns, issues, and opportunities for improvement - Work directly with engineering to understand and measure the impact of system changes - Learn state-of-the-art AI monitoring techniques including performance drift detection, human scoring pipelines, and dataset preparation - Support our largest customers by providing operational insights and recommendations - Partner with product and engineering to translate operational findings into platform improvements Required Qualifications: - 1–3 years of professional experience or equivalent internship experience - Strong analytical and critical thinking skills with exceptional attention to detail - Ability to review complex information and identify patterns or issues - Comfortable learning new technical systems independently - Strong written and verbal communication skills - Excited to work in a fast-paced startup environment Preferred Qualifications: - Exposure to financial services, fintech, or regulated industries - Experience at a fast-paced startup - Familiarity with coding/scripting — SQL, dashboards, quick scripts - Customer-facing experience in any capacity - Genuine interest in AI/ML systems and how they're deployed in the real world Required Skills: Analytical skills Important Skills: SQL, Customer-facing experience Nice-to-Have Skills: Learning new systems, Interest in AI/ML Benefits: Comprehensive healthcare, 401k matching, Commuter benefits, 15 days PTO + holidays, Unlimited sick days, Flexible leave options, Working late? We’ve got you covered with DoorDash and an Uber home