AI Performance Analyst at MRI Software · Manila

AI Performance Analyst Location: Manila, Philippines What You Will Do •    Define, track, and report on AI performance metrics: resolution rates, deflection rates, escalation accuracy, CSAT, and response quality •    Build and maintain AI performance dashboards and reporting infrastructure •    Conduct regular evaluations of AI agent outputs to identify accuracy failures, hallucinations, and behavior regressions •    Design and run controlled experiments (A/B tests) to evaluate the impact of AI changes on performance metrics •    Partner with AI Engineers and the AI Support Architect to diagnose performance issues and recommend interventions •    Work with Intent & Demand Analysts to connect intent data with AI resolution outcomes •    Produce regular performance reviews and present insights to Knowledge & AI leadership •    Develop evaluation frameworks and test sets for ongoing AI quality assurance •    Monitor AI performance trends over time and flag regressions or deterioration early What We Are Looking For Required Experience & Qualifications •    3+ years of experience in data analysis, experimentation, or AI/ML performance evaluation •    Experience measuring and evaluating AI or ML model performance in a production environment •    Proficiency in SQL and at least one BI or data visualization tool •    Experience designing and analyzing A/B tests or other controlled experiments •    Familiarity with NLP evaluation metrics (precision, recall, F1, BLEU, etc.) is a strong advantage •    Background in support analytics or AI product evaluation preferred Skills & Competencies •    Data analysis: SQL proficiency and experience with large datasets in support or AI contexts •    Experimentation: designing and running A/B tests and evaluating statistical significance of results •    AI metrics: understanding of AI performance indicators specific to conversational and resolution-based AI •    Dashboard and reporting: building clear performance dashboards in BI tools for technical and non-technical audiences •    Diagnosis: ability to trace AI performance failures back to root causes in knowledge, prompts, or model behavior •    Communication: presenting data-driven findings and recommendations to engineering, architecture, and leadership audiences Core Attributes •    Precision-focused and understands that AI performance measurement requires rigor, not approximation •    Proactive: identifies performance problems before they escalate and brings solutions •    Comfortable working at the intersection of data and AI system behavior •    Collaborative: works closely with engineers and architects without needing to be an engineer themselves •    Growth-oriented: motivated by the continuous improvement cycle that makes AI measurably better over time Benefits •    Career growth opportunities in a diverse and supportive work environment. •    Competitive Salary Package •    Prime office location in Ortigas (Easy access to MRT stations, restaurants, and banks) •    Hybrid Working Arrangement (2 days WFH) •    HMO coverage with free dependent upon hire. •    Free 24/7 access to our office gyms (Ortigas and Makati) •    Total of 20 annual leaves to be used on your own discretion (including 5 credits convertible to cash) •    Plus, additional benefits About Us From the day we opened our doors, MRI Software has built flexible, game-changing real estate software that powers thriving communities and helps make the world a better place to live, work and play. Fulfilling that mission is only possible because of one thing: exceptional people. People like you! Our people-first approach to PropTech is defining a new industry standard for client experiences that, quite frankly, can’t be duplicated. Experiences that deliver real value every day. And we know those experiences begin with our people. We believe MRI is more than just a workplace; it’s a connected community of people who truly feel they belong. Whether we’re investing in employee resource groups or providing tailored resources for each person to reach their full potential, we’re passionate about creating a work environment that makes you excited to show up every single day. At MRI, one of our core values is to strive to amaze.  From the intelligent solutions we create to the culture we cultivate, that’s our goal every day. Because that’s what industry leaders do. Whether you’re joining as a new Pride member or rejoining us after a short time away, your talent is vital to us, our partners and our clients. Amazing growth requires amazing employees. Are you up to the challenge? We know confidence gap and imposter syndrome can get in the way of meeting remarkable candidates, so please don’t hesitate to apply. We’d love to hear from you! MRI is proud to be an inclusive employer. We welcome and celebrate diversity across all backgrounds, including ethnicity, religion, sexual orientation, gender identity, disability, age, military, veteran status and more. We believe that Belonging is a direct result of Diversity, Equity, and Inclusion. Those values are woven into the fabric of who we are and are foundational to our continued success. Come and see for yourself!