Quality Engineering Lead – Agent Evaluation & AI… at Comcast · Mount…

Quality Engineering Lead – Agent Evaluation & AI Platforms Location: NJ - Mount Laurel, 1800 Bishops Gate Make your mark at Comcast -- a Fortune 30 global media and technology company. From the connectivity and platforms we provide, to the content and experiences we create, we reach hundreds of millions of customers, viewers, and guests worldwide. Become part of our award-winning technology team that turns big ideas into cutting-edge products, platforms, and solutions that our customers love. We create space to innovate, and we recognize, reward, and invest in your ideas, while ensuring you can proudly bring your authentic self to the workplace. Join us. You’ll do the best work of your career right here at Comcast. (In most cases, Comcast prefers to have employees on-site collaborating unless the team has been designated as virtual due to the nature of their work. If a position is listed with both office locations and virtual offerings, Comcast may be willing to consider candidates who live greater than 100 miles from the office for the remote option.) Job Summary We are seeking seasoned Quality Lead (SDET 4) to orchestrate our enterprise release cycles and lead the quality strategy for our next-generation AI initiatives. This role is designed for a strategic technical leader who combines deep Comcast domain expertise across all 4 Lines of Business (LOBs) and Device Provisioning with specialized experience in Agentic AI validations. You will lead the quality process from the front: running scrums, managing the end-to-end defect lifecycle, conducting structural code reviews, and serving as the principal architect for our Agent Evaluation strategies within Google ADK environments. The AI Agentic Team designs, builds, and runs intelligent agents end to end. We own agent logic, workflows, integrations, production deployment, and ongoing performance. By combining build and operations, we deliver agentic systems that are reliable, secure, and scalable in real-world use. Job Description High-Impact Leadership & Process Ownership Agile Governance: Run daily Scrums and stand-ups, optimizing team velocity, managing task assignments, and ensuring delivery milestones are met on time. Defect Triaging: Lead rigorous defect triaging meetings. You will be the primary driver tracking high-priority issues, coordinating directly with development teams, and pushing defects to rapid closure. Requirement & Architecture Inception: Embed yourself in upstream requirement meetings. You will challenge assumptions, identify missing edge cases, and establish the technical "Definition of Done." Business & UAT Orchestration: Act as the strategic partner to Business teams, leading User Acceptance Testing (UAT) and translating complex technical deployments into business-ready validation. AI Governance & Agent Evaluation Strategy Agentic AI Expertise: Bring at least 2+ years of hands-on experience in Agentic AI validations, navigating the non-deterministic nature of large language models and intelligent workflows. Evaluation Frameworks: Drive the strategy for Agent Evaluation techniques, ensuring the team effectively implements automated verification patterns (such as LLM-as-a-judge) within Google ADK environments. Prompt Engineering Oversight: Apply deep knowledge of Prompt Engineering to validate agent behavior, test fallback mechanisms, and ensure intent optimization. Quality Governance & Code Reviews: Conduct technical Code Reviews for the team's automation repositories. Ensure that Python or TypeScript test scripts targeting the AI layers are modular, efficient, and scale effectively. Comcast LOB, Provisioning & Quality Governance Full LOB Coverage: Apply deep operational knowledge across all 4 Comcast LOBs (Xfinity Video, High-Speed Data, Digital Voice, and Xfinity Mobile) to ensure cross-functional workflows are flawlessly validated. Device Provisioning Master: Leverage an intricate understanding of backend Device Provisioning pipelines, activation flows, and equipment states to isolate complex environments and system-level issues. Automation Oversight & Code Reviews: Maintain strong technical oversight of the team's codebase. You will possess a solid understanding of test automation methodologies and basic knowledge of testing frameworks to conduct structural Code Reviews, ensuring the automation scripts meet quality, maintainability, and scalability standards. E2E Strategy & Mentorship: Guide junior engineers in designing comprehensive Test Plans and Strategies. You will define what to test and how to test it, ensuring total coverage of complex subscriber journeys. Execution, Observability & Reporting Technical Troubleshooting: Navigate Kibana logs and system metrics independently to perform deep-dive root cause analysis on failures before handing them off to development. Custom JIRA Architecture: Build and manage advanced JIRA dashboards to visualize real-time quality trends, escape-defect rates, and overall sprint health for leadership. Executive Reporting:Prepare and present Daily and Weekly Status Reports to stakeholders, providing absolute transparency on risks and release readiness Required Experience & Skill Set Leadership Track: 10+ years of experience in Software Quality Engineering, with a seasoned background in managing large-scale delivery and leading functional testing strategies. AI Specialization: 2+ years of specific experience with Agentic AI validations, Prompt Engineering, and Agent Eval techniques (specifically with Google ADK or similar models). Comcast Domain Mastery: Deep, demonstrable knowledge of all 4 Comcast LOBs and their underlying technical architectures. (asIS) Provisioning Infrastructure: Comprehensive understanding of the functional lifecycle for device provisioning, including complex billing system (ACSR and AMDOCS) handoffs and activation workflows. Automation & Framework Literacy: Strong functional understanding of automation frameworks (e.g., Python, TypeScript), providing the technical insight necessary to oversee code quality and ensure automated suites align with business requirements. Process Tooling: Power-user proficiency in JIRA for advanced dashboarding and Kibana for deep-dive log analytics to drive data-driven quality decisions. Skills Adaptability, Agentic AI, AI Agent Orchestration, AI Platforms, Critical Thinking, Test Automation Compensation Primary Location Pay Range: $135,683.35 - $203,525.03 Comcast intends to offer the selected candidate base pay within this range, dependent on job-related, non-discriminatory factors such as experience. The application window is 30 days from the date job is posted, unless the number of applicants requires it to close sooner or later. Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work.    Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus.  Additionally, Comcast provides best-in-class Benefits to eligible employees.  We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most.  That’s why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality – to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary on our careers site for more details. Education Bachelor's Degree While possessing the stated degree is preferred, Comcast also may consider applicants who hold some combination of coursework and experience, or who have extensive related professional experience. Certifications (if applicable) Relevant Work Experience 7-10 Years Comcast is an equal opportunity workplace. We will consider all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability, veteran status, genetic information, or any other basis protected by applicable law. Additional Locations: PA - West Chester, 1354 Boot Rd