Senior Technical Architect (DPO) at HCLTech in Romania

Job Summary

Agentic QA Solution Architect Quality Solution Architect for AI Agents & Agentic Automation About the Role AI agents are non-deterministic: the same input can produce different outputs across runs. Testing them requires a fundamentally different discipline than scripted test automation - evaluation frameworks, scoring functions, and statistical judgment instead of exact-match assertions. As an Agentic QA Engineer, you will bring that discipline directly to our customers. This is a forward-deployed, customer-facing engineering role. You will work side by side with enterprise customers to design, build, test, and harden agentic workflows in their live environments - combining the hands-on coding of a software engineer, the quality mindset of a test automation expert, and the end-to-end design perspective of a solution architect. You will own the question every customer asks before putting an AI agent into production: "How do we know we can trust it?"

Key Responsibilities

What You Will Do • Deploy and validate agentic solutions at customers. Work with enterprise customers to implement, integrate, and operationalize AI agents and agentic workflows in production environments - including APIs, data pipelines, and integration with existing customer systems. • Design testing agents and evaluation strategies for deterministic and non-deterministic systems. Define what "quality" means for each agentic use case and turn it into measurable signals: evaluation suites, scoring functions, regression baselines, and pass/fail criteria that account for variability in agent behavior. • Build and run evaluation frameworks. Implement automated evals (LLM-as-judge, semantic similarity, structured-output checks, human annotation rubrics) that track accuracy, latency, and cost across agent runs, and compare every release against baselines to catch regressions. • Perform adversarial and safety testing. Probe agents for failure modes, hallucinations, prompt-injection vulnerabilities, and out-of-scope actions; validate guardrails, human-in-the-loop checkpoints, and escalation paths before go-live. • Automate the testing lifecycle. Build and maintain automated test suites for both deterministic components (integrations, workflows, UI) and agentic components, wired into CI/CD so quality gates run on every deployment. • Act as a trusted technical advisor. Translate customer requirements into solution and test architecture; explain evaluation results, risks, and trade-offs to audiences ranging from QA engineers to non-technical executives. • Architect for scale. Apply solution-architecture judgment to design quality strategies that span the full solution - agents, RPA/workflow/tools automation, integrations, and human-in-the-loop steps - and that customers can operate and extend themselves. • Close the loop with product and engineering. Feed field findings - failure patterns, missing capabilities, evaluation gaps - back into the product roadmap and internal best practices, and contribute to reusable test assets and accelerators.

Skill Requirements

What You Bring (Required) • Software engineering foundation. 7+ years of hands-on software development experience in Python, C#, VB.NET, or comparable languages, with the ability to write production-quality code and test harnesses. • Testing and QA depth. 3+ years in software testing / test automation, including test strategy design, automated test frameworks, and CI/CD-integrated quality gates. • Hands-on experience with AI agents. Practical experience building, deploying, or testing LLM-based agents and agentic workflows (e.g., tool/function calling, RAG, multi-step orchestration, frameworks such as LangChain/LangGraph or the Model Context Protocol), ideally in customer or production settings. • LLM evaluation literacy. Working understanding of non-determinism, prompting, grounding, and hallucination, and experience setting up evals - including LLM-as-judge approaches, rubric-based scoring, and regression testing of model or prompt changes. • Customer-facing experience. Proven track record in a forward-deployed, professional services, implementation, or consulting role: leading technical workshops, managing stakeholder expectations, and delivering under real-world constraints in customer environments. • Solution architecture background. Experience designing end-to-end solutions across applications, APIs, and automation platforms - able to see the whole system, not just the component under test. • Communication. Excellent written and verbal communication; comfortable presenting technical findings and quality risks to executive stakeholders. Fluent English.

Other Requirements

What Sets You Apart (Preferred) • Experience with agentic testing or AI-augmented QA platforms (e.g., UiPath Test Cloud / Test Manager, or comparable AI-powered testing tools) and with automation platforms more broadly (RPA, workflow orchestration). • Experience with evaluation and observability tooling for LLM applications (e.g., LangSmith, Braintrust, Promptfoo, or custom pytest-based eval harnesses) and with tracing/monitoring agent behavior in production. • Statistical literacy - able to distinguish signal from noise in non-deterministic outputs and reason about sample sizes, baselines, and confidence. • Experience with test management and ALM tooling (e.g., ADO, Jira - Xray, TestRail, Test Manager) and requirements-to-test traceability. • Knowledge of AI governance, safety, and compliance considerations for deploying agents in regulated industries. • Degree in Computer Science, Engineering, or a related field, or equivalent practical experience. • Willingness to travel to customer sites as needed. How Success Is Measured • Customers move agentic use cases from pilot to production with documented, repeatable evaluation and test coverage. • Evaluation suites catch regressions before customers do - measured by escaped-defect and eval-coverage trends. • Customer stakeholders trust and understand the quality evidence behind every go-live decision. • Field learnings are converted into reusable test assets, patterns, and product feedback.