Seven-plus years in test automation and quality engineering, most recently building the evaluation infrastructure that tells us whether an LLM system is actually getting better.
I own the evaluation framework for an internal order-operations assistant used by support operations and QE. It scores six dimensions in Python, keeps five of them deterministic, runs a 600-item benchmark with derived golden answers, and gates the pipeline on the numbers that matter. Alongside it I maintain MCP server integrations for agent tooling and automate backend validation across 30+ service clients over gRPC and REST.
Owned test strategy and documentation from acceptance criteria through automation, drove sanity, functional, and system testing on AWS across a three-person execution team, and automated microservice workflows with RestAssured and Java. Built and managed CI/CD pipelines that cut deployment time by 40%, plus database validation via GraphQL and Postman, load testing in JMeter, and security regressions in Jenkins.
Built UI regression and smoke automation with Selenium WebDriver that cut testing time by 40%, and a Cucumber-and-Java framework for back-end and database testing with RestAssured covering RESTful APIs over HTTP and HTTPS. Added a validation utilities layer for config-driven API tests — request/response comparison, schema-level assertions, expected-versus-actual checks — and maintained the requirements traceability matrix in JIRA.
If ground truth is computable, compute it. A judge model is the last resort for free-text judgment, not the default scorer — and when it is used, it gets calibrated against the mechanical scorers and reported with its agreement rate.
One sample tells you almost nothing. Multi-sample runs, pass-rate scoring, and a tracked consistency metric separate an unstable item from a real regression, so nobody debugs a model change that never happened.
An all-or-nothing gate teaches a team to re-run until green. Tiered gates — hard failure on the dimensions that matter, warnings on the rest — keep the signal credible and keep it read.
Grouped by what it does rather than by logo, because the interesting question is never which tool, it's which layer of the quality problem it answers.
Deciding what "correct" means for a system that doesn't return the same answer twice.
The tool layer between a model and the services it calls, tested on its own terms.
The traditional half of the job: contracts, UI, data, and everything that has a known right answer.
Where the numbers land, and who has to read them before a release goes out.
Open to new challenges and collaborations grounded in real human needs.