Tooploox · eBay · Solvd

Turning evaluation research into operational systems

I worked in eBay’s Security Organization through Tooploox and Solvd, serving as the project’s technical lead on the Tooploox side. As the only research engineer in a team of Research Scientists, I translated research methods, papers and new ideas into systems that teams could actually use.

My work centred on the RAMP library and service within eBay’s existing AI Platform. I extended the existing evaluation tooling, connecting research methods with scalable execution, versioned evidence and an operational AI review workflow.

What I built

• Parallel evaluation on datasets with thousands of examples, including live model endpoints and configurable evaluation metadata.
• Automatic benchmark selection, coverage-gap detection and self-service registration of team datasets and evaluation runs.
• A reporting service, model leaderboards and intake dashboards that made results and evaluation history accessible to reviewers.
• Versioned evaluations combining LLM judges, classifier-based scorers and statistical tests, with experiment tracking in MLflow.
• Automated red-teaming using DeepEval and DeepTeam, alongside specialist security work performed by a partner engineer.

Evaluation scope

I worked on evaluation of text, vision and vision-language models, with benchmarks for harmful content, bias, security and responsible model behaviour. The work connected model execution, scoring, experiment history and the interfaces used to inspect results.

Architecture and engineering decisions

I built and deployed backend and frontend services on Kubernetes and used the internal platform for evaluation workloads. I migrated orchestration from a Ray-actor DAG to asyncio where that better matched the workload. Model evaluation ran through the platform’s existing cluster and experiment infrastructure.

For small-scale embedding needs, embeddings were computed on demand rather than introducing a vector database. These choices kept the architecture aligned with the actual workload and avoided unnecessary infrastructure.

Python · FastAPI · asyncio · Kubernetes · Ray · MLflow · DeepEval · DeepTeam · Azure

Sharing the work

I presented RAMP at eBay AI Week in San Jose and explained the implementation of a collaborator’s work on operationalizing Responsible AI policies with LLMs.