Data Scientist - Agentic AI Evaluation

About Cognizant

Cognizant is a global leader in information technology, consulting, and business process services dedicated to empowering major enterprises worldwide. Headquartered in Teaneck, New Jersey, it combines a commitment to client satisfaction, technology innovation, and domain expertise with a diverse and collaborative workforce, shaping the future of work. Recognized among NASDAQ-100, S&P 500, Forbes Global 2000, and Fortune 500, Cognizant is renowned for strong performance and rapid growth.

Company Culture

At Cognizant, your passion, integrity, and expertise are vital to our collective success. Join a dynamic global IT and business consultancy where your individuality is valued, and client partnership is central. You will find ample opportunities to advance your career in a supportive, diverse environment that fosters collaboration and innovation. We actively promote a multicultural and gender-diverse workforce, ensuring equitable opportunities based on merit and accomplishment.

Role Summary

We are looking for an experienced Data Scientist specializing in Agentic AI Evaluation to develop and implement assessment frameworks for conversational and autonomous AI systems focused on commerce and customer engagement. This role involves measuring agent performance on task completion, tool accuracy, dialogue coherence, recommendation quality, hallucination rates, and customer satisfaction metrics.

Key Responsibilities

• Develop comprehensive evaluation frameworks for agentic AI technologies including conversational shopping assistants and autonomous agents, tracking metrics like task success rates and multi-turn dialogue consistency.
• Create both automated and human-in-the-loop evaluation processes using Azure AI Foundry, leveraging techniques such as LLM-as-judge and statistical testing.
• Establish and monitor quality indicators specific to commerce agents, including recommendation relevance, hallucination occurrence, task completion, escalation accuracy, and proxies for customer satisfaction like CSAT and resolution rates.
• Design adversarial testing strategies and red-teaming protocols to challenge agent robustness against prompt injection, jailbreaks, and complex e-commerce scenarios such as returns and fraud.
• Implement regression testing suites to detect quality variations across model updates, prompt changes, and knowledge base modifications.
• Examine live production telemetry to uncover failure modes, biases, and fairness concerns and translate findings into improvements in prompt engineering, fine-tuning, and retrieval augmented generation (RAG) enhancements.
• Partner with AI Engineers and Solution Architects to integrate observability tools (logging, tracing) and embed evaluation gates in MLOps and CI/CD pipelines.
• Communicate analytical insights and risk evaluations clearly to technical and business teams to guide release decisions and support responsible AI governance and compliance.

Required Experience

• 5 to 8 years in data science or machine learning, including 2 to 3 years dedicated to evaluating production-scale LLM-based or agentic AI systems, preferably in retail, e-commerce, or customer conversational AI environments.

Employee Benefits

• Engage with top industry talent and remain at the forefront of emerging AI and technology trends.
• Opportunities for career progression along innovative tracks.
• Access to professional growth through structured education and training programs.
• Support for maintaining health and future planning via comprehensive compensation, benefits, and wellness offerings.

Additional Information

Applicants are encouraged to connect with our welcoming recruitment team to express their interest and apply for this full-time position. We are eager to support you in achieving your professional goals within our expanding organization.