hireejobs
Register for Job
Hyderabad Jobs
Banglore Jobs
Chennai Jobs
Delhi Jobs
Ahmedabad Jobs
Mumbai Jobs
Pune Jobs
Vijayawada Jobs
Gurgaon Jobs
Noida Jobs
Oil & Gas Jobs
Banking Jobs
Construction Jobs
Top Management Jobs
IT - Software Jobs
Medical Healthcare Jobs
Purchase / Logistics Jobs
Sales
Ajax Jobs
Designing Jobs
ASP .NET Jobs
Java Jobs
MySQL Jobs
Sap hr Jobs
Software Testing Jobs
Html Jobs
IT Jobs
Logistics Jobs
Customer Service Jobs
Airport Jobs
Banking Jobs
Driver Jobs
Part Time Jobs
Civil Engineering Jobs
Accountant Jobs
Safety Officer Jobs
Nursing Jobs
Civil Engineering Jobs
Hospitality Jobs
Part Time Jobs
Security Jobs
Finance Jobs
Marketing Jobs
Shipping Jobs
Real Estate Jobs
Telecom Jobs

Gen AI Tester

Fresher   Bangalore, HyderabadHyderabad, Pune   19 Jan, 2026
LTIMindtree Limited
Job LocationBangalore, HyderabadHyderabad, Pune
EducationNot Mentioned
SalaryNot Disclosed
IndustryIT Services & Consulting
Functional AreaIT Operations / EDP / MIS
EmploymentTypeFull-time

Job Description

    GenAI TesterRole OverviewWe are seeking a Gen AI Tester who can test and validate enterprise grade RAG, Agentic AI and Generative AI applications on Azure This hybrid role combines hands on designing unit test cases along with rigorous testing and evaluation to ensure solutions are reliable grounded and accurate The ideal candidate will have strong technical expertise in Azure AI services Python frameworks and automated testing methodologies Candidate should also have hands on experience in evaluating matrices like Accuracy Correctness Faithfulness and other similar matrix in RAG based systems applicationsExperience Requirements
    • 5 years of overall experience in core IT and software development
    • 3 years of handson experience with Azure Cloud with Azure AI Services
    • 2 years of experience in Generative AI RAG LLMs Azure Open AI Python Lang Chain Lang Graph and related technologies in testing AI Agents
    Key Responsibilities
    • Test RAG bots Agentic AI bots and Gen AI applications using Azure Open AI Azure AI Search Python Flask Azure Functions HTTP Queue Blob Timer and APIM with secure configuration via environment variables
    • Test scalable data layers using Cosmos DB and Azure Storage Blob Queue Table optimize chunking embeddings and hybrid vector search pipelines
    • Validate prompts and tuning across system prompts retrieval prompts and tooluse prompts test guardrails to minimize hallucinations
    • Establish automated testing with pytest and RAG evaluation using Ragas faithfulness precision recall answer correctness
    • Ensure observability with Application Insights advanced logging and endtoend traceability
    • Test orchestrate workflows using LangChainLangGraph for stateful agent flows
    • Test production SLAs by tuning rate limits scaling AI Search tiers and optimizing cost performance
    Must Have Skills
    • 3 years of hands on experience with Azure Cloud with Azure AI Services, AI Search Azure Open AI Cosmos DB and Azure Storage
    • Proven experience with prompt engineering and tuning techniques
    • Expertise in test automation using pytest and evaluation with Ragas knowledge of grounding techniques hybrid search semantic ranking and context filtering
    • Hands on experience in using any testing tool to automate and validate APIs
    Nice to Have Skills
    • Experience with Lang Graph for agentic flows
    • Familiarity with CICD pipelines Azure DevOps GitHub Actions
    Preferred CertificationsMicrosoft Certified Azure AI Engineer AI102

Keyskills :
pythonazuretestinggenerationretrievalaugmentedagentsgenerative

APPLY NOW

Gen AI Tester Related Jobs

© 2019 Hireejobs All Rights Reserved