About the job
Job Title: QA Engineer – Generative AI Testing
Location: Hybrid – Hyderabad / Visakhapatnam
Job Type: Full-time
Job Summary:
We are seeking a skilled and detail-oriented QA Engineer with expertise in Generative AI Testing to join our team. In this role, you will be responsible for ensuring the accuracy, safety, and performance of AI-driven applications, particularly in the field of Generative AI. You will design and execute test strategies, develop automation frameworks, and validate AI outputs to maintain high-quality user experiences.
Key Responsibilities:
- Design, develop, and execute comprehensive test plans for Generative AI models and applications.
- Validate AI-generated outputs for accuracy, coherence, bias, ethical considerations, and alignment with business requirements.
- Develop and maintain automated testing frameworks for AI-based applications, ensuring scalability and efficiency.
- Perform adversarial testing, edge case analysis, and security testing to assess AI vulnerabilities.
- Collaborate with AI/ML engineers, product managers, and developers to refine model performance and mitigate errors.
- Monitor AI model drift and ensure consistent performance across different inputs and datasets.
- Conduct performance testing to evaluate response times, scalability, and reliability of AI systems.
- Document test cases, test results, and defects while ensuring compliance with AI governance and ethical guidelines.
- Continuously explore new tools and methodologies to improve AI testing processes.
Requirements:
- Bachelors or Master’s degree in Computer Science, Engineering, or a related field.
- 3+ years of experience in Quality Assurance, with at least 1-2 years in AI/ML testing or Generative AI applications.
- Strong understanding of AI/ML concepts, NLP models, LLMs, and deep learning architectures.
- Experience with AI testing tools such as LangTest, DeepChecks, Trulens, or custom AI evaluation frameworks.
- Proficiency in test automation using Python, Selenium, PyTest, or similar frameworks.
- Familiarity with model evaluation metrics such as BLEU, ROUGE, perplexity, and precision-recall for AI-generated content.
- Knowledge of bias detection, adversarial testing, and ethical AI considerations.
- Experience working with APIs, cloud platforms (AWS, Azure, GCP), and MLOps practices.
- Strong analytical and problem-solving skills with attention to detail.
- Excellent communication and collaboration skills to work in cross-functional teams.
Nice to Have:
- Experience in testing AI chatbots, voice assistants, or image/video-generating AI.
- Knowledge of LLM fine-tuning and reinforcement learning from human feedback (RLHF).
- Exposure to regulatory and compliance frameworks for AI governance.
Why Join Us?
- Opportunity to work on cutting-edge AI products.
- Collaborate with a team of AI and software experts.
- Competitive salary, benefits, and career growth opportunities.
If you’re interested, Please share your resume with Dkadam@eprosoft.com
