Playwright QA AIAtem Corp • United States

Playwright QA AI

Atem Corp • United States

2 days ago

Job type

Full-time

Quick Apply

Job description

Skills Mandatory : Playwright, Python , AI / ML, Agentic AI - (AI Projects with Agentic Automation)

Overview

We are seeking an innovative and experienced QA / Dev Automation Engineer to join our team, focusing on testing cutting-edge Artificial Intelligence (AI) projects. This role is pivotal in ensuring the quality, reliability, and performance of our AI models and applications, specifically utilizing an Agentic Test Automation Framework. The ideal candidate will have a strong background in software testing, deep knowledge of Python or similar languages, and hands-on experience in developing and deploying intelligent, autonomous test agents capable of exploring, simulating, and validating complex AI behaviors.

Key Responsibilities

Design and Develop Agentic Tests : Architect, build, and maintain an Agentic Test Automation Framework capable of autonomously generating test scenarios, interacting with AI systems (e.g., LLMs, predictive models, decision-making systems) as a user or another agent, and reporting outcomes.
Test Strategy for AI : Define and implement comprehensive QA strategies tailored for AI systems, focusing on areas like model drift, data integrity, fairness, bias, performance under load, and edge case identification.
Automation Framework Management : Extend and optimize existing automation infrastructure using languages like Python and standard QA tools (e.g., pytest, Selenium, Playwright), integrating specialized AI testing libraries.
Continuous Integration / Deployment (CI / CD) : Integrate the agentic testing pipeline into the CI / CD process to ensure continuous validation and immediate feedback on every build.
Debugging and Analysis : Investigate, reproduce, and diagnose complex issues found by the test agents, collaborating closely with AI Scientists and Software Engineers to propose solutions.
Metrics and Reporting : Develop and report on key quality metrics specific to AI projects (e.g., accuracy, latency, robustness, coverage of the state space explored by agents).

Required Qualifications

Experience : 3+ years of experience in Software QA, with at least 2+ years focused on test automation development.

Technical Skills :

Expert proficiency in Python (or a similar language) for automation and scripting.

Hands-on experience developing or working with intelligent / agent-based systems or Agentic Test Automation Frameworks (e.g., using frameworks like LangChain, custom multi-agent systems, or similar architectural patterns).

Solid understanding of CI / CD tools (e.g., Jenkins, GitLab CI, GitHub Actions) and version control (Git).

AI / ML Knowledge : Foundational understanding of Machine Learning (ML) concepts (e.g., model training, inference, data pipelines, common failure modes like overfitting / underfitting).

Testing Methodologies : Proven expertise in various testing types, including functional, non-functional, integration, performance, and security testing.

Problem-Solving : Strong analytical and problem-solving skills with an ability to reverse-engineer complex systems and anticipate AI behavior.

Preferred Qualifications

Experience testing Large Language Models (LLMs), Generative AI, or complex decision-making systems.

Experience with using / building MCP servers.

Familiarity with cloud platforms (AWS, Azure, or GCP) and containerization technologies (Docker, Kubernetes).

Background in data science, statistics, or formal verification.

Experience with synthetic data generation for testing purposes.

Create a job alert for this search

Qa • United States