Responsabilities:

Design and execute test strategies for machine learning models (classification, regression, NLP, CV, etc.).
Validate model accuracy, fairness, bias detection, explainability, and robustness under varying datasets.
Perform adversarial testing and edge-case validation for AI systems.
Test autonomous agents, multi-step reasoning paths, and state transitions.
Validate correctness, grounding, consistency, and safety of LLM outputs.
Evaluate prompt robustness and behavioral variations across scenarios.
Validate retrieval accuracy, grounding quality, and hallucination reduction.
Test vector store behavior, document chunking logic, and retriever configurations.
Validate fallback behaviors when tools or external services fail.
Execute adversarial, prompt-injection, and red team style testing.
Validate compliance with data privacy, business rules, and insurance guidelines.
Benchmark LLM latency, throughput, and multi-agent performance.
Validate concurrency handling, degradation logic, and retry mechanisms.
Document test cases, behavior maps, prompt variations, and evaluation reports.
Work closely with AI engineers and SMEs to refine agent workflows.
Functional & Non-Functional Testing:

o Conduct API testing for AI services and model endpoints.

o Validate performance, scalability, and latency of AI inference services.

o Perform security testing for data pipelines and model-serving endpoints.

Quality Governance & Compliance:

o Ensure compliance with AI ethics, data privacy, and regulatory standards.

o Establish KPIs for model quality (precision, recall, F1-score, drift detection).

o Document test strategies, results, and provide audit-ready evidence.

Collaboration & Leadership:

o Work closely with data scientists, ML engineers, and DevOps teams to integrate testing into the AI lifecycle.

o Mentor junior testers and contribute to building reusable test accelerators.

o Represent the testing function in client discussions, providing insights on AI quality assurance.

Develop automated test frameworks for AI/ML pipelines.
Implement synthetic data generation for edge-case testing.
Conduct bias and fairness testing across diverse datasets.
Validate monitoring dashboards for model drift and performance degradation.
Perform regression testing for retrained models.
Create reusable test assets for AI projects (scripts, datasets, frameworks).
Provide detailed defect analysis and SLA-driven closure.

About the project:

A propos de CYCLAD:

Cyclad est une entreprise spécialisée dans le recrutement de profils qualifiés dans le domaine de l’informatique. Nous sommes présents en France, en Pologne et au Maroc. Nous misons sur la transparence et la proximité avec les experts qui nous rejoignent.

Requirements:

Experience : 6 - 8 years

Technical skills:

Strong experience in software testing (manual + automation) with focus on AI systems.
Hands-on expertise in Python for test automation, data validation, and AI testing scripts.
Knowledge of Gen AI frameworks (Langchain, Langgraph, CrewAI).
Familiarity with CI/CD pipelines (Jenkins, GitHub Actions, GitLab CI).
Strong understanding of data validation, model evaluation metrics, and drift detection.
API testing experience (REST, GraphQL) with tools like Postman or REST Assured.
Solid knowledge of cloud platforms (AWS, Azure, GCP) for AI deployments.
Exposure to AI ethics and compliance testing.
Knowledge of containerization & orchestration (Docker, Kubernetes).
Experience with Big Data ecosystems (Spark, Hadoop) for large-scale AI testing.
Familiarity with observability tools (Prometheus, Grafana, ELK stack).
Understanding of security testing for AI pipelines (data poisoning, adversarial attacks).

Senior Test Engineer- AI Testing