Quality Test Engineer

About the Role:

This position performs complex quality engineering and test design assignments for AI-driven and agentic solutions built on our Salesforce platform (2,000+ users) and adjacent integrated systems (Oracle Sales Cloud Incentive Comp, Gainsight, Monday.com, and others). This role owns the quality strategy for both traditional deterministic Salesforce functionality (Apex, LWC, Flow, integrations) and non-deterministic AI/agentic capabilities (Agentforce agents, prompt-driven automations, generative AI features), where "correctness" must be evaluated differently than in conventional software. This position mentors more-junior QA engineers, partners closely with developers and business analysts to shape testable requirements, and represents quality and responsible-AI considerations throughout the SDLC — from design through production monitoring.

Responsibilities:

• 5+ years of Software Quality Engineering / Test Engineering experience, with 2+ years testing enterprise Salesforce implementations
• BS Engineering/Computer Science or equivalent experience required
• Experience testing AI/ML-driven, LLM-powered, or generative AI features strongly preferred
• Salesforce certifications (Platform App Builder, and/or AI Associate/AI Specialist) a plus
• Experience in regulated, audited, or compliance-driven release environments a plus
• Interface with developers, business analysts, and product owners to finalize testable requirements and acceptance criteria for both deterministic and AI-driven features
• Write and review detailed test plans, test cases, and evaluation frameworks for complex system components, including AI agents, automations, and generative AI features
• Design and maintain evaluation suites (golden datasets, regression benchmarks, output-quality rubrics) for agentic and generative AI capabilities where traditional pass/fail testing doesn't apply
• Execute and automate functional, regression, integration, and performance testing across the Salesforce platform and adjacent systems (Oracle Incentive Comp, Gainsight, Monday.com)
• Identify, document, and drive resolution of complex defects, including subtle AI behavior issues (hallucination, drift, bias, inconsistent outputs)
• Partner with developers to shift quality left — reviewing designs, prompts, and agent configurations before build, not just after
• Establish and evangelize quality and responsible-AI standards across the squad, including guardrails for reviewing and validating AI-generated code and AI-generated outputs before production release
• Support release management, packaging, and compliance/audit requirements with appropriate test evidence and traceability
• Monitor production AI features for quality drift and performance degradation, and define escalation paths when thresholds are breached
• Train and mentor less-senior QA engineers on Salesforce testing methodologies, automation frameworks, and AI/agentic testing practices
• Keep abreast of new Salesforce platform releases, Agent force/AI feature developments, and emerging tools and techniques in AI quality assurance
• All other duties as assigned

Requirements:

• Advanced knowledge of software quality methodologies (Agile/Scrum, risk-based testing, shift-left testing) as applied to Salesforce release cycles and org governance
• Strong proficiency in Salesforce test automation: Apex unit/integration testing, LWC Jest testing, UI automation frameworks (e.g., Provar, Selenium, Playwright) for Salesforce
• Strong understanding of Salesforce data model, sharing/security model, and how data quality and permission structures affect both functional and AI feature testing
• Working knowledge of SOQL/SOSL for test data validation and querying, and of integration testing across REST/SOAP APIs, Platform Events, and middleware (e.g., MuleSoft)
• Hands-on experience testing AI-driven and agentic features, including:
  • Designing test approaches for non-deterministic outputs: evaluation rubrics, golden datasets, regression suites for prompts/agent behavior, and acceptable-variance thresholds
  • Testing Agent force agents, Copilot/Einstein-style features, and prompt-builder workflows for accuracy, grounding, hallucination risk, and adherence to business rules
  • Evaluating AI outputs for bias, data privacy, and responsible-AI concerns, and partnering with stakeholders to define acceptance criteria for "good enough" AI behavior
  • Using AI-assisted testing tools (e.g., AI-driven test case generation, self-healing test automation, AI code/PR review assistants) to increase test coverage and efficiency
  • Monitoring AI feature performance and drift in production, and defining feedback loops for continuous evaluation
• Strong knowledge of test-driven and behavior-driven development practices, and ability to shape acceptance criteria before code is written
• Ability to write and review detailed test plans, test cases, and evaluation frameworks for complex system components, including AI agent/automation designs
• Ability to triage and drive resolution of complex defects across Apex, LWC, Flow, integrations, and AI/agentic behavior
• Strong research skills, including staying current on Salesforce release notes, Agentforce/AI feature rollouts, and emerging AI quality-assurance practices
• Knowledge of the broader sales technology tool stack (CPQ/Incentive Comp platforms, Gainsight, Monday.com) and how quality risk surfaces across integrated systems
• Ability to interface competently with developers, business analysts, product owners, and platform admins to finalize testable requirements
• Good oral and written communication skills, including the ability to explain AI evaluation results, confidence levels, and residual risk to non-technical stakeholders.