Mallika Fernandes

Accenture

With increasing adoption of AI in software and cyber-physical systems, the system behaviour is rapidly evolving from a rule driven response to intelligence driven response. Such a response is dynamic and data driven and is never deterministic thus rendering the conventional testing and monitoring paradigms ineffective. Moreover, since the system behaviour is autonomous at run time, there are ethical, transparency, regulatory and compliance issues that need to be validated, monitored and assured before the system is deemed fit for production

In this webinar, Mallika explains the business and technical challenges in developing, testing and continuously monitoring AI driven systems and a proposed solution

The Challenge – Conventional testing has been primarily focused on creating test cases based on requirements and executing them to detect bugs. These requirements are converted to ‘Test Oracles’ which in turn are used to detect incorrect behaviour and report bugs. This process assumes that the system behaviour is rule based and deterministic.

For AI based systems, this testing approach is ineffective because the non-deterministic, dynamic, data driven response of AI based systems have the following implications:

• The system decision and response may have inherent bias (gender, race etc.), safety and fairness issues and may also potentially violate regulatory and compliance requirements.
• System response may be misaligned to business intent and may cause adverse business impact
• Diagnosis of an observed failure is not easy or straightforward

Proposed Solution – Consists of a recommended AI Testing life-cycle and various techniques –

• To handle training data debiasing
• AI Model validation and evaluation (post training)
• New testing techniques for AI models

Key Takeaways:

1. Understand industry challenges when adopting Artificial Intelligence
2. How traditional testing techniques are changing and what does a new AI Testing lifecycle look like
3. Techniques to assure the quality of AI systems

Do you love learning about software testing and improving your testing? Online learning is the first step of your journey with the EuroSTAR Community – the longest established and widest testing community in Europe. Join Huddle as a member a receive unlimited FREE learning resources.

 Want to learn in person? Come meet the EuroSTAR Community at our annual conference!