Thank you.

Your uploaded files are waiting for moderation.

<< Go Back

Quality Assurance of Artificial Intelligence Systems

Reading Time: 2 minutes

With increasing adoption of AI in software and cyber-physical systems, the system behaviour is rapidly evolving from a rule driven response to intelligence driven response. Such a response is dynamic and data driven and is never deterministic thus rendering the conventional testing and monitoring paradigms ineffective. Moreover, since the system behaviour is autonomous at run time, there are ethical, transparency, regulatory and compliance issues that need to be validated, monitored and assured before the system is deemed fit for production

In this webinar, Mallika explains the business and technical challenges in developing, testing and continuously monitoring AI driven systems and a proposed solution

The Challenge – Conventional testing has been primarily focused on creating test cases based on requirements and executing them to detect bugs. These requirements are converted to ‘Test Oracles’ which in turn are used to detect incorrect behaviour and report bugs. This process assumes that the system behaviour is rule based and deterministic.

For AI based systems, this testing approach is ineffective because the non-deterministic, dynamic, data driven response of AI based systems have the following implications:

  • The system decision and response may have inherent bias (gender, race etc.), safety and fairness issues and may also potentially violate regulatory and compliance requirements.
  • System response may be misaligned to business intent and may cause adverse business impact
  • Diagnosis of an observed failure is not easy or straightforward

Proposed Solution – Consists of a recommended AI Testing life-cycle and various techniques –

  • To handle training data debiasing
  • AI Model validation and evaluation (post training)
  • New testing techniques for AI models

Key Takeaways:

  1. Understand industry challenges when adopting Artificial Intelligence
  2. How traditional testing techniques are changing and what does a new AI Testing lifecycle look like
  3. Techniques to assure the quality of AI systems

Do you love learning about software testing and improving your testing? Online learning is the first step of your journey with the EuroSTAR Community – the longest established and widest testing community in Europe. Join Huddle as a member a receive unlimited FREE learning resources.

 Want to learn in person? Come meet the EuroSTAR Community at our annual conference!

Editor's Image

Mallika Fernandes

  • IT Leader with 20 years of extensive Innovation and Quality Engineering experience in the IT industry
  • Leads Artificial Intelligence in Testing. Expertise in application of Cognitive Computing and Machine Learning models in Quality Engineering.
  • Leads the Asset Development cluster for Testing, driving the latest automation innovations and AI powered tooling
  • Holder of patent for innovations in Quality Engineering.
  • Speaker at various International conferences like A2IC International AI conference 2018, Barcelona and O’Reilly AI conference 2018, NY
  • Project & Program management expertise – Test Strategy & Planning, Risk Management, Test Metrics & SLA definition and tracking, Unit of Work pricing models, Risk based testing, Automation ROI analysis, Knowledge Management

Leave a Reply

Skip to toolbar