Is Crowdsourcing Testing Good for Quality Assurance?

More organisations have considered using crowdsourcing testing to handle their quality assurance needs, but the approach presents a number of concerns.

IT professionals are consistently tasked with carrying out a litany of repetitive and monotonous processes. These routine duties can be seen as a waste of resources, as employees may be overqualified to carry out such simple and mundane jobs. Speaking with One Million by One Million’s Sramana Mitra, CrowdFlower founder Lukas Biewald explained that during his time as an artificial intelligence expert with Yahoo, he was assigned projects that required gathering numerous data sets and records. He found that with minimal labor costs, he could complete these projects much more quickly and with fewer headaches by crowdsourcing those duties.

Those same principles have been applied to quality assurance in recent years, as organisations pursue opportunities to reduce their operating expenses. As noted by Forbes’ Andy Greenberg, many of the world’s largest and most esteemed tech companies have launched bug bounty competitions, offering participants monetary rewards for finding flaws or vulnerabilities in their software.

“The way a company traditionally finds security issues is by hiring a consultant, and they get a report or presentation. Instead, we run a contest, everyone’s invited to find issues in the systems we’re testing, and if you find something and are the first to report it, we give you a cash reward and social recognition,” Casey Ellis, chief executive of start-up consultancy group Bugcrowd, told the news outlet. “Instead of a consultant who’s paid for his time, this is much more like how the bad guys are doing it. We invite smart people who can think like bad guys and are only paid when they find something.”

Crowdsourcing isn’t just used for identifying security concerns, either, as more organisations have turned to external users to find software flaws in their products that could affect functionality. The argument made by proponents like crowdsourcing software provider Passbrains is that this approach to quality assurance places the product in real-world conditions, providing a more user-centric view of its performance while reducing labour costs.

Disseminating Resources Could Backfire

However, crowdsourcing presents a number of challenges that should concern software developers and quality assurance professionals. Instead of leaning on a tight-knit team of test experts, businesses will need to entrust their QA duties to numerous individuals spread across the world. From the outset, maintaining consistent and direct lines of communication will be extremely difficult to achieve. As has been reflected by the proliferation of agile development principles, integrating QA personnel with the development process is crucial to keeping up with shortening production cycles while ensuring that a quality program is released.

Dispatching QA duties to part-time contractors dotted across the globe may present upfront budget savings, but the loss of consistent communication will be felt when developers and QA personnel are unable to synchronise their efforts and quickly make adjustments to the production process as needed. QA management officials can better organize their test resources by sticking with traditional and outsourced team structures. A high-quality test management system can bridge the gap between these groups by condensing quality assurance tasks within a single interface. These software testing solutions will provide team leaders with far greater oversight than a crowdsourcing approach ever could.

If QA teams want to make the most of crowdsourcing testing and resources without disrupting internal operations, it may be wise to only lean on outside assistance after internal testing is completed. This way, organisations can get a new set of eyes on a piece of software without introducing communication and collaboration hurdles that could derail a given project.

About the Author


As the VP of Client Service for Zephyr, Sanjay brings over 15 years of leadership experience in IT and Technical Support Services. Throughout his career, Sanjay has successfully established and grown premier IT and Support Services teams across multiple geographies for both large and small companies. Most recently, he was Associate Vice President at Patni Computers (NYSE: PTI) responsible for the Telecoms IT Managed Services Practice where he established IT Operations teams supporting Virgin Mobile, ESPN Mobile, Disney Mobile and Carphone Warehouse. Prior to this Sanjay was responsible for Global Technical Support at Bay Networks, a leading routing and switching vendor, which was acquired by Nortel. Sanjay has also held management positions in Support Service organizations at start-up Silicon Valley Networks, a vendor of Test Management software, and SynOptics.
Find out more about @sanjayzalavadia