You cannot easily predict what type of product you’ll develop and test next. A lot of people are used to test install-able mobile apps, but when I joined GetSocial I couldn’t find anything on the internet about SDK testing. So let me share some insights about being a QA in a product which is not final.
Common testing issues for usual mobile apps and SDK
- Fragmentation hell. I mean all mobile fragmentation: OS versions diversity, screen resolutions, architecture differences, a zoo of mobile browsers etc.
- Network-related problems. Our SDK works only with an internet connection, so I had a lot of fun testing switch between Wi-Fi and 3G, walking between high buildings in a rush hour, using our demo app in an elevator etc. BTW, this book by J.Kohl has an amazing set of ideas.
Specific SDK testing issues
- Long delivery of new versions and hotfixes. We are shipping binary packages and customers can take weeks to update to the recent version…and then upload their app to the App Store/Google Play… and it still doesn’t mean that all the app’s end users will immediately upload the most recent update. Probably all of you heard about the exponential growth of costs of fixing a bug in different project phases. In case you are a service developer which clients are other developers with their own software products these costs are multiplied by 10.
- Rely on third-parties. One of our biggest features provides an ability to share app invites via a dozen of invite channels (messenger apps and social media). For some of these channels, we needed to write plugins, as integration wasn’t that easy, especially for Unity platform. All these integrations mean that we need to keep all the integrations up-to-date. Some updates of these external parties are unexpected and tricky, e.g. this Facebook app invites bug, which looked like our bug (I even created a ticket for it) until a dev send me the bug link and asked to upvote it. BTW, FB app invites were deprecated few months ago.
- Don’t confuse demo app bugs with SDK bugs. It was a serious issue at the beginning of my work as a QA in this project. We have few demo apps (at least one app per platform) with basic UI and representation of all features our SDK can provide. And sometimes it’s very easy to confuse an issue which is present only in our demo app and doesn’t mean there is a bug in SDK. Of course, such issues have less priority and could be ignored at all.
- Some types of testing usual for installable mobile apps are not needed in SDK. “Features not to be tested” is a common term in test planning, but here I mean testing types which could be fully omitted. It happens because we do not provide UI for all our features. Most part of UI and accessibility testing will be related to our demo app and not helpful with end apps which integrated our SDK.
- What you really need to test is integration process with your SDK. Even if your SDK is perfect, it doesn’t mean people will integrate it correctly. We are constantly working on making our integration as easy as possible (for example, we have a Gradle plugin for automatic integration with Android). But still, integration is one of the most common support requests. As a QA you could try to integrate your SDK on your own and then rise any questions that appear during integration. Even better, ask other devs to integrate it and make an interview.
Tickets to EuroSTAR 2019 Now Available – Book Now and Save!
Your testing is far from “the end”
Even if a particular integration with your SDK was perfect, you still cannot guarantee that everything will work as expected, as each app is unique and could have its own integration issues. What can you do with that? Ask your clients for beta versions of their apps with your SDK and participate in user support even if it’s not your main job.
It’s not so scary
Even if you are a QA at a very unusual project and couldn’t find any examples of how to deal with it, relax: general best practices of testing will be applicable to your case. Just carefully think of how to use the wisdom created by other professionals and add your own observations. After that share your own insights with others!
