This is the third part of a series of posts on How to Do Performance Testing. Read the previous episodes of the series on Setup of Performance Testing (Part 1) and on Virtual Machines and Performance testing plan (part 2).
————————————————————————————————————————————————
Back in 2012, RES developed a server component, initially on Windows and later also on Linux. In the previous blog, I explained this triggered us to consider virtualization (VMWare ESX, Microsoft Hyper-V) and cloud (Microsoft Azure) as an alternative for using physical machines in a separated on-premises physical network: We simply needed more space to run tests, more isolation not to break down running tests while doing maintenance, debugging and developing new tests.
How to Do Performance Testing
It appeared to be not that easy. Today’s story covers 2013 and 2014, how we learned to clone our physical automated performance test lab into Azure.
Starting in 2013, I began to study how to implement what I’d learned in the Azure workshop I attended in the previous year. The typical way to connect to the cloud is to set up a VPN to extend part of the corporate network. However, in our corporate network, the subnet used for automated performance testing was isolated:
- No on premises IP Subnet with the associated IP router that could be used to be an Azure VPN termination point.
- No AD/DNS (sub)domain that could be extended to a read-only-domain-controller in Azure.
I decided not to use a VPN, and rather take an easier approach: Make a duplicate of the existing automated performance tests setup. To make this happen, I needed the following in the cloud:
- Isolated virtual network (VNet) and subnetwork (Subnet) (that’s easy);
- AD/DNS on this VNet/Subnet (not possible via Azure Console).
PowerShell Alternative
I had learned in the Azure workshop to find a PowerShell alternative to use for any Console call (using Fiddler). That convinced me that, if something was not available in the Console, PowerShell was the way. (Well, remember PowerShell. Little did I know how important this scripting language was going to be).
Only in the summer of 2013 I got to spend some more time on this idea, and I figured out how to deploy the AD/DNS VM into Azure, and how to configure the other VMs to find their DNS and AD, and how to have them join the Windows Domain. This was my starting point of setting up a duplicate of the physical lab: I now had a Virtual Network in Azure using (almost) the same IP addresses, the same Active Directory and DNS configuration. This is what I learned:
- The way IPv4 addresses work in an Azure virtual network: The first address is always .4, addresses increment when turning on VMs, (only later found a way to use static IP addresses (which was not possible from the old Console);
- To configure a VM to register to the Active Directory Domain;
- That IP Multicast did not work, I worked around that for our Automation product: Configure the Agents with the Server IP address;
- To work around an issue with leftover virtual network interfaces after a VM reboot: After about 100 times, the redirector stopped providing access to network shares;
- How to open 4018/4019 ports to remotely debug an application in Azure;
- How to use the digital certificates.
Having the Azure virtual network setup ready, I could copy (PtoV) all physical systems to Azure and power them on as Azure VMs:
- One VM running the Active Directory/DNS, File Server, SQL Server Database, automation product, aka the ‘Control server’, which was the one to start first so that it’s fixed on the .4 address.
- A Couple of Device Under Test systems (at this point we used VMs) where our software was installed on and was tested (‘Servers’)
- 3 machines used to set up (RDP) sessions on the Device Under Test machines (‘Agents’)
The system to schedule and execute the tests was made of our own Automation product, with embedded:
- C# command line utilities;
- DOS batch scripts;
- To turn on/off VMs etc. in Azure, PowerShell scripts;
and this was now running in the Azure duplicate as well.
Using Clones
In the meantime, it became evident that I needed help, and after a long search, Frank Brouwers joined RES in February 2014. With Frank, we also received another Azure MSDN subscription. While he got familiar with RES, we picked up the task of getting the cloned labs setup more seriously, and in summer of 2014 we had 2 clones of the physical labs in place: One in my MSDN Subscription, one in Frank’s.
We used the clones to develop and maintain our tests in, and when done, copy the tests to the physical network. This means that we could now run tests in the physical lab during the daytime as well! So, we made some progress:
- Better isolation between development and production testing: Doing an enhancement did not typically break existing tests anymore.
- A bit more time available for running production tests in the physical lab, also during the day, however it was still not possible to run tests in parallel.
2014 ended happily: Of course, Automated Performance Testing was not the only thing colleagues were interested in in Azure, and our corporate IT department purchased an Enterprise Agreement in Azure for Christmas! It would provide us plenty of space for our lab clones in the year to come.
Lessons learned
Further to continually-exploring-things I mentioned in the previous blog:
- Put any idea on your separate Automated Test Development backlog.
- During the busier times, pick ideas from the backlog in small pieces. Explore them and document results on a WIKI (so other colleagues see where you’re moving towards and they can benefit from results).
- Find the less busy period in the year. For me it’s the summer holidays. This is the right moment you get time to implement items from your Automated Test Development backlog
The change from being on my own to working with a colleague, made the need for change and the direction to take much clearer.
But that’s for a next episode.
Read Part 4: Automated Performance Tests on Demand In The Cloud
