Four Crucial Tips for Automating Your Web Tests

[Daragh]

At a recent webinar Jim Holmes offered four tips for automating web tests:

1. Test the right things

2. Understand good locators

3. handle asynchronous operations properly

4. Create a backing infrastructure to help you write better tests

View the webinar here

Do you agree? Or do you have an alternative set of tips?

[Richard Bradshaw]

ID’s are only unique if you make them unique.
Some (most) compilers won’t tell you if you repeat an ID in a page.

[Richard Bradshaw]

Using XPath to navigate up and back down the dom, yuck!
The example is OK there for finding an input via a label, but tying yourself to a label is dangerous, especially if you sites supports many languages and you want to run against many languages.
Or when perhaps labels get additional values because they, lets say are mandatory they get a *, your check will now fail.
Also if you are going to go down the label approach, consider making it case sensitive.

[Richard Bradshaw]

I wrote a TableControlObject available on GitHub for dealing with tables and WebDriver C#, http://www.thefriendlytester.co.uk/2012/12/table-controlobject.html

[Daragh]

Question from Karin Basel:

Could you explain more the developer support by divs in the case of concurrent ajax elements like on Amazon?

[Chris]

Richard, what;s your preferred method for traversing up and down the dom if not xpath?

[Richard Bradshaw]

Checking whether a value has saved via the UI in a DB for me depends on the context.
If you application shows you something has saved, ie. a message stating as such.
You need to focus on what your check is checking, if the UI shows you this messge the UI believes it to be saved. This can be enough.

I would suggest several check that checks how that success message is displayed which also check the DB is saved, preferable at a lower level then the UI.

[Jim]

@richard: Of course you don’t want to explicitly hard code in text to locators. I stated as much during the talk! Obviously you’d parameterize that label’s value so you can handle localization or other similar issues.

This example was demo code versus production code.

FWIW, I’ve used this label + input approach a number of times. It’s handy and simple.

As with all locators, teams need to figure out what works for this. This is just one example.

[Jim]

@richard: Most web frameworks handle ID generation (and deconfliction) automatically. Rails, ASP.NET WebForms, and a few others take ID creation out of your hands.

No, compilers won’t tell you about duplicated IDs, but this is also part of understanding how your toolset works.

[Richard Bradshaw]

Hey Chris,

My preferred method is to not do it. Being able to code and insisting I have access to the source control allows to me avoid having to do this, as I can go add ID’s or classnames of something unique to the element I was struggling to retrieve.

If I do have a to then XPath does work, or if you can’t alter the source. I have also tried to make this strong in the past with webdriver, by checking the element I move up to is as expected before moving down against to give me more confidence. Perhaps checking its the tag I was expecting.

[Tom]

Thanks, Jim. That was a really good presentation.

I was watching this with a colleague who is just getting started with test automation (specifically web automation with Selenium WebDriver). You managed to explain the concepts to her much better than I have been able to.

[Chaya]

As I Observed, Most of the time when telerik control changes, The corresponding locator not identifiable in application. Is it due to display properties changes at certain parts where telerik control changes happen. I faced these challenges. Anyhow I managed them using Javascript by simply removing display attribute

[Jim]

@Karen

Regarding handling concurrent AJAX/asynch calls: the devs who own that page should be able to determine when all calls on the page are complete. There are a number of different ways to handle this–it’s totally dependent on the platform stack being used.

The idea is once they’ve determined the calls are finished, they can add an empty/hidden element to the page, something like:

<div callsCompleted=”true” />

Now you can set an explicit wait for the presence of that div and you’ll know the various async operations are complete.

Note: do *not* toggle attributes in an existing element. I’ve talked with some of the WebDriver project owners, and relying on state of an attribute in an existing element can be dangerous.

Hopefully this clears things up!

[Jim]

@Chris: There’s no way other than XPath to traverse up and down the DOM. JQuery selectors won’t do it, nothing else will.

If you’re trying to navigate the DOM based on criteria in different areas of the page, then the only alternative is to do a multi-step block: find the element with the value you’re looking for, extract it to a variable in the script. Then use that value for finding things elsewhere on the page.

XPath is a tool. Used badly it’s a horrible experience. Used carefully and with consideration it’s a terrific tool that can solve some interesting issues.

Also, different folks approach problems in different ways, so my hammer may differ from yours. 🙂

[Marko]

Thanks for the presentation Jim.

I want to ask do you have experience with test automation for web pages that contain flash (flex) content?
If so, what tools do you use for this kind of tests?

thanks.

[Tom]

Richard, as a developer-turned-tester I would love to be able to get access to the code to make my own changes. I don’t think that this would be a good idea generally though, as test independence is an important tenet of testing. Either develop a relationship with development so they can make the change for you or, as in the example given, find a way around the problem.

Also, it was explained at the start of the presentation that it was aimed at a mid-level audience so, whilst you are not wrong in wanting to put more checks in, it wasn’t necessarily part of the scope.

[Jim]

@tom: I think there’s a good case to be made for testers gaining trust from developers for specific tasks around testability. Adding IDs is often a good case where devs can eventually trust me to not blow things up while adding in IDs to smooth automation.

They do NOT want me mucking around with threading or database transactions, though. 😀

[Chris]

Hi Richard, nice simple idea re. adding your own ID’s to webpages to improve testability – it’s often the simple ideas that are the best. I also like the idea of checking around me to ensure I’ve dropped in to the DOM where I think I am (look up, look down, look all around)

@Jim – I’m sure there’s lots of folks about with different shaped hammers, screwdrivers, etc

[Jim]

@tom: Thanks for the kind words. I’m glad you enjoyed the webinar!

(Make sure to go read Richard’s blog, too! http://www.thefriendlytester.co.uk/)

[Jim]

@marko: I personally haven’t worked on automating Flex/Flash. The only tool I know that supports Flash/Flex is SmartBear’s Test Complete.

I am absolutely sure there are other tools out there! Test Complete’s the only one I know of off the top of my head.

[Richard Bradshaw]

@Tom unfortunately I disagree on “test independence”. The code base doesn’t belong to the developers, it belongs to the company. The team is trying to deliver quality, so if its deemed someone has the skills to make a change let them. Testability is key with automation, coding around being unable to add a simple ID will bite you in the long term? Why waste a skilled developers time doing this, when it can be done my as the same time as my code? The checks are in place to protect against issues. Also my comments were not in relation to the talk, there were general based on this topic.

[Jim]

@richard: +1 for working to carefully define where testers can work in a codebase. It requires a LOT of discussion, collaboration, and slowly-gained trust, but it sure pays off in the long run.

It’s also absolutely NOT appropriate for everyone. Not all teams or organizations are set up skillwise to handle it. I’m telling you nothing you don’t already know. 🙂

[Richard Bradshaw]

@marko Flash is difficult. I have in the past used Sikuli to automate the “happy paths”, however the maintenance isn’t nice, and as per the news today from Toyota, sometimes automation isn’t the answer, human can be more efficient.

[Stephen Blower]

This was a useful overview, even though I haven’t yet created any automation scripts, I’ve been involved with creating automation frameworks. The main problem as I have seen it is that maintenance is rarely considered from the outset.

Key points for me:

What selectors to use and when.
I agree with Richard about not using XPath in the script, although useful for locating whilst creating scripts.
Scripts shouldn’t have hard coded values but should be using variables from the framework
Get Devs involved early and use their knowledge to make improvements.

[Jim]

@stephen Complete, vehement agreement that maintenance has to be considered from the start. My tenet with students and clients is “maintainability is even more important than accuracy.” That doesn’t mean write incorrect tests, it means focus on how the test will live over the long run.

Teams that try automation efforts without close involvement with the developers are doomed to struggle or outright fail.

[Richard Bradshaw]

@Stephen I never said don’t use XPath. It has it’s place. Those places being where Jim mentioned, finding a place to start looking from. This can also be done with CssSelectors and traversing the DOM, its the traversing of the DOM I dislike, but if it ends up being your only option, then its XPath.

To expand on the scripts, for me a script shouldn’t have a clue how something is being located, I could also without going to complex say that a script shouldn’t even know how we are interacting with them (eg WebDriver etc).

Getting devs involved is a huge win. My code has come a long way from interaction with devs. Like moving from ForEach loops to Linq. But also as mentioned, changes to the page, understanding as Jim mentioned the page / technologies work, devs know all this. Also great information to help you test.

Selectors are key, its a skill that takes practise and exposure to a lot of different applications to fully grasp

[Jim]

@Stephen: +1 for Richard’s point about locators NOT being in the scripts. This is another tremendous difference between my demo code and good production code. Page Objects are the way to go; however, there’s only so much I can cover in one hour, and I always struggle on whether or not to use POs from the start in these intro talks.

[Richard Bradshaw]

Agree with @Jim, however its something to bank in the memory.
The intro was perfectly delivered, I feel you have to start from a script. I feel a tester/dev automating needs that learning progression, to fully appreciate what a PO pattern/others can offer.
Plus its also the quickest way to get results, by this I mean get something you have creating running and checking a site. Important to feel that ASAP imo.

[Richard Bradshaw]

But again its also contextual. You have <25 scripts, other patterns could be deemed overkill. Some people would say yes, but then deciding you need more than 25 means editing all the previous checks. Perhaps, but the learning gained from starting at scripts, bring a lot of value.

[Author]

Posts