Testing in the cloud versus real-life testing – two sides of the same coin

When it comes to software testing, innovation is key. It’s hard to keep up with all kinds of trends popping up like mushrooms, and to keep an overview of the unique vocabulary. It has now been nearly two months since I started working at Testbirds without prior knowledge about IT or development, so this article can also be seen as a recap of what I have learned so far.

Crowdtesting was pretty easy to understand, as I am one of the end users of apps and websites myself and already had bad encounters with faulty apps. Our project TestChameleon however was quite a challenge for me, as it is deeply technical. But worry not, this blog post will stick to the basics and make automated testing with virtual machines a little clearer. In the following post, I will give you a short introduction to test automation and the most common buzzwords such as Agile Development, DevOps or Virtual Machines.

First, a little background info: As you are reading this blog, you are probably familiar with some parts of software testing, especially crowd testing. It’s pretty obvious that software needs to be tested before it’s released to the open market to avoid bad user experiences and functional problems. Perhaps you already tested a website for bugs and usability issues and know the importance of the feedback that comes from the user side.

However, when it comes to bug and compatibility testing, there is another approach that is getting more and more popular: Test Automation. Testbirds offers a service called TestChameleon (TeCh), which we have introduced to you in a post earlier this year. To keep it short: It’s a web service that creates every possible test infrastructure you could wish for in a virtual machine, similar to a chameleon that adapts its’ color to its’ current environment. Software testing in TeCh can be done manually or automatically, depending on what you need – but as the variety of operating systems, browser types, versions and add-ons in the market is changing almost daily, an automated test is the logical answer for getting feedback on the functionality and compatibility of the tested software.

Before getting into the benefits of test automation, we should probably talk about some development basics before. A rather new trend in development departments is called Agile or Lean Development. The base of Agile is a catalogue of methods and rules that ensure a more flexible development process as an alternative (and in many cases improvement) to traditional sequential or waterfall development. In Agile, the development activities are split up into so called sprints. For example, the waterfall method does all development steps consecutively, such as design, code or testing. Agile methods on the other hand run sprints with all those activities on all stages of the development process to find bugs and adapt to changes more quickly. It is a continuous improvement of all components of the product that is being developed to assure an early delivery.

Agile Development

The summary of agile methods can be found in the Agile Manifesto, which can be seen as a catalogue of essential rules for everyone who wants to adapt to that style of development.

Well, as one of the basic rules of agile development is the close collaboration of business people and developers, we come to the next term that was completely unfamiliar to me: DevOps. This is a combination of the words “development” and “operations”. DevOps is, like agile, more of a movement than a fixed term. The aim is to establish a flexible and productive working environment concerning the activities of building, testing and releasing software. So basically the barriers between operational organizers and the development department shrink to a minimum. Therefore, TestChameleon can be a part of the DevOps approach, as it enables continuous quality testing and is a helping tool in combining those two areas of work.

Next stop on the long way to intermediate TeCh knowledge would be: VMs – Virtual Machines. If you need to see how your website looks, for example, on one of those laptops with a fruit on its back, on the latest Firefox browser and using a strong add-blocker, you can create a virtual machine using TestChameleon. That means you can virtualize every device you would like to test your website on and control it from your own computer. As said before, the possibilities of different combinations of operating systems, browsers and add-ons are overwhelming. Test automation deals with those issues, as you can use programs like Selenium to create a script that runs automated tests. This method is so fast and flexible that you could test a website overnight on all common devices and by the next morning, you could start working on the bugs TeCh found.

At that point you might ask yourself: When do I use which testing method? What are the key benefits of each method?

First of all and most obvious – even though our cloud based TeCh can do quick and easy functional testing, there is clearly no way for automated testing to provide information about usability. This is and will be a task that the wisdom (and the humaneness) of the crowd is essential for. Exploratory testing is rather difficult with TeCh, as the tests run according to one script, which exactly specifies how and what will be tested. Also the crowd can provide us with a “dirty” testing environment. This means that several apps and components, which the person who wrote the testing script didn’t think of, will run in parallel.

Cloud and Crowd

Crowdtesting and automated testing should rather be seen as friends that are complementing each other. Automated Testing is mostly used for regression testing. That means that for example, if the development team implements a new feature to a website, regression testing can easily see if the feature works and if the rest of the website still runs flawlessly and remains unaffected by the change – on all kinds of devices. Crowdtesting is rather a real life test, often used shortly before release. Those two methods are no competitors, and using both testing techniques would let the client benefit from test that covers a multitude of quality issues. The basic principle: Cloud and Crowd work hand in hand.

I tried my best to explain a few of those words that are following me home for two months now after leaving the office. Sitting across our development team makes me realize that there is still a lot of stuff I don’t understand, but this also leaves us space for follow-up blog posts I guess. If you like, keep these facts in mind and they will probably lead you to a very agile conversation at the next IT Party. (Agile, ha-ha. Get it?)

3 Comments Testing in the cloud versus real-life testing – two sides of the same coin

    1. saisha

      Very useful information regarding Testing in the cloud versus real-life testing – two sides of the same coin. helped me to understand the concept of Agile development, DevOps. Thank you.

      Reply
  1. Naga987

    Hi

    Your given details information about cloud testing. It has more helpful. Give me some more information with real time examples.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>