Product

Software testing evolution and challenges

Gustavo Almeida
Gustavo Almeida September 11, 2020
Software testing evolution and challenges

A few years ago, the role of testers or Test Analysts was limited to documentation, reports, and manual software testing. They worked closely with the Requirements Analyst or Project Manager and, from the system requirements, began to transform the system’s use cases into test cases. These, in turn, were compiled into a document that would serve as a basis to monitor them, as a checklist that should be validated for each version of the system.

This compilation is known as Test Plan. It contained the step-by-step and other important information for conducting the tests, such as scope, environment settings, test data, roles and responsibilities of those involved; dates for test execution, maintenance windows, data dictionary, metrics and a multitude of information that supports this very fundamental stage of the software life cycle.

Example of a Test Case:

How to do software test cases
Source

The number of test cases could easily reach hundreds, depending on the complexity of the system. This documentation was often kept in Word (.doc) or Excel (.xls) files, which made maintenance difficult.

A simple change in the name of the button implied changes in dozens of pages of that document. So you can imagine how complicated it was when changes were made to complex rules or prerequisites.

These documents also worked as deliverables defined in the minutes of meetings and internal audits, and this approach of a more rigid document was possible thanks to the Waterfall development model, which was widely used in companies. This model followed a line of sequential phases, like a cascade, as suggested by its name.

Waterfall model representation:

Software testing waterfall development model

At some point, the first exclusive tools for test automation and others that allowed more flexibility in the registration and maintenance of test plans and bug records began to appear. Some of them were created by companies such as Microsoft and IBM – the IBM Rational, for example.

In the meantime, software development methodologies evolved in a more agile and less rigid way, with many iterations and deliveries, and subject to constant requirement changes. In a quick Internet search, you can find thousands of blogs, articles, and books about Agile, Lean, SCRUM, Kanban, etc. Other methodologies that introduce testing at a stage prior to the code, which focus on behavior and/or software mastery, also began to emerge: DDD, TDD, BDD.

Programmers created their tests, teams took their first steps in agile methodologies, and testing professionals began to develop software, looking for ways to automate the repetitive tests that occurred with each iteration. There was a convergence of efforts to deliver the highest quality embedded in the products, which is still applied in companies today, including at VTEX.

Complexity and exponential increase of systems

There was a boom in web development, scalable SaaS systems, and we entered the mobile-first era, so the scope of testing kept increasing. We have reached a time when we have different web browsers, each with its own particularities, and two main operating systems for mobile devices. There are a multitude of these devices with different screen sizes and resolutions.

In order to keep up with these changes, cross-browser and cross-device testing services have emerged to help leverage automated testing, providing scripts with more speed and flexibility.

These services offer cloud infrastructure and easy parallelization of test runs.

software testing services for cloud infrastructure

In addition to different browsers and screen sizes, validation between different versions of the same browser and/or operating system is also a constant concern. For example, a web page could work perfectly in the version X of the browser, but register failures in version Y. In general, version Y should be supported by the system – due to contractual, customer relationship, or other reasons – even if the manufacturer no longer provided support for that version anymore.

On top of this, tests aimed at non-functional requirements have also increased. As an example, we have video streaming applications that should continue to work even with a low signal from the telephone antenna, systems that use internal device features such as accelerometers, and a multitude of small details that can further increase the scope of testing.

Summarizing, it is noted that the complexity, dimensions, and challenges of software testing have increased and continue to increase at an exorbitant speed.

What about the software testing professional?


This transformation also impacted the profile of the software testing professional, who is now better able to write code, use DevOps tools and continuous integration systems, for example.

As a result of this transformation, this position received different names over the years. Today, there are countless terms to identify these professionals that, despite having different roles, always carry the feeling of being the owner of the product and focus on quality. We can list several of those job titles: Software Tester, Test Analyst, Quality Analyst, Quality Assurance Analyst, Test Manager, Agile Tester, Test Automation Analyst, Test Engineer, etc.

At VTEX, the profile that best fits our team is the Software Engineer in Test (SET). This professional, who gained notoriety through large technology companies in Silicon Valley, is best described as a software engineer with a background in testing and an emphasis on the development of automation systems, frameworks, and systems aimed at software testing in general.

We have an open position for SET. Check it out here.

The future is already here

We are surrounded by data, and more data-driven applications are increasingly emerging. This type of application makes it difficult to use traditional testing techniques, such as Equivalence Partitioning or Boundary Value Analysis, as the answers are not necessarily binary.

For example, Artificial Intelligence applications, Machine Learning, speech recognition, facial and image recognition in general; how can we validate that a system has correctly recognized a cat in an image? We know that there are false positives and a lot of background in statistics in these systems.

Will we have increasingly qualified professionals for embedded technology and hardware? Many virtual assistants are to come and, with them, many testing challenges.

Training in statistics and data science is a path these professionals could follow. Considering the pace at which software testing has evolved, it is expected that software testing professionals continue to pursue further specialization. After all, their challenges and responsibilities are increasing.

Do you want to build a future-proof career? Check out our vacancies.

Keep reading: Related stories
Product

Eight practical actions for migrating to the product area

Are you thinking about making a transition in your career and working with product? Learn more about a…

Alice Sabino
Alice Sabino
Product

“Kon’nichiwa Japan!” VTEX platform is now available in Japanese  

After opening an office in Singapore to best serve global customers in the APAC Region, VTEX is now…

Cassiane Vilvert
Cassiane Vilvert
Product

Beyond Predictable: Powerful Product Teams at VTEX

How committed people, product mindset in our teams and future-based language unlock results beyond what’s predictable for our…

Guilherme Rodrigues
Guilherme Rodrigues
Product

Free your business from the monolithic commerce platform – it may not be as hard as you think

It’s a fact that in software, and particularly in ecommerce software, if a provider fails to invest in…

Robert Werkema
Robert Werkema
Product

Inside a Visionary’s roadmap: What you can still expect from VTEX in 2020

Recently, VTEX was named a Visionary in the 2020 Gartner Magic Quadrant for Digital Commerce, and according to…

Cassiane Vilvert
Cassiane Vilvert
Product

Grupo Exito’s AI-powered intelligent search experience

Textiles, perishables, electronics, or furniture. If you can name it, Grupo Exito most probably has it. The ongoing…

Sorana Gheorghiade
Sorana Gheorghiade
Product

Tap into new markets with the new VTEX and Lengow App

Supporting merchants across the globe, expanding their reach and impact — this is the vision shared between VTEX…

Lucas Bacic
Lucas Bacic
Product

Store Framework: How VTEX optimized store performance with React

The challenge of improving ecommerce performance is not simple. Losing sales due to a slow website is one…

Rodrigo Affonso
Rodrigo Affonso
Product

What we learned from Data Indexing System here at VTEX

The science and technology of working with data are now one of the main drivers of success in…

Rodrigo Abinader
Rodrigo Abinader
See More