Product

Software testing evolution and challenges

Gustavo Almeida
Gustavo Almeida September 11, 2020
Software testing evolution and challenges

A few years ago, the role of testers or Test Analysts was limited to documentation, reports, and manual software testing. They worked closely with the Requirements Analyst or Project Manager and, from the system requirements, began to transform the system’s use cases into test cases. These, in turn, were compiled into a document that would serve as a basis to monitor them, as a checklist that should be validated for each version of the system.

This compilation is known as Test Plan. It contained the step-by-step and other important information for conducting the tests, such as scope, environment settings, test data, roles and responsibilities of those involved; dates for test execution, maintenance windows, data dictionary, metrics and a multitude of information that supports this very fundamental stage of the software life cycle.

Example of a Test Case:

How to do software test cases
Source

The number of test cases could easily reach hundreds, depending on the complexity of the system. This documentation was often kept in Word (.doc) or Excel (.xls) files, which made maintenance difficult.

A simple change in the name of the button implied changes in dozens of pages of that document. So you can imagine how complicated it was when changes were made to complex rules or prerequisites.

These documents also worked as deliverables defined in the minutes of meetings and internal audits, and this approach of a more rigid document was possible thanks to the Waterfall development model, which was widely used in companies. This model followed a line of sequential phases, like a cascade, as suggested by its name.

Waterfall model representation:

Software testing waterfall development model

At some point, the first exclusive tools for test automation and others that allowed more flexibility in the registration and maintenance of test plans and bug records began to appear. Some of them were created by companies such as Microsoft and IBM – the IBM Rational, for example.

In the meantime, software development methodologies evolved in a more agile and less rigid way, with many iterations and deliveries, and subject to constant requirement changes. In a quick Internet search, you can find thousands of blogs, articles, and books about Agile, Lean, SCRUM, Kanban, etc. Other methodologies that introduce testing at a stage prior to the code, which focus on behavior and/or software mastery, also began to emerge: DDD, TDD, BDD.

Programmers created their tests, teams took their first steps in agile methodologies, and testing professionals began to develop software, looking for ways to automate the repetitive tests that occurred with each iteration. There was a convergence of efforts to deliver the highest quality embedded in the products, which is still applied in companies today, including at VTEX.

Complexity and exponential increase of systems

There was a boom in web development, scalable SaaS systems, and we entered the mobile-first era, so the scope of testing kept increasing. We have reached a time when we have different web browsers, each with its own particularities, and two main operating systems for mobile devices. There are a multitude of these devices with different screen sizes and resolutions.

In order to keep up with these changes, cross-browser and cross-device testing services have emerged to help leverage automated testing, providing scripts with more speed and flexibility.

These services offer cloud infrastructure and easy parallelization of test runs.

software testing services for cloud infrastructure

In addition to different browsers and screen sizes, validation between different versions of the same browser and/or operating system is also a constant concern. For example, a web page could work perfectly in the version X of the browser, but register failures in version Y. In general, version Y should be supported by the system – due to contractual, customer relationship, or other reasons – even if the manufacturer no longer provided support for that version anymore.

On top of this, tests aimed at non-functional requirements have also increased. As an example, we have video streaming applications that should continue to work even with a low signal from the telephone antenna, systems that use internal device features such as accelerometers, and a multitude of small details that can further increase the scope of testing.

Summarizing, it is noted that the complexity, dimensions, and challenges of software testing have increased and continue to increase at an exorbitant speed.

What about the software testing professional?


This transformation also impacted the profile of the software testing professional, who is now better able to write code, use DevOps tools and continuous integration systems, for example.

As a result of this transformation, this position received different names over the years. Today, there are countless terms to identify these professionals that, despite having different roles, always carry the feeling of being the owner of the product and focus on quality. We can list several of those job titles: Software Tester, Test Analyst, Quality Analyst, Quality Assurance Analyst, Test Manager, Agile Tester, Test Automation Analyst, Test Engineer, etc.

At VTEX, the profile that best fits our team is the Software Engineer in Test (SET). This professional, who gained notoriety through large technology companies in Silicon Valley, is best described as a software engineer with a background in testing and an emphasis on the development of automation systems, frameworks, and systems aimed at software testing in general.

We have an open position for SET. Check it out here.

The future is already here

We are surrounded by data, and more data-driven applications are increasingly emerging. This type of application makes it difficult to use traditional testing techniques, such as Equivalence Partitioning or Boundary Value Analysis, as the answers are not necessarily binary.

For example, Artificial Intelligence applications, Machine Learning, speech recognition, facial and image recognition in general; how can we validate that a system has correctly recognized a cat in an image? We know that there are false positives and a lot of background in statistics in these systems.

Will we have increasingly qualified professionals for embedded technology and hardware? Many virtual assistants are to come and, with them, many testing challenges.

Training in statistics and data science is a path these professionals could follow. Considering the pace at which software testing has evolved, it is expected that software testing professionals continue to pursue further specialization. After all, their challenges and responsibilities are increasing.

Do you want to build a future-proof career? Check out our vacancies.

Keep reading: Related stories
Product

Grow your business with VTEX’s Sales Performance Dashboard

Data insights are more important than ever and at VTEX, we’ve been constantly working on ways to provide…

Andreea Pop
Andreea Pop
Product

DTC Launchpad: An easy way for CPG companies to launch a direct-to-consumer business

The backbone of the consumer packaged goods (CPG) business model was stable for decades — shoppers consumed brand…

Robert Poratti
Robert Poratti
Product

VTEX Marketplace Network: Connecting sellers and marketplaces around the world under one ecosystem

The Marketplace Network is a VTEX platform functionality that helps connect marketplaces and sellers operating in the VTEX…

Sorana Gheorghiade
Sorana Gheorghiade
Product

The DOs and DON’Ts when applying for a Product Design job

After ten years of experience in this field and over the course of five years as a Product…

Bianca Galvão
Bianca Galvão
Institutional

Commerce at MACH Speed – Why VTEX Joined the MACH Alliance

The VTEX Commerce Platform is now MACH Certified and a member of the MACH Alliance. The acronym MACH…

Robert Poratti
Robert Poratti
Product

Startup vs. Unicorn: what it’s like working with a product in two different realities

Steer a small startup or lead a team in a big unicorn? Gustavo Vedotti, Product Manager at VTEX,…

Gustavo Vedotti
Gustavo Vedotti
Product

VTEX Community: Powering collaboration in the digital commerce ecosystem

Inspired by our charter, where we committed to sharing valuable knowledge that shapes the industry, advancing solutions that…

Cassiane Vilvert
Cassiane Vilvert
Product

Hoi, salut, 안녕하세요! VTEX platform is now available in Dutch, French, and Korean

Taking another fundamental step towards our goal in writing the future of commerce, we are pleased to announce…

Larissa Dias
Larissa Dias
Product

Eight practical actions for migrating to the product area

Are you thinking about making a transition in your career and working with product? Learn more about a…

Alice Sabino
Alice Sabino
See More