Reduced Test Execution Time
Increased Test Coverage
Innovative Interfaces, a Proquest company and a subsidiary of EBSCO Information Services, is a library management software company whose mission is to deliver quality products that exceed user expectations. In a world where libraries are competing for people’s time and attention, Innovative Interfaces helps libraries thrive in more than 65 countries.
Through a new ultramodern platform Vega, Innovative Interfaces boosts engagement with public library collections, thus making the process of searching and browsing online catalogs easier and enjoyable for library patrons. In an effort to meet all of its client’s testing objectives, Innovative Interfaces worked with LambdaTest to deploy a scalable testing infrastructure and was able to achieve a 70% increase in test coverage and reduced test execution time from 1 minute to 20 seconds.
When Matt joined as a Principal Automation Engineer, his primary responsibility was to establish best automation practices and identify a mechanism that is instrumental in reducing the test execution time by a huge margin. However, the organization used to test scripts through a Selenium Grid hosted on AWS, which required continuous maintenance and frequent upgrades to keep up pace with the rapidly evolving browser and OS landscape. His first major challenge was managing the grid and keeping browsers up to date, which wasn’t easy. The need for a scalable solution came about when the organization realized that it could no longer test with its present infrastructure as keeping it updated was proving to be impossible.
Therefore, they needed a grid that could identify and support the specific need of each test and require minimal changes in the implementation.
The problem they had with their existing grid was two folds:
They quickly realized that their team was spread very thin, managing the browser releases, building new features, and taking care of stability issues with bugs reported.
“While looking for a solution, I had three choices: Firstly, I could throw more hardware at it, but our hosting costs were already high, and throwing more hardware at it would just raise our hosting costs higher. Secondly, I could cut some corners on browser testing, but that would have resulted in more bugs being shipped to customers. Finally, we had the choice of going for a cloud-based cross browser testing solution like LambdaTest.” - Matt Rench, Principal Automation Engineer
Switching to LambdaTest drastically reduced the overhead of maintaining the in-house Selenium Grid whilst reducing the effort involved in managing and maintaining cross browser testing infrastructure.
LambdaTest lets us cut costs and increase speed by allowing us to run Selenium Grid nodes in the cloud. As an added bonus, LambdaTest also makes it easy to scale up during peak usage and then scale down when we’re not using Selenium.”
- Matt Rench, Principal Automation Engineer
Mitigating regression bugs was another challenge that Matt and his team faced when they were rolling out new builds in every two-week sprint. Again, the team needed tighter control of the product and a systematic solution to prevent regression bugs due to the lack of an end-to-end testing mechanism for every new build.
While performing regression testing, the testers encountered bugs that were not considered when the test cases were designed. Furthermore, since the test cases were narrowly tailored for testing a few specific functionalities, it was difficult to foresee all the possible outcomes in the system under test.
After comparing the bugs that were found during the manual testing and regression testing, it was concluded that there were more bugs in regression testing, as it considered all possible scenarios. These could have been easily caught if more test cases had been written for functionalities that weren’t usually tested.
However, once they started to scale and release more frequently, it was important to have a comprehensive end-to-end testing mechanism. Earlier, due to lack of automation and a comprehensive end-to-end testing approach, critical bugs were rarely caught in the QA process. Therefore, their goal was to create a comprehensive end-to-end testing mechanism that would allow them to test every code change.
While looking to scale their testing operations, one of their top priorities was to migrate to cloud-based infrastructure for cross-browser compatibility testing and automation to scale their test coverage.
Matt, in his previous firm, had used other cross browser testing tools to test websites and web apps on multiple browsers and operating systems. However, when their users started reporting bugs, they couldn’t reproduce them. Matt also spent a lot of time going through the documentation and translating it into a project-specific solution. But, nothing worked in their favor. While looking for an alternative, Matt came across LambdaTest.
So, from Matt’s past experience, he knew about LambdaTest. They decided to go ahead with LambdaTest as it fit their needs perfectly because of its speed, ease of use, and support for mobile apps. As a company that is focused on creating business apps, they had to comply with Apple's App Store Guidelines. They decided to do this by adding the ability for the app to function on mobile devices. Since they started testing on emulators and simulators, they've been able to find bugs in their app before releasing new builds to the App Store.
LambdaTest was a great decision for Innovative Interfaces because it helped them successfully build a culture of shift-left testing in their organization. It also ensured they could run more targeted tests throughout their product development process while still testing the overall product every day.
“There are other tools in the ecosystem that provide similar benefits, but ironically couldn't get it to work well enough because of how difficult it was to set up compared to setting up a local Selenium Grid.” - Matt Rench, Principal Automation Engineer
While they were initially able to handle a high volume of browser requests, as time went on, they experienced more traffic, which meant that all of their individual browsers were being used simultaneously. This made it harder for them to manage their testing process because everyone wanted to use the browser simultaneously, which caused an overload on the system.
With LambdaTest, they can add more machines to their account to keep up with all the requests without any issues. In addition, the team performs cross-browser testing across multiple browsers, executing thousands of test cases in a few minutes, thereby saving execution time and cost.
“We run tests on many different browsers, but our scheduled builds run on Chrome, Edge, and Firefox. We test our builds on those browsers to ensure they work for the customers. We don't test as much as we could, but the fact that we use Chrome, Edge, and Firefox for development means that we ship features faster than we would if we only used some other browser for development.” - Matt Rench, Principal Automation Engineer
The fact that LambdaTest provides integrations with top CI/CD tools, including Jenkins, helped them pace up automated cross browser testing by leveraging the LambdaTest Jenkins plugin. The LambdaTest Jenkins plugin allowed them to easily automate their Selenium test scripts by connecting their Jenkins CI instance to the LambdaTest Selenium Grid. With LambdaTest, they can run the pipeline as often as every 5 minutes during peak hours and every hour during the rest of the day. This has helped improve the quality of their codebase and reduced production defects significantly.
“LambdaTest runs all of our automated tests once a day in a CI pipeline, and this is the only way we run the tests. They provide fast feedback for us on what specific part of the code doesn't work with each new change, so we can fix it quickly with 24/7 support availability." - Matt Rench, Principal Automation Engineer
Furthermore, they do a lot of testing on their repositories and pull requests. They've found that doing this helps them catch bugs faster than any other approach. Every time someone merges into master, the regression suite runs automatically. When the regression suite passes, they know that everything that was broken in the master before has been fixed in the master now.
“Our automation stack uses Java, Selenium, TestNG, Gradle, RESTAssured, Jenkins, and Git. With LambdaTest automation, we run our critical automated test suite on every pull request and main branch merge. We run our regression suite on a nightly schedule. We're also doing unit tests, component tests, contract tests, performance tests, and more outside of LambdaTest." - Matt Rench, Principal Automation Engineer
According to Matt, testing session videos have been a game-changer. This is because it gives them a chance to look at the application differently. For instance, if someone has been reviewing the application every day and has been working with it all this while, he/she may have forgotten what needs to be changed or improved. In addition, video recordings of testing sessions also help them see that through their customer's eyes.
The video playback on the LambdaTest cloud has enabled them to streamline their work and focus on the important things. This has helped them reduce manual QA testing effort, which is both slower and less efficient than automated testing. Instead, they now spend more time working with the device data and building a richer feedback loop that has helped them improve the product faster.
For early-stage startups and enterprises alike, cloud-based cross browser automation is the agile, efficient solution that is poised to revolutionize the quality assurance process. With LambdaTest, they can easily test their website or app across multiple browsers and devices in parallel and get automated test reports to improve the quality of their products and features. Now, Innovative Interfaces ensures continuous and consistent deployment, eliminates costly bugs in each new release, and much more: