Top Gift For Testers, 70% Problem, Test Coverage and More

Do you know the perfect Holiday gift to give that special tester in your life?

Have your heard about 70% problem: when it comes to AI-assisted coding

What Does Test Coverage Mean?

Find out in this edition of the Test Guild New Shows Recap Newsletter for the week of of Dec 15. So, grab your favorite cup of coffee or tea, and let's do this.

The Perfect Gift for Testers

Get the lowest prices of the season for Automation Guild 25 online tickets.

It’s that perfect gift for that special automation testing in your life.

We also offer bulk discount if you have an education budget that is use it or lose it.

This is a great time to 👉👉👉👉👉 grab your ticket now and accelerate your 2025 automation efforts in the new year

The 70% problem: Hard truths about AI-assisted coding

A new analysis by Addy Osmani a Engineering leader at Google reveals a significant disconnect between AI-powered software development speed and actual product quality.

Despite developers reporting increased productivity with AI coding tools like GitHub Copilot, Cursor, and Cline, 👉 the investigation finds that software quality isn't showing corresponding improvements.

The report identifies what researchers call "the 70% problem" - AI tools can rapidly generate initial code but struggle with the crucial final 30% needed for production-ready software. This includes handling edge cases, error management, and maintaining code stability.

The study highlights a particular concern for testing teams: AI-generated code often lacks robust error handling and comprehensive test coverage. Senior developers were observed spending significant time refactoring AI suggestions and strengthening type definitions - steps that junior developers frequently missed.

The analysis also reveals that AI tools are creating a new category of testing challenges. Teams using AI for rapid prototyping often produce what's termed "house of cards code" - applications that work during demos but fail under real-world conditions. Common issues include confusing error messages, overlooked edge cases, and accessibility problems.

The investigation found that successful teams are implementing a "trust but verify" approach, combining AI-generated code with rigorous manual review processes and automated testing of edge cases.

AI-assisted development is introducing new types of software vulnerabilities that require enhanced testing strategies, particularly around error handling, edge cases, and real-world usage scenarios.

Cypress-AJV Schema Validator Plugin v1.4.0 Released

Sebastian Clavijo, a Staff Quality Assurance Engineer at KUBRA and an active contributor to the Cypress ecosystem, announced the release of version 1.4.0 of the 👉 Cypress-AJV Schema Validator plugin. This update includes:

• Added types for the validateSchema() command, contributed by Murat Ozcan.
• Resolved an issue with package-lock.json.
• Introduced an .npmrc file for enhanced package management.

The plugin, which is open-source, simplifies API testing by validating API responses against JSON schemas, Swagger documents, and OpenAPI specifications. It leverages the Ajv JSON Schema validator, known for its speed and reliability. The tool integrates seamlessly with popular Cypress plugins, such as Gleb Bahmutov's @bahmutov/cy-api and Filip Hric's cypress-plugin-api, enabling direct schema validation following the cy.api() command.

This update enhances type safety and stability, making it a powerful addition to the toolkit of software testers focused on API testing and validation.

What Does Test Coverage Mean?

In a recent blog post, 👉 What Does Test Coverage Mean?, the complexities behind test coverage and its implications for software quality assurance are thoroughly examined. The post delineates test coverage as the measure of how much of a system's functionality is exercised by the tests in place. It emphasizes that while high test coverage can indicate thorough testing, it does not necessarily ensure product quality or defect-free software.

The article outlines various methods for measuring test coverage, such as code coverage, requirements coverage, and risk coverage, each providing different insights into the software's testing status. It highlights the common misconception that 100% test coverage guarantees no bugs, clarifying that it instead reflects the thoroughness of the tests executed against documented requirements or code paths.

Software testers must understand that test coverage is a tool to assess the breadth—not the depth—of testing, and should be used alongside other quality metrics to ensure comprehensive software assessment.

temp-disposable-email

A new 👉 NPM package, temp-disposable-email, has been released by QA Manager and blogger Pirasanth Jesugeevegan. The package offers an efficient solution for managing temporary email accounts, specifically designed to streamline email-based workflows such as account sign-ups, password resets, and email verifications.

temp-disposable-email enables testers to create unique, disposable email addresses with ease, retrieve and verify email content, and automatically clean up accounts post-testing. Notably, the package integrates seamlessly with popular test automation tools like Playwright, Cypress, and Node.js, making it a versatile addition to test suites. It supports both ES Modules and CommonJS, ensuring compatibility across various project setups.

Key features include configurable polling for retrieving emails, the ability to extract email content for validation, and automated clean-up processes to maintain a tidy testing environment.

QMetry is now part of the SmartBear

SmartBear, a prominent provider of software development and visibility tools, has announced the 👉 acquisition of QMetry, a firm known for its test management solutions. This acquisition is poised to expand SmartBear’s portfolio, integrating QMetry's offerings to provide comprehensive test management capabilities to users.

SmartBear aims to streamline end-to-end testing processes by incorporating QMetry’s expertise in test management, which is valued for facilitating collaborative workflows and enhancing software quality.

LambdaTest Partners with Autify to Enhance No-Code Mobile App Testing with Real Device Support

LambdaTest has announced a 👉 strategic partnership with Autify to augment its no-code mobile app testing platform by introducing real device testing capabilities. This collaboration aims to enhance the testing process by allowing users to perform more accurate and reliable tests on real devices, providing an essential tool for testers to emulate genuine user experiences more effectively.

LambdaTest, known for its cloud-based cross-browser testing platform, now integrates Autify's no-code testing functionalities. This integration empowers testers to create and execute automated tests on actual devices without writing code, broadening the accessibility of sophisticated testing methods to a wider range of users. This development responds to the growing demand for more accurate testing environments that reflect real-world conditions, minimizing the gap between testing scenarios and live user experiences.

Playwright Test Runner reports are now supported in Artillery Cloud

Artillery Cloud, a platform known for its load testing capabilities, has announced 👉 full support for Playwright Test Runner reports. This new feature allows users to monitor Playwright tests in real time, view detailed HTML reports complete with traces and screenshots, and automate pull request (PR) comments via GitHub Actions integration.

The integration is simple to implement, requiring just two lines of configuration in the playwright.config.ts file. It is available to all Artillery Cloud users, including those on the free Developer plan, with no additional cost. This enhancement bridges functional and load testing workflows, enabling testers to track and visualize Playwright test results within the Artillery ecosystem.

OpenTelemetry for Generative AI

OpenTelemetry has announced its 👉 integration of generative AI technology into its observability framework.

This integration aims to enhance the capabilities of software testers by automating data analysis processes. OpenTelemetry, widely adopted for its open-source observability solutions, will use generative AI to provide deeper insights and streamline error detection, leading to faster issue resolution. The incorporation of AI is expected to reduce the manual effort required for data interpretation, thereby increasing efficiency and effectiveness in testing processes.

Software testers should prepare to leverage generative AI tools within OpenTelemetry to optimize their data analysis efforts and improve the speed and accuracy of identifying issues during testing.

That's a Wrap

Just a heads up this will be the last news show for 2024 – I’ll be away celebrating the holidays and taking a long cruise celebrating my 25 wedding anniversary so I’ll be back mid-January.

Make sure to 👉 subscribe to never miss another episode.

I'm Joe Colantonio, and my mission is to help you succeed in creating end-to-end full-stack DevSecOps automation awesomeness.

As always, test everything and keep the good. Cheers!

Next Steps

Join our private Automation Testing Community when you 👉👉👉 register for Automation Guild 2025 and get access to other like minded experts 24x7