Testing Best Practices
In the PMHQ Slack community, we regularly get thought-provoking questions that we feel should be explored in-depth and documented for future reference. We’re starting a new set of Q&A posts called Highlights to dive into these kinds of questions, and enable everyone in the community to revisit the answers and contribute further.
“Should QA be part of the Dev or the Product team? At the moment I’m working for a startup that has a dev team with 8 developers and a product team of 5. Our current QA is part of the dev team.
The question is what team should have the ownership and accountability of QA, since product quality in the end comes back to product. I would love to learn how you’ve solved this in the past.”
– Kenneth Swedlund, Head of Product at Edukasyon.ph
Our community of product leaders had lots of great insights to share on this topic. Below is the summary of their QA and testing best practices.
Side note: QA stands for “quality assurance”. Some organizations may call this “QC”, or “quality control”.
Leo’s Approach
Leo Zakour, CEO at bons.me, noted that testing depends on the size of the release. That is, the level of testing required depends on what areas of the product would change, and by how much. For example, a visual change wouldn’t need to involve engineering, and a copy update could be validated by just support and customer success.
He felt that for considerably sized features that makes changes across multiple verticals, the best way to ensure quality is to involve everyone: engineering, product, design, customer success, management, and users (in that order).
To implement such a cross-functional testing process, Leo advises that each group should be informed on what should be tested, both from a holistic level and from a specific user flow. That is, the feature itself should be tested, but each tester should also spend some time within the larger product itself to look for any unexpected interactions.
From a reporting structure perspective, Leo’s perspective is that Product owns everything related to product, including engineering and QA. The goal of the Product team is not to micromanage QA, but rather to use a holistic product approach to keep all stakeholders informed and aligned.
Andrea’s Approach
Andrea Saez, Customer Experience Manager at ProdPad, agreed with Leo that everyone in the company is responsible for the quality of the product.
That being said, since Kenneth has dedicated QA resources, she recommends that the QA’s should be provided with user stories as part of testing best practices.
Juan’s Approach
Juan Mejías, Product Manager at Golden Race, shared how his company tackled the problem.
His company started as a 5-person operation and has grown to over 100 employees. Creating a separate QA department with dedicated team members was a serious improvement, as these QAs have the ability to devote time to develop test cases and create a system around testing best practices.
Additionally, Juan found that dedicated QA teams provide fresh sets of eyes. In his experience, developers can be too close to the material to properly abstract themselves and perceive potential issues.
David’s Approach
David Singer, Head of Product Management at The Nation Network, has experience managing QA on both product and dev teams.
He prefers that QA is a part of the development team, since the development team is at the forefront of the features being released. By having QA as part of the development team, he’s seen that testers have more insight into what’s being released, what’s changed, timelines, and other critical context.
Amie’s Approach
Amie Durr, Vice President of Product Management at SparkPost, noted that her team actually dropped QA as a specific individual responsibility several years ago. The engineers are responsible for their own testing, and they must pass a major set of unit tests before any code makes it to production as part of testing best practices.
Furthermore, for any product release, Amie’s team first ensures that support and customer success are trained on the new product release. As these teams run through the product, they often flag any missed issues, and also call out any place where there’s missing documentation.
Kevin’s Approach
Kevin Lee, the founder of Product HQ, noted that at one of his previous companies (with around 30 engineers, 3 designers, and 5 PMs), their engineers conducted unit testing as well. On top of that, PMs blocked off half a day to conduct their own QA through the product a few days before releasing to production.
At some point, this work became unsustainable for the Product team as the organization continued to scale. They brought on a QA manager that they trained to offload testing, starting with the most complex products first.
Tyler’s Approach
Tyler Swartz, Senior Product Manager at Reddit, commented that at both companies that he’s worked at before (small and mid-size), neither had dedicated QA. Engineering was responsible for unit testing and basic QA, whereas Product was responsible for the rest of QA.
They also held feature QA “parties” where about 10 employees would sit in a room and try to break the feature in a development environment.
You can check out the approaches of other product experts within the PMHQ community too if you want.
Analysis
Any software company lives and dies by the quality of its product offering. Therefore, testing is critical, no matter who performs the testing.
The key question that appears as teams scale is, “who should own testing?” At first, in small organizations, there may not be enough money to afford a dedicated QA resource. Therefore, while engineers should write unit test as part of good development hygiene, product managers are the ones who must own the end-to-end quality of the product at this scale.
Then, as companies scale, they are able to hire dedicated testers to focus on testing. At this point, multiple approaches can work, depending on company culture, departmental structure, and product complexity. Testers may either be part of engineering, part of product, or remain as a standalone department.
In some cases, testers are in charge of just manual testing. In other cases, the QA team is also in charge of automating testing and creating testing best practices to ensure that all stakeholders are aligned.
No matter what, product quality is critical to success. By considering the various approaches above, you’ll strengthen the quality of your product, and therefore strengthen the health of your company!
About Our Contributors
Kenneth Swedlund is Head of Product at Edukasyon.ph.
Leo Zakour is CEO and Product Manager at bons.me.
Andrea Saez is Customer Experience Manager at ProdPad.
Juan Mejías is a Product Manager at Golden Race.
David Singer is Head of Product Management at The Nation Network.
Amie Durr is Vice President of Product Management at SparkPost.
Kevin Lee is Chief of Staff of the Investment Team at Pear Ventures, and is also the founder of Product HQ.
Tyler Swartz is a Senior Product Manager at Reddit.
Have thoughts that you’d like to contribute around QA and testing best practices? Chat with other product leaders around the world in our PMHQ Community!
