use cases

 Bullet Point Quality Control
 Bullet Point Clear Objectives
 Bullet Point Global Product Management
 Bullet Point Under Performing Product
 Bullet Point Consumer Expectations


quality control

scenario

The company we were working for supplied a software platform that was used by some of the world's top telecoms providers. The product was the company's leading software offering and it was immediately apparent that many customers were very unhappy; to the point of putting out RFIs for replacement solutions. Constant quality issues were wearing the customers down. With each new software release, only 80% of the identified bugs were being fixed and another set were being introduced. This didn't reflect the true worth of an exceptional product and very capable company.

The product was predominantly implemented by System Integrators (SIs) who we found ourselves working against as, to keep favour with their customers, they would point the finger at our software. It also turned out that some of the SIs were adding bespoke components to our software, which was not allowed under the terms of the license.

Development was done in an agile environment; however it was obvious that bugs were not being caught before release. Further investigation showed that the QA team were aware of many of the issues but were so snowed under that they gave into the pressures of the release and either ignored them or just documented them in the release notes.

tasks

We  Our first task was to qualify our concerns by:

  • Establishing a better line of communication with both the customers and the SIs
  • Doing an audit of all reported bugs (e.g. who reported them, what priority, cause of problem, what response had been given, status of fix, whether there been any customer follow up, etc.) and prioritising the findings
  • Understanding where bugs originated (development or implementation)
  • Identifying why bugs were not being picked up during development and QA cycles
  • Setting a roadmap to resolve the situation

actions

Engage with all customers

Initially it was difficult to get over the hostility shown by the customer.  They had been told on so many occasions that the problems would be fixed. We showed faith by identifying the three biggest issues and agreeing to a patch release that we would be judged on. On delivery we would adopt a similar approach for the remaining critical issues.

Improve relationship with the SIs 

The SIs had little communication with our company and had never formally requested training. They had no special relationship with us and would go through the customers channels when a bug was suspected. The SIs seemed to be unaware of the licensing issues with adding bespoke components. In addition, for most of their 'fixes' there was a better solution already in place in the product.

We immediately set-up some off site meetings with the SI and offered to deliver some on site training on solutions architecture and best practice. We asked them to identify a single point of contact so that we would become aware of issues early enough for them not to become an emergency. We also agreed to build better APIs to service custom code, this would also help identify where the problems lay.

Customer Support Training 

We set-up a programme of training and FAQs to help support operatives identify whether an issue was due to development or implementation and how to respond depending on the outcome.

Rationalise the development process 

It became apparent for a number of reasons that the development team were using the QA team to filter bugs. A lot of egos had to be gently massaged to get the developers to take more ownership of code errors by introducing more stringent unit and integration testing. The QA teams remit was refocused to make sure they test the solution as agreed in the requirements document and not purely the output from the development team.  In turn the architecture team were instructed to tick of each and every requirement in the product requirement documentation when proposing a solution.

The QA team had become a bottle neck in the delivery process a) because they were not involved in the overall quality control but in bug identification and b) because they were not planning their testing schedule early enough in the delivery process. With immediate effect better project management was introduced to plan the completion of development task so that QA could begin testing early on in the development cycle and greatly reduce the spike in the weeks leading up to delivery. A side effect of this was a much happier QA team who were constantly feeling under hammer from both the developers and the release target.

Reprioritise the next release 

The next release would then focus on quality issues. This message was sent out by the management team to customers, SIs, the Professional Services team, Customer Support and the Sales team.

result

I would be lying if I told you that the problems evaporated over night. However there was a marked improvement in the quality of our products and the relationship between the company and their customers. Both of the customers who had sent out RFIs abandoned the process and the customer satisfaction rating shot up after six months.

Going forward we agreed a regular programme of meetings with all internal and external stakeholders. In time the customers and partner SIs moved from being critics to advocates and we received several bid requests by recommendation. We also started to get much better qualified enhancement requests that were easier to specify and deliver.

Probably the biggest effect was on the morale of the development and QA teams. Instead of feeling continuously hunted they were able to measure their output more efficiently and feed back into the requirements in a more enlightened manner.

© Visjung 2009