28 Sep 2015

User Tests: Improving Fairlingo Through Validated Learning

Categories: User-testing, Validated-learning, Quality-assurance

Hey everyone, today I’m back to share a little about the importance of user testing, and how we’re using it to improve Fairlingo! Additionally, a little bit about validated learning, and its place in the development of a successful product. Let’s dig in!

User Testing: How important is it, really?

It is hard to overstate the importance of testing any product before releasing it. At Label305, we write automated tests, ensuring the quality of our code. Before any change to the code is released to production, tests are automatically run so we know everything still works as it should. If not, we’ll know it and fix it. If the tests pass, it can be released to production.

But there is one thing such tests cannot check: Whether someone actually understands the user interface, and enjoys using it. Computer says yes, user says no. Even a very pleasant-looking interface could turn out to be frustrating to use, deliver a big hit to the adoption of the product. Not something you want to happen to your expertly developed new product, as it enters the marketplace for the first time. So, to insure the quality of the interface, indeed the product, you need to test it with real people. There is really no substitute for getting real people in a room, having them use the product, and see what happens.

This is exactly what is currently happening at Fairlingo; an innovative translation online platform under development here at Label305. At the time of this writing, Fairlingo has been undergoing tests by real translators for a few weeks. So far this has resulted in issues being found that otherwise would’ve gone unnoticed until its public release. Most of these were small and seemed obvious in hindsight. But lots of small things can greatly detract from the overall experience.

What this has to do with validated learning

Validated learning is a term used in the lean start-up scene, and originated in the book with the same name “The Lean Startup”. But what it means applies more universally and can be summarized in the following steps:

  1. Make something with specific goals in mind, goals that can be tested by valid metrics (set goals, choose metrics, and create)
  2. Get your product into the hands of some of your intended audience and let them use your product, and measure whether your product reaches the goals you have set for it (test, measure, and learn)
  3. If not, use your metrics as input for improvement for step 1 (evaluate and validate)
  4. If so, your product is viable for release!

So, in an even smaller nutshell, you pre-test your product before releasing it. This in such a way that you can reliably assess whether a product needs more work before release, or whether it’s even worth of extended development. For software products, user tests are an important part of this process. After all, if your product passes mustard with a sample of your intended audience, you can rightfully be more confident the product can be released to the public.

Fewer relevant tests == Greater real risks

Not testing a product before release is a very risky move, and sadly sometimes even fatal. After all, without user testing any existing issues will be found AFTER release, and may thus frustrate a far larger group of people. And these people are unmotivated to be patient with your product unlike a test-group would have been. Frustrating a large group of users early on can mean the death of your product. You don’t get a second chance to make a first impression.

Testing on the cheap(er): Put your product in beta

At Label305 we meet a lot of startups with great ideas, but none have money to burn. To save money in the short run, they often cut user testing out of the budget. However, user testing does not need to be expensive. It can even be free! An easy way to limit the potential problems of an untested product, is to go for a release with restricted access. Simply put, a beta stage where really interested users get early access and try it out before the public release. Depending on the product it can be very easy to find some early adopters, eager to try your product. If not, all it requires is some more elbowgrease to drum up a small group of people willing to test, and off you go. Even if it’s just one person from the outside having a go with the product, any issues he/she finds can be fixed for a larger release. And that will save you money and many headaches in the long run.

Written by: Alexander van Brakel