A/B testing or split testing is a method to test a performance of different versions of your website against the baseline version. It’s most commonly used for marketing purposes to figure out how different messages lead to conversions and sales. Developers are also using it increasingly to test how new features affect the user’s behavior. It’s a great tool for data driven company to validate the direction and see whether new changes actually perform better or worse.
Using A/B testing to validate translation quality
So could you use A/B testing also to test your translations? You could use A/B method to test how your localized version performs against your baseline version e.g. English. However there’s some challenges and it’s not that simple as it sounds. In order to get valid results from A/B testing, it’s important that the results are comparable. Both control groups should be identical to each other so when you compare languages, you also accidentally compare different markets. This variable, how you perform on each market depends of your product or service. Demand, pricing, local competition and other factors like shipping costs and currency exchange rates can affect it. So how we could eliminate all these variables and test just the translation?
First, test how baseline performs versus translated version on each geographically distinct market.
This gives you valuable information whether your translated version actually performs better. If it doesn’t perform better, you can be absolutely sure that there’s something wrong with the translation. If it provides similar or only slightly better results, there’s room for improvement in the site overall localization that includes design e.g. colors, layout and translations.
When you’re sure that your localized version is actually increasing your conversions and sales, you can use the first localized version as a baseline and start rolling out changes to it. For example, you could order a totally new set of translations from different agency or translator to see how they perform. There’s no limit how much time and effort you can put to this work, but it’s an awesome way to find out what works and what doesn’t. It’s also only way to really know that your translated site is good as you most likely don’t understand all the languages.
Get Localization offers A/B testing solution for measuring translation quality on websites. Just contact us with the form below if you are interested to hear more!