How To Evaluate Translation Quality: 8 Point Checklist

Due to my experience in the translation industry, I am frequently called upon to review others’ work and to help train other translators in how to evaluate and review an initial translation.

The first thing I do when evaluating a translation is to read the source text completely, focusing on the message and the delivery style. Next, I compare the translation to the source text to verify that the same message and delivery style was carried over into the target language as closely as possible.

Once I have completed these two tasks, I then go back through the translated text word by word looking for the following:

How To Evaluate Translation Quality

1. Did the translator comprehend the source text?

• The translator’s comprehension of the source text is the foundation of a successful translation process. Without a clear understanding of the source material, it is virtually impossible to produce an accurate and meaningful translation. Comprehension involves not only grasping the literal meaning of the words but also understanding the context, nuances, cultural references, and the author’s intended message.
• The impact of a mistake in this aspect is profound. Misinterpreting the source text can lead to mistranslation, loss of important details, and even the transmission of incorrect information to the target audience. It can cause confusion, and miscommunication, and damage the credibility of the translated content.

2. Does the translation flow naturally in the target language?

• Natural fluency in the target language is vital for ensuring that the translated content is not only accurate but also easy to read and comprehend. The goal is to make the text feel as though it was originally written in the target language, seamlessly blending with the cultural and linguistic norms.
• A mistake in this area can make the translation feel awkward, stilted, or disjointed, leading to a lack of engagement from the audience. Readers may struggle to connect with the content, ultimately undermining its effectiveness.

3. Is the translation correct & localized for the intended audience?

• Localization is the process of adapting content to suit the cultural, linguistic, and social preferences of the target audience. It goes beyond mere translation to ensure that the content resonates with the audience, taking into account their unique sensibilities.
• Failure to localize properly can result in cultural insensitivity or offense. For example, humor that doesn’t translate well or references that are unfamiliar to the target audience can cause confusion or even alienation.

4. Is the text consistent? Were words and phrases consistent throughout the translation?

• Consistency is crucial to maintain the integrity of the translated content. It ensures that terminology, style, tone, and other elements remain uniform throughout the translation, creating a coherent and cohesive reading experience.
• Inconsistencies can confuse the audience and disrupt the overall message. A single translation error can introduce inconsistencies, weakening the overall impact and causing frustration or mistrust among readers.

5. Accuracy of the translated text

• Accuracy is the bedrock of any translation effort. An accurate translation is one that faithfully conveys the meaning of the source text without introducing errors, omissions, or distortions.
• The impact of inaccuracies in translation can be severe. In fields like medicine or law, errors can have life-altering consequences. Even in less critical contexts, inaccuracies can lead to misinformation, damaged reputation, or lost business opportunities.

6. Are numbers and measurements translated correctly?

• Precision in translating numbers, units, and measurements is vital, particularly in scientific, technical, or financial content. Mistakes in this area can lead to costly errors or even pose safety risks.
• For example, in engineering documents, a mistranslated measurement could result in structural failures, while in financial reports, incorrect currency conversions could lead to financial losses or legal issues.

7. Are names, trademarks, and other non-translatable words preserved from the source text?

• Preserving non-translatable terms such as proper nouns, brand names, trademarks, and specific cultural references is essential for maintaining clarity, consistency, and legal compliance.
• Failure to preserve these elements can lead to confusion, mistrust, or legal disputes. In the case of brand names or trademarks, it can also harm the reputation and identity of the products or services involved.

8. Was proper punctuation used for the target language?

• Proper punctuation is crucial for ensuring clarity, readability, and the intended meaning in the target language. Incorrect punctuation can alter the interpretation of a sentence or make it difficult to understand.
• In legal documents, for example, a misplaced comma or an incorrectly used semicolon can change the legal implications of a contract. In marketing materials, punctuation errors can affect the persuasiveness and professionalism of the message.

Three Qualities of a Good Translation

• Accuracy: A good translation must faithfully convey the meaning, nuances, and intent of the source text. It should be free from factual errors, omissions, and mistranslations, ensuring that the content is reliable and credible in the target language.
• Fluency: Fluency refers to the translation’s natural flow and readability in the target language. It should read as if it were originally crafted in that language, maintaining the reader’s engagement and comprehension without awkward phrasing or linguistic disruptions.
• Cultural Sensitivity: Cultural sensitivity involves adapting the translation to the cultural norms, references, and preferences of the target audience. This quality ensures that the content resonates with the cultural context and avoids potential pitfalls of insensitivity or offense.

Two Most Important Factors in Translation Quality

• Accuracy: Accuracy is paramount in translation. Even minor inaccuracies can lead to misunderstandings, misinformation, or legal issues. An accurate translation ensures that the message remains true to the source text’s meaning and intent, maintaining the integrity of the content.
• Cultural Appropriateness: Cultural appropriateness encompasses localization, ensuring that the translation aligns with the cultural expectations and sensitivities of the target audience. This factor is crucial for making the content relatable and avoiding cultural insensitivity or misinterpretation, which can hinder effective communication.

Translation Quality Standards

If you’re turning to a language services provider to handle translation projects, chances are, you’re not going to be all that familiar with the target language. While this can make assessing translation difficult, any reputable agency will offer quality standard certifications for peace of mind. The International Organization for Standardization (ISO) outlines useful standards that can be used to measure translation quality. 

In the translation industry, ISO 17100 is used internationally to govern the quality of translations. The standard outlines key criteria that a quality translation service should provide, including anything that directly impacts the final product. 

Translation service providers adhering to this particular standard will need to ensure that ISO 17100 criteria are applied when translating content from the source language into the target one. It also covers the review stage of a project, when a second linguist steps in to look for translation errors, compares the text to the original, and makes any necessary corrections. 

Obtaining 1SO 17100 certification 

To obtain ISO 17100 certification, a quality translation service needs to meet strict requirements. Although the certification process is simple, assessment quickly separates the wheat from the chaff. 

After an agency makes contact with the relevant ISO department, an assessment can be arranged. An auditor then steps in, checking to see how compliant a translation agency is, and identifying any changes and improvements that need to be made. New procedures are then outlined and incorporated into the existing framework of a business. ISO 17100 certification is only approved once a company meets all the necessary requirements. 

Although the process can be completed in as little as six weeks, an ISO certificate is only valid for three years. As such, companies need to regularly review processes in order to maintain ISO certification and reassure customers that they’re offering high-quality translations.

Challenges of measuring translation quality (and how to overcome them)

Measuring the quality of a translation can be difficult. If you have no knowledge of the target language, translation assessment isn’t something you’ll be able to do alone. Whether it’s corporate copy for a website or technical writing for a user manual, placing complete faith in a translation service provider isn’t always a good idea. 

Common translation errors 

While a high-quality translation shouldn’t include too many errors, they’re often unavoidable. Errors can be as simple as a single spelling mistake or a glaringly obvious grammatical error. Other times, translations may stray too far from the source material, with the final text lacking cohesion and relevance to the original piece. To overcome this, identify errors and categorize them by type and severity. You can then make amendments and additions to translation glossaries, along with new priorities for proofreaders. 

Threadbare feedback 

Gathering feedback is an important step for measuring the quality of translations. However, clients who handle this process themselves are often left with single-word answers that aren’t particularly useful. It’s even more of an issue if the people doing the reviewing aren’t translators themselves. The easiest way to overcome this challenge is to partner with an experienced translation language service provider. A reputable agency will take charge of the review stage, using industry-leading tools and processes to assess translation quality. 

Inconsistent terminology 

Inconsistent translations are another problem and particularly common with larger projects. The translation itself may not be incorrect, but a lack of consistency can prove confusing for the end user. Thankfully, translation software can quickly make inconsistencies a thing of the past. Translation memory ensures linguists are only ever using pre-approved terms, while handy prompts can help improve productivity. 

Not everyone’s on the same page 

Even an experienced translator may reach out for clarification from time to time. It’s common when dealing with content loaded with industry-specific terms or marketing material including abstract messaging. If answers aren’t shared with wider translation teams, including reviewers, conflicting options can derail the process. 

Establishing clear guidelines and outlining criteria can alleviate this. Distributing reference materials to everyone at the start of a project can also help. The simplest way to sidestep this issue is to use a translation management system, ensuring updates are communicated to everyone involved.

What is the accuracy of machine translation?

The quality estimation accuracy of machine translation has improved significantly over the years, thanks to advances in natural language processing and machine learning. However, it still varies widely depending on the language pair, context, and the quality estimation used of the specific machine translation system being used.

For widely spoken languages and straightforward content, machine translation can achieve translations of reasonably high quality and accuracy, often producing translations that are comprehensible and contextually accurate. In such cases, it can serve as a helpful tool for quickly translating large volumes of text.

Yet, for languages with complex grammar, idiomatic expressions, or specialized terminology, machine translation may struggle to maintain high accuracy and quality. Critical nuances and cultural subtleties can be lost, making human translation essential for precise translations and contextually appropriate results.

With this handy checklist and a consistent routine, I am able to effectively evaluate a translation to ensure accuracy, high quality, and professionalism. I am also able to then provide substantive feedback to the client, translation project, and the translator based on my evaluation. This not only builds value for our client and the translation company but also improves the skills of our translators, resulting in a win-win situation for everyone.

What is the importance of machine translation evaluation?

Advances in AI-powered technology spell good news for the translation industry. Machine translation is just one example, offering an automated approach to the translation process. Historically, machine translation relied on statistical models. Today, natural language processing analyzes source material before translating text into a target language. It can significantly streamline workflows, with machine translation offering multiple benefits. 

For businesses worried about the bottom line, machine translation offers a cost-effective alternative to relying on human linguists. While human translators still have a role in the overall process, machine translation makes it possible to convert large volumes of text from one language into another. While machine translation models weren’t particularly reliable, continually improving algorithms mean that machine translations can be produced with impressive levels of accuracy. 

For organizations looking to expand internationally, machine translation is an invaluable tool. It allows employers to connect with global workforces, while compelling customer-facing content can be achieved quickly and affordably. Even smaller businesses can benefit from machine translation. As it’s a scalable solution, it can handle increasing volumes of text, without any negative impact on quality. Ideal if looming deadlines and mounting workloads have become the new normal.

Two approaches for evaluating machine translation quality

When assessing machine translation quality, you can take one of two approaches. Firstly, there’s manual evaluation. A human linguist is in charge of assessment here, scouring the final translation for basic errors, fluency, and context. Generally speaking, it tends to produce more accurate results. However, it can be time-consuming and has financial implications. Furthermore, one translator may have different benchmarks when it comes to quality. 

Another option is automatic evaluation. Here, assessment is taken care of using algorithms. Translations produced by human translators, alongside automatic metrics, are used to evaluate overall quality. Automatic evaluation isn’t as accurate as a manual approach, especially when it comes to long sentences and more complex text. However, it still remains fairly reliable and is a cost-effective solution. For companies looking for a scalable evaluation method, it’s a good choice. 

Machine translation quality estimation vs. evaluation: how do they differ?

Not sure which assessment method is right for your translation project? To make an informed decision, it’s useful to look at the differences between machine translation quality evaluation and machine translation quality estimation (MTQE). While the former is a human-first approach, MTQE removes human linguists from the equation entirely. 

Machine translation quality evaluation is measured by human translators using a specific metric, such as BLEU. Established rules are used to evaluate specific segments in a sample set, resulting in an overall score that determines quality. 

With MTQE, you’re looking at a machine-only method of evaluation. Rather than use reference translations produced by human linguists, machine learning looks for correlations between the original text and translated output. Problem segments are quickly identified, making the post-editing process far more efficient. 

Looking for professional translation services? Contact us today!

Liraz Postan

Liraz is an International SEO and Content Expert with over 13 years of experience.

Need fast, high-quality translation?

Translate now