Machine Translation (MT) has been around for decades. In recent years, it has evolved at an exponential rate. As companies produce ever-growing amounts of content in different languages, they have come to see MT as an opportunity to extend their reach in an increasingly globalized world.

Companies seeking to deploy MT can explore the following four basic strategies.

Public MT

This strategy includes services like Google Translator or Bing Translate. Such services are readily available at no cost. However, the engines are completely unsecure and not trained for specific domains or particular use cases.

On-Premise MT

This strategy requires a company to deploy an MT server in its own IT environment. While this is the most secure option, it comes at a significant cost, is complex to deploy and manage, and requires ongoing maintenance. Importantly, this strategy often produces suboptimal MT output across multiple language pairs or content types.

Cloud MT

This strategy works like Public MT, is also hosted in the cloud, but creates a company-dedicated instance. Any data shared with the service is tightly secured and not shared with third parties. It provides additional capabilities around terminology customization and has other benefits. However, it can result in vendor lock-in and less-than-optimal MT quality across multiple language pairs.

Best of Breed MT

This is a single platform that allows companies to leverage multiple MT engines. It provides a single layer of terminology customization, an easy-to-manage interface and the ability to choose the best engines for different language pairs, industries or domains, and types of content.

No matter which strategy you are contemplating, selecting the right engine may be challenging without the proper data and MT experience. Lionbridge is an expert in MT. In addition to having more than two decades of MT experience, we have gathered a large volume of linguistic and quality data about MT technology that will help you make the right choice. This webpage provides basic information about the performance of popular MT engines for the most common language pairs to help you select the best option based on your content.

Which Machine Translation (MT) engine is best? There’s no simple answer.

When choosing among the many available MT systems, it’s important to note some engines address a specific function or domain. If your needs don’t align with that purpose, the engine may perform sub-optimally no matter how advanced it is. To determine the best option, first, identify why you are using MT.

If you want an MT engine for general use, it may be appropriate to use Google Translate or Bing Translator. If you seek MT services for a specific language or domain, you may achieve better results by turning to Amazon Translate or DeepL Translator.

Lionbridge’s Machine Translation Tracker analyzes engine performance monthly to help you figure out the best MT engine depending on the language pairs you use. The next time you ask which MT engine is best, reframe the question to, “Which MT engine is best for me?” And count on Lionbridge for guidance.

Want to learn more about the different types of MT technologies? Check out our blog Machine Translation in Translation.

Lionbridge Expert's Commentary

March 2023
February 2023

March 2023

Generative Artificial Intelligence (AI) has achieved a significant milestone: It outperformed a Neural Machine Translation (MT) engine in one of our comparative evaluations. Specifically, Large Language Model (LLM) GPT-4 provided slightly better quality than Yandex for the English-to-Chinese language pair, as shown in Figure 1.

This development is noteworthy because it’s the first time a different type of MT approach has beaten a Neural MT engine since the advent of Neural MT. Moreover, a non-MT approach — a multi-purpose language automation not specifically prepared for Machine Translation — has beaten the Neural MT engine.

Why should you care about this occurrence? If you are an MT provider, you must be at the forefront of technological advancements and consider how they will impact your current MT offering to stay competitive. If you are an MT buyer, you must be privy to these developments to make sound MT investments, which will likely include some LLM-based technology instead of pure Neural MT offerings.

It's worth noting that generative AI is still in its early stages. As such, it falls short in some key areas. For instance, it produces variable outputs during multiple runs, has Application Programming Interface (API) instability, and makes more errors than Neural MT engines. These issues must be resolved for the technology to mature, and we are already seeing improvements being made at breathtaking speed.

The incredible speed at which LLMs can improve supports the notion that LLMs will become the next paradigm for Machine Translation. We expect a hybrid period whereby Neural MT providers integrate some aspects of LLMs into the Neural MT architecture as the paradigm evolves.

Read our blog for a translation quality comparison between Neural MT and LLMs for two more language pairs and additional thoughts on whether it is the beginning of the end of the Neural Machine Translation paradigm.

—Rafa Moral, Lionbridge Vice President, Innovation

February 2023

Generic Machine Translation (MT) engines will frequently provide an adequate output for companies seeking to automate their translations. However, these engines may produce subpar suggestions — especially when dealing with technological or highly specialized content.

Companies seeking to improve Machine Translation (MT) results to meet specific goals can consider two options: MT customization and/or MT training. Either method — or a combination of both — can produce better results during the automated translation process.

However, the approaches differ from one another, and they are not interchangeable. Table 1 provides an overview of MT customization and MT training and offers some considerations when evaluating each method.

Machine Translation Customization vs. Machine Translation Training

	MT Customization	MT Training
What it is and how it works	An adaptation of a pre-existing Machine Translation engine with a glossary and Do Not Translate (DNT) list to improve the accuracy of machine-generated translations	The building and training of an MT engine by using extensive bilingual data from corpora and Translation Memories (TMs) to improve the accuracy of machine-generated translations
What it does	Improves MT’s suggestions for more accurate output and reduces the need for post-editing	Improves MT’s suggestions for more accurate output and reduces the need for post-editing
Specific benefits	Enables companies to adhere to their brand name and terminology and achieve regional variations	Enables companies to attain a specific brand voice, tone, and style and achieve regional variations
The risks of using it	The MT could make poor suggestions and negatively impact overall quality when executed improperly	MT training may fail to impact output if there is not enough quality data to train the engine; the MT could generate poor suggestions and negatively impact overall quality if inexperienced authors overuse terminology
When to use it	Ideal for technological and detail-oriented content and any content that requires: Accurate translations of terminology Regional variation, but you lack sufficient data for MT training	Ideal for highly specialized content, marketing and creative content, and any content that requires: A specific brand voice, tone, or style Regional variation, and you have enough data for MT training
Success factors	An experienced MT expert who can successfully manage input and output normalization rules, glossaries, and DNT	A minimum of 15K unique segments to adequately train the engine
Cost considerations	There is a one-time cost to update the profile that goes into the MT engine and some ongoing costs to maintain a glossary over time; costs are relatively inexpensive when factoring in the potential benefits and are typically lower than MT training costs	There are costs associated with the first training and potential costs for additional training, which may be considered over time if the MT performance monitoring indicates room for improvement; MT training can be worth the investment in certain cases when factoring in the potential benefits

Table 1. A comparison between MT customization and MT training

Read our blog to learn more about Machine Translation customization vs. Machine Translation training.

—Thomas McCarthy, Lionbridge MT Business Analyst

Click here to read previous expert commentaries.

Evaluating Overall MT Performance

Time

Chart 1: Comparison of the average quality of top MT engines for German, Russian, Spanish and Chinese. Many factors influence the performance of MT engine. These factors include the quality of the source content, the combination of source and target languages, the subject matter and whether the MT engine was trained. In the overall quality comparison, Google NMT leads in the long-term performance for all languages. Bing and Amazon are the MT engines that have experienced the highest fluctuations in their quality scores, with significant spikes in quality.

Per Language Pair Quality

Choose between German, Spanish, Russian, or Chinese in the drop-down menu

Time

Chart 2: Comparison of the average quality of top MT engines for specific language pairs.

Per Domain Performance

Select Domain/Subject Matter in the drop-down menu

Time

Chart 3: Comparison of the average quality of top MT engines for a specific domain.

For more insights and future trends about Machine Translation, read our Future of Language Tech blog post – Future of Machine Translation.

Lionbridge Machine Translation Tracker Methodology

Lionbridge uses inverse edit distance as a scoring method. The edit distance measures the number of characters (for Asian languages) or words (for Western languages) that need to be changed by a human post-editor before it achieves the quality level produced by a human translator. The higher the metric, the better the quality.

Out of the four MT engines we assessed, Google and Bing NMT performed best across different language pairs and for general content. However, specialized engines performed best for certain language combinations. For instance, DeepL had the strongest performance in German, and Amazon translated Chinese best.

^Disclaimer

^{Machine Translation engines in this report are assessed monthly by Lionbridge.
The data provided is for illustration purposes and each case should be treated and assessed individually.
This report is generated based on source data preselected by Lionbridge Machine Translation teams. The same source data is submitted to every Machine Translation engine and language pair each time, making comparisons between translation engines possible.
No customer data has been used in the generation of the report.}

Click the image below to view key definitions for understanding Machine Translation

Lionbridge Machine Translation Tracker

Comparing Four Machine Translation Deployment Strategies

Public MT

On-Premise MT

Cloud MT

Best of Breed MT

Which Machine Translation (MT) engine is best? There’s no simple answer.

Lionbridge Expert's Commentary

March 2023

February 2023

Machine Translation Customization vs. Machine Translation Training

Click here to read previous expert commentaries.

Lionbridge Machine Translation Tracker Methodology

Get Smaⁱrter™ About MT

Smaⁱrt MT™: Machine Translation for the Digital Age

The Future of Language Technology: The Future of Machine Translation

Machine Translation vs. Machine Translation Plus Post-Editing

Neural Machine Translation: How Artificial Intelligence Works When Translating Language

Machine Translation in Translation

Contact Us

WHO WE ARE

WHAT WE DO

INDUSTRIES

LAⁱNGUAGE CLOUD™

Lionbridge Machine Translation Tracker

Comparing Four Machine Translation Deployment Strategies

Public MT

On-Premise MT

Cloud MT

Best of Breed MT

Which Machine Translation (MT) engine is best? There’s no simple answer.

Lionbridge Expert's Commentary

March 2023

February 2023

Machine Translation Customization vs. Machine Translation Training

Click here to read previous expert commentaries.

Lionbridge Machine Translation Tracker Methodology

Get Smaⁱrter™ About MT

Smaⁱrt MT™: Machine Translation for the Digital Age

The Future of Language Technology: The Future of Machine Translation

Machine Translation vs. Machine Translation Plus Post-Editing

Neural Machine Translation: How Artificial Intelligence Works When Translating Language

Machine Translation in Translation

Contact Us

LAⁱNGUAGE CLOUD™

INDUSTRIES