Our people are our pride, helping companies resonate with their customers for 20+ years. 

About Us
Key Facts
Leadership
Insights
News
Trust Center

 

Harness the Lionbridge Lainguage Cloud to support your end-to-end localization and content lifecycle

Lainguage Cloud™ Platform
Connectivity
Translation Community
Workflow Technology
Smairt Content™
Smairt MT™
Smairt Data™
Language Quality
Analytics

SELECT LANGUAGE:

MT Engine Analysis

Lionbridge Machine Translation experts examine top MT engine performance and share insights into the latest industry trends. 

How seriously are big tech companies taking Machine Translation (MT)? What are they doing to try to break away from the pack? Which engines perform best in any given month or any given language? These are some of the questions Lionbridge MT specialists set out to answer each month. Arm yourself with knowledge to make wise MT investments.

Executive summary for each month:

April 2022 — Yandex performance in April

March 2022 — Custom MT comparative evaluations

February 2022 — The future of Neural Machine Translation (NMT) 

January 2022 — MT engine performance in January

December 2021 — Lionbridge adds Yandex MT to the MT Quality Tracker competitive check

November 2021 — Bing Translator makes improvements

October 2021 — How Amazon’s MT engine is progressing

September 2021 — Amazon makes improvements to MT quality

August 2021 — Top tech companies and their MT engine development

April 2022

After several months of flat MT engine performance, Yandex has made some progress, particularly with its German engine.

In one detailed analysis, we saw advancements in Yandex engines' handling of sentences with punctuation characters — such as question marks, exclamation points, parentheses, and slashes — and units of measurement. These developments may result from some fine-tuning to the MT settings rather than improvements in the models. However, we also saw improvement in our tracking of rare terms, so Yandex’s progress may also be due to some refinements of the models or more data training.

Around this time last year, several MT engines showed some improvements that we found interesting. Is there a time pattern involved with these advancements? Will we see something like what we observed in 2021 this year? We’re tracking the MT performance of these engines, and we'll report our findings in the next month or so.

Generally, there is an increased interest in MT engine evaluation. Today, most everyone will agree that MT is a mature technology. People recognize the technology’s usefulness for almost any translation case — with or without human intervention and hybrid approaches. But MT users are still struggling to find the right way to evaluate, measure, and improve MT results.

—Rafa Moral, Lionbridge Vice President, Innovation

Top

March 2022

If you’ve been following these pages, you’re familiar with our generic MT comparative evaluations. Each month, we identify which MT engines are performing best for given language pairs and track engine improvements. In March, the performance of the different MT engines was flat. It’s a trend we’ve been noticing for some time already. As we commented last month, it may indicate that a new MT paradigm is needed.

While we share generic results, companies are increasingly pursuing custom MT comparative evaluations. Unlike the generic version, these evaluations take a company’s specific needs into consideration when determining the most advantageous MT engines.

When a company wants to start using MT or improve the way it currently uses MT, it is critical to identify which MT engines will work best. When we execute custom evaluations, we take a similar approach to the one demonstrated on this page, but we make recommendations based on a company’s content type and language pair requirements.

While custom MT comparative evaluations have been available for years, there’s greater demand for them. We attribute this trend to the important role MT plays in helping companies succeed in a digital marketplace.

—Rafa Moral, Lionbridge Vice President, Innovation

Top

February 2022

Google’s MT engine showed a tiny improvement during January and February of 2022, while the other engines we track remained stagnant. These observations may lead us to start asking some pointed questions. Is the Neural Machine Translation (NMT) paradigm reaching a plateau? Is a new paradigm shift needed given the engines’ inability to make significant strides? We observed similar trends when NMT replaced Statistical MT.

At the end of the Statistical MT era, there was virtually no change to MT quality output. In addition, the quality output of different MT engines converged. We see similar trends. While NMT may not be imminently replaced, if we believe in exponential growth and accelerating returns theories — and consider Rule-based MT’s 30-year run and Statistical MT’s decade-long prominence and note NMT is now in its sixth year — a new paradigm shift may not be so far away.

—Rafa Moral, Lionbridge Vice President, Innovation

Top

January 2022

During January, the main Machine Translation (MT) engines did not show significant changes in their performance. 

Google demonstrated small, incremental improvements across some languages and domains. The performance of most of the other engines has been flat. Microsoft had improvements over the last few months, but performance plateaued in January. Overall, the quality of Google Translate continues to lead in general-purpose MT technology.  

In December, we added a fifth MT engine to our tracker. By monitoring Yandex, we can better analyze the MT quality of the Russian language.

—Rafa Moral, Lionbridge Vice President, Innovation

Top

December 2021

In December, we added Yandex MT to our MT Quality Tracker comparative check.

According to our test sets, so far, Yandex:

  • Performs better than MS Bing, similarly to Google, and not as well as Amazon and DeepL for Russian.
  • Performs similarly to Amazon and MS Bing for German.
  • Does not perform as well as the main MT engines for the other language pairs we track.
  • Works well when addressing sentences that are longer than 50 words.

In other observations, MS Bing has improved its output in a nice way during the last months of 2021. In particular, translations into Chinese improved. Amazon has also made some strides. As we start the new year, Google is taking the baton and improving its output. Specifically, translations into Spanish, Russian, and German have improved. Yandex’s line has been flat during the five weeks we have been tracking it.

—Rafa Moral, Lionbridge Vice President, Innovation

Top

November 2021

After a few weeks of experimentation and fluctuation in overall performance, it’s clear that Microsoft NLP Engineers are on to something. Bing Translator has shown overall improvements during the past few weeks and improvements for Chinese in particular, making this MT engine last month’s big winner. Bing Translator has closed some gaps in most areas, even surpassing the performance of some of its competitors. Bing Translator remains one of the most trainable engines, and its enhancements position it to be a good choice when building customized models that are specific to your content.

—Jordi Macias, Lionbridge Vice President, Language Excellence

Top

October 2021

Amazon’s Machine Translation (MT) engines continued to evolve positively during the month of October, building upon what they started doing about a month ago. These continued enhancements are the second set of incremental improvements we’ve seen in the last few months.  

As a reminder, here are some of the areas where Amazon’s MT engines have continue to evolve over the past couple of months:

  • They are putting out a more informal style than before
  • They treat units of measurement differently
  • Both imperial and metric measurements are now consistently put out
  • Imperial measurements now appear before metric measurements
  • Numbers that correspond to measurements are now translated and correct
  • "Euro" is now spelled out and replaces the currency symbol €

—Jordi Macias, Lionbridge Vice President, Language Excellence

Top

September 2021

September has proven to be a good month for Amazon’s Machine Translation (MT) engines. First, the company improved its MT quality output for the German and Russian languages. Then, we saw spikes for the Spanish and Chinese language pairs. These enhancements are the second set of incremental improvements we’ve seen in the last few months.

Here are some more changes to the Amazon MT engines:

  • They are putting out a more informal style than before 
  • They treat units of measurement differently 
  • Both imperial and metric measurements are now consistently put out 
  • Imperial measurements now appear before metric measurements 
  • Numbers that correspond to measurements are now translated and correct 
  • "Euro" is now spelled out and replaces the currency symbol € 

Yolanda Martin, Lionbridge MT Specialist

Top

August 2021

All the big technology companies have developed their own MT engines, including Microsoft, Google, Amazon, Facebook and now Apple. Many other big players in markets outside of the U.S. are also competing in the space. Clearly, big tech companies believe that MT and Natural Language Processing (NLP) are must-have tools for today’s interconnected, global world.

Watch this space as Lionbridge follows the competition. We’ll identify the best MT engine options based on a company’s specific needs, taking its desired language pair and content type into account.

We expect the MT/NLP race to accelerate with so many top tech companies investing in this space. There’s no doubt that Apple—with its attention to detail and quality—will drive other companies to step up their game.

—Rafa Moral, Lionbridge Vice President, Innovation

Top