Meta's Hokkien-English synchronous translation function has sparked a discussion in Taiwan, leading people to wonder what Meta is planning to do with this new technology. Ming-Shun Tsai, the president of school affairs for the Taiwan AI Academy, stated that this is likely to promote breakthroughs in social media platforms and the metaverse.
The video of Meta CEO Mark Zuckerberg and Taiwanese Meta engineer Peng-Jen Chen demonstrating the Hokkien-English synchronous AI translation system has caused quite a stir. This system allows Hokkien speakers to converse with English speakers. Chen mentioned that he developed this system so his Taiwanese (a Hokkien variant used in Taiwan)-speaking father can communicate with others. He hopes that his daughter can continue this language lineage as well.
Meta stated that besides developing methods to evaluate the accuracy of the translations, it also created the first standard dataset for two-way Hokkien-English speech translations based on a Hokkien dialogue corpus named Taiwanese Across Taiwan. The open-source code of the translation system is part of Meta's Universal Speech Translator (UST).
The main challenge during development is the lack of Hokkien language material. The solution was to use Mandarin as a middle language to create tags and manual translations. In practice, this means translating English (or Hokkien) to Mandarin characters first, then translating that to Hokkien (or English) before adding them to the training dataset.
As for where this new function is going to be used, Meta stated that the system is still in development. They will learn from this experience and expand this to translation research of other spoken languages in the future.
How can this new technology benefit the creation of the metaverse? According to Meta, languages are a tool for people all over the world to show culture and identity. There are still many people in the world who couldn't access internet content or fully participate in social media discussions with their native language. This is especially prevalent in Africa and Asia, where billions of people use a variety of different languages. To advance the future development of the metaverse, Meta AI researchers have already established the "NLLB" project (No Language Left Behind).
When asked about Meta's announcement of this technology, Tsai pointed out that it could be using speech unit translation, a speech-to-text function. Although the Taiwanese language doesn't have an official writing system, people in Taiwan have already created Taiwanese characters and built a corpus for them.
When asked about the reason for Meta's sudden involvement with Hokkien translation, Tsai speculated that it was because Meta certainly needs to create new topics. This new technology also helps push its metaverse plans as it can reduce the language barrier for users interacting in the virtual world. Said technology can also be used in its Facebook and Instagram social platforms to allow cross-generation users to communicate smoothly.
Besides Taiwan, China, and the Southeast Asia region, there are also many Hokkien speakers in the Chinese immigrant communities in the US. The first generation of Taiwanese immigrants in the US could be fluent in both Taiwanese and English, but their grandchildren may only speak English. Tsai believes that once the Hokkien-English translation technology becomes mature, it can also promote international communication between family members. In the future, Meta could add translation functions for other minor languages like Hakka. However, it will likely consider the user base and market size of each language.
Photo: Meta CEO Mark Zuckerberg and Taiwanese engineer Peng-Jen Chen demonstrating the Hokkien-English synchronous translation function.
Credit: Meta
Article translated by Jack Wu
