It’s pretty smart, but what I find impressive is it’s ability to find the context-specific meaning of a word or phrase. What’s the tech behind it?
It’s pretty smart, but what I find impressive is it’s ability to find the context-specific meaning of a word or phrase. What’s the tech behind it?
Comments
It’s AI. Behind the scenes, it uses a deep learning model (kind of like the brain of an AI), specifically something called a Transformer model – the same kind of tech that powers ChatGPT (T stands for Transformer). These models are trained on huge datasets and learn to recognise patterns in how languages work.
Do you start every question that way? Cuz most smart people don’t.
Pattern analysis: they “simply” put a large amount of texts through the wringer and look for patterns, like “this word usually comes after that word if it’s close to the other one”. There’s nothing AI about it – it’s just number crunching. The same way computers play chess: have it run through an awful lot of games and copy what works.
Contextualized word embeddings 🙂 also Google invented the transformer model with their 2018 paper “Attention is All You Need” (it ofc built heavily off of existing research); GPT stands for Generative Pretrained Transformer. It’s a decoder-only transformer; google translate is probably using a full encoder/decoder transformer. Source: I’m a researcher in NLP.
Its just a database of words and their meanings in other languages.
Then they add a list of common phrases – and have people translate those phrases into other languages.
Then they just continue to add more and more phrases to the database over time.
Eventually it gets to the point where you can input a wide range of things and it will do the most specific conversion that it can find in the database.
Must depend on the language then, I find for the languages I need to translate most (Chinese, Thai, Japanese), ChatGPT usually provides way more accurate context-specific translations.