It may come as a surprise to discover that vision models and text models trained on apparently different sets of inputs converge in the same model, but it really shouldn’t. After all, language in isolation has no meaning. It is merely a set of labels that represent sounds that we associate with experiences.<\/p>\n\n\n\n
“Hang on a minute”, I hear you say, “language has no meaning? That’s nonsense”.<\/p>\n\n\n\n
I disagree.You’ve probably tried learning a new language. During that process there’s a time where a new word has no meaning in itself. It invokes nothing in your mind until you convert it into your native language. Only then does the meaning emerge.<\/p>\n\n\n\n
Imagine a new creature referred to as the “gronda”. The word means nothing until you meet one. Even a vivid description only gives an association with existing experiences, none of which are about “grondas”.<\/p>\n\n\n\n
Maybe, based on the description you’re given, you imagine something similar to a large dog, but that’s just a translation of the description into an existing experience. That’s all language is – a tagging system for your memories.<\/p>\n\n\n\n
Meet a gronda and everything changes. Now the word gronda links to real experiences which you believe you explain. That belief is wrong. You know it is because it happened to you, but you’re so conditioned to believe language conveys meaning you think this time it’ll be different.<\/p>\n\n\n\n
Have you met a wolf in the wild, or a tiger? If you have, you’ll know that no amount of descriptions about big dogs or huge cats came even remotely close to the reality.<\/p>\n\n\n\n
Once you have the experience, those descriptions suddenly acquire meaning. They become a tag for a new memory, not just a tag to an existing unrelated memory.<\/p>\n\n\n\n
That’s why LLMs trained on text form similar models to those trained on images. The words are just tags for the things in the images and inevitably they group up in the same way.<\/p>\n\n\n\n
It’s also one of the reasons that LLMs can’t do anything meaningful with language beyond paraphrasing or regenerating what’s already there. They have no experiences to relate the text to. Even multi-modal models fall far short. Add in sound, visions and even taste and you’re still missing a big part of the picture.<\/p>\n\n\n\n
Emotions.<\/p>\n\n\n\n
Emotions don’t exist at the language level, or even in the five main senses that we could potentially imbue an AI with. They exist at a deeper chemical level and play a huge role in the shaping of human synaptic connections. <\/p>\n\n\n\n
Without this, language means nothing. A tiger is a big cat, but when you encounter a tiger very little of the experience is about that. It’s primarily about the emotional response. The words, and even the images, are just tags that connect to that emotional memory.<\/p>\n\n\n\n
Language, textual or visual, is just a way to connect shared experiences and without a shared experience there is very little communication taking place.<\/p>\n\n\n\n
In conclusion, the reason people trust LLMs too much is because they believe too strongly in the ability of language to communicate new concepts in the absence of direct experience.<\/p>\n\n\n\n
Words are nothing without shared experience.<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
It may come as a surprise to discover that vision models and text models trained on apparently different sets of inputs converge in the same model, but it really shouldn’t. After all, language in isolation has no meaning. It is merely a set of labels that represent sounds that we associate with experiences. “Hang on […]<\/p>\n","protected":false},"author":1,"featured_media":457,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9,103,11,68,10],"tags":[6,36,120,119,118],"class_list":["post-456","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","category-neuroscience","category-personal","category-science","category-technology","tag-ai","tag-communication","tag-emotions","tag-humanity","tag-language"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t