Non-language Data and AI

This data was originally featured in the August 23, 2023 newsletter found here: https://www.trustinsights.ai/blog/2023/08/inbox-insights-august-23-2023-ai-culture-shift-non-language-data-and-ai/.

This week, let’s talk about data that isn’t necessarily language. In the rush to embrace generative AI for literally everything, sometimes we forget that an awful lot of data [a] isn’t in generative models due to training times (ChatGPT famously has no memory after September 2021 for most tasks) and [b] isn’t language per se that large language models are capable of processing so well.

For example, let’s take Spotify playlists. Spotify playlists are rich sources of data; when users publish them, we learn quite a bit about what music is resonating with pop culture. Yet if you were to ask most large language models what the most popular songs are on Spotify, most of them are going to come up with either no answer or hallucinatory answers, because that data isn’t readily available.

So why would a large language model like GPT-4 (the one that powers ChatGPT’s paid version) or others not be the right tool? Isn’t AI smart enough to know what to do? The short answer is no, and here’s why. In this specific use case – and there are many others, like page titles on your blog, company names in your CRM data, etc. – these song titles appear like language, but they’re not language.

Wait, what? How is a song title like “Welcome to the Black Parade” not language? To answer that question, we have to answer what language is. The OED defines language as this:

“the principal method of human communication, consisting of words used in a structured and conventional way and conveyed by speech, writing, or gesture.”

When we use words in a structured and conventional way, we are using not just the words, but words in a specific and rational order to convey meaning. That’s why large language models are good at language, because within their statistical libraries, they not only understand words, they understand the relationships of words to other words. They know that “I ate at Burger King” is NOT the same as “Burger I at King Ate”, even though those two pieces of text use exactly the same words.

The atomic unit of data in a Spotify playlist is the song, and most playlists do not use song titles in the structure of language. There are patterns, to be sure – playlists exist that are composed entirely of sad songs or songs for someone’s birthday, but that structure is not language because it’s not structured or conventional. You can’t pick up a pile of playlists and infer the meaning of the playlist solely from the use of songs and their order on the list.

When we use analytics data, we’re not necessarily interested in the language itself, but the dimensionality of the data. For example, suppose we were to put together a list of the top X songs on Spotify this summer. Each song title’s language is not something we’re especially interested in; what we’re interested is the frequency of song titles in their appearances on playlists.

That’s not language.

That’s math.

And language models, despite the bombastic claims of many, are not particularly good at math. In fact, natively, they’re quite poor at math because they can’t actually do math. They can only predict what the next word is likely to be in a sequence based on other words they’ve already seen. They know 2 + 2 = 4 only because they’ve seen that particular string of text – language – many times, and thus they have a statistical understanding that when 2 + 2 appears, the next words in the sequence are likely to be = 4.

When we look at song titles and popularity lists, we’re really treating them as just high-dimensionality data. We don’t actually care what the song title is (language), we just care about counting them (math).

Let’s take a look at an example. We downloaded more than 6,000 publicly published Spotify playlists from the last 3 months, which all have a format like this:

Wolf Alice – Don’t Delete The Kisses.mp3
Bleach Lab – I Could Be Your Safe Place.mp3
Zeph – world.mp3
Christian Kuria – Sunbleach.mp3
Del Water Gap – Ode to a Conversation Stuck in Your Throat.mp3
Carly Rae Jepsen – Bends.mp3

Is this language? No. It’s three dimensions of data – a musical artist, a song title, and a file format. Processing this data involves taking the data and slicing it up into its components so we can correctly count it.

The top 40 songs of the summer on Spotify look like this:

Again, this is math, and because it’s a math task, it’s not well suited for large language models to handle. To the extent that they can do so, it will be of lower quality and lesser outcome than traditional machine learning or even just basic data science techniques. It’s counting.

So what? What’s the point of this, besides having some new songs to add to your own playlists? Understanding what language is and is not helps us understand what tasks a large language model and its associated software will and won’t be good at. ChatGPT cannot capably give us the top songs of the summer, even if it had current data. The same is true for the contents of your CRM, content on your blog, etc. – if you’re trying to do math on something, a language model (at least in their current incarnations today) isn’t natively best suited to handle the task.

Suppose you still wanted to use generative AI. How would you do that? It turns out that language models have something of a workaround: you can ask them to code. You can ask them to write Python code or R code or the coding language of your choice to accomplish the math tasks you want to do, which is what I did with the Spotify playlists. I didn’t ask ChatGPT to tell me what the top songs of 2023 are. I did ask Spotify to help me write the code necessary for downloading the data, and then to process the data into tabular format, something that you can open in spreadsheet software. Why does this work? Because coding IS language, and thus language models are good at it.

Take this lesson and apply it to all your data. If you’re using language, a language model will help. If you’re not – if you’re doing math or other non-language tasks – then a language model will not be the right tool for the job.