You Can Lead a Model to Water....

You can lead a model to water but you can't make it drink. Ok, it might be a bit of a mangled analogy but you can see the issue; If you spend all your time teaching a model to walk from point A to point B, when you get to the next step it doesn't know what to do. It's just like teaching a toddler to do new things. If they haven't seen it done before they are going to struggle to do it themselves. So what does this mean for machine learning models? How can they understand things it's never been exposed to before?

For any type of model, be it predictive or classification or even generative, if it hasn't seen the data it is recieveing before it can have some interesting results; and not necessarily good ones. A most basic example is a classificaiton model trainined to classify different breeds of dogs from images. It's great at it! But what happens when we give it an image of a cat? It doesn't know what a cat is so it .. takes the equivalent of a guess!

This model doesn't know it was given a cat instead of a dog so it just process the image like all the others. No errors or flags, just an interesting result! The same can be said of a prediction model that predicts when the next piece of space debri will decend into the lower atmosphere. The model doesn't know that you've given it the positions of the ISS instead of the space debris but it'll still give you an answer.

What about generative models, like LLMs? They are the the closest in machine intellegence that resembles a humans way of writting. Is it because they really are artifical intellegence? No, just like predictive or classification models it has learnt from data. In the case of LLMs; This is a MASSIVE amount of text data extracted from the internet. So when we ask an LLM a question it combines a complex search for similar text to what you've written and some guidelines and parameters to give the LLM a shorter or longer leash depending on how creative an answer you'd like and anything it shouldn't return.

If we asked an LLM to write a brand new, not heard of before, sci-fi novel it is very likely that there will be essences of sci-fi that has come before. Because this is what the LLM is trained on: Things that already exist. So even if we specifically fine-tune an LLM for sci-fi, lead it to water, it still can't create something completley new.

So next time you're using a model, of any kind, think about what went into the training, how or what the model was trained for and what you are actually expecting of the output. You may be surprised how this helps you understand what your "AI" is thinking.