What does
AImean today?
AImean today?
Artificial Intelligence, or AI, is a name for an aspiration, not an achievement or approach. No approach yet has approximated intelligence, nor do we have a roadmap for getting there. Even artificial
is an odd term: recent approaches have created algorithms that involve less artifice and more brute force than is common in other areas of computing.
AI has been a term since the 1950s, and has been applied fairly freely to anything the researchers of the day thought might result in human-like intelligence and anything building off of something previously called AI. The general pattern is a new approach is thought up; early work seems promising, so it gets included in AI; then its limitations are understood and do not approach intelligence,
but it’s still useful and already named AI, so it keeps that label while another new approach is thought up and added to the field.
As of 2023, AI is primarily used11 I mean used in mass media and other nontechnical spaces. Academics and AI researchers often use AI in a much broader sense with many subfields the mass media is mostly unaware of. to refer to highly data-driven algorithms; these are the subject of the rest of this post.
You have both a conscious and subconscious mind. Your conscious mind can only focus on a few things at a time, but can do very intricate and creative things within that limitation. Your subconscious can cope with hundreds of inputs and memories simultaneously but is limited to just a few operations.
The operation of the subconscious can be characterized in various ways, but one I find useful and memorable is to break it into three parts: pattern recognition, anomaly avoidance, and habit.
Pattern recognition correlates sensations with meanings. It recognizes a shape from a collection of optic nerve reactions, a face from a collection of nearby shapes, an expression from a particular position of a face, a mood from an expression, and so on.
Anomaly avoidance uses unexpected
as a proxy for bad
. Often that’s very useful, avoiding unexpected objects in the road and stopping you from swallowing unexpected objects in your food. It’s not clever about this, though: it will doubt medical advice from a physician dressed in an unexpected outfit even if you consciously know outfits are not related to medical knowledge.
Your habits automate actions without needing to engage your conscious mind, using pattern recognition, anomaly avoidance, and frequency of past actions to select what to do next. Habits are faster than conscious thought and avoid presenting you with choices unless faced with a situation they don’t have a way of handling.
The subconscious greatly improves the power of the conscious: it lets those few things you can focus on be high-level patterns like physician
instead of lower-level patterns like person in lab coat and slacks
or even lower-level patterns like which cones in your eye are firing with what frequency.
How people learn is not fully understood by cognitive science today, and what I know of what is understood is too large to cover here, but a rough approximation can be a useful working model.
It’s worth noting that the third step above is automatic, outside our conscious control. For example, if most people I meet in a particular profession have the same haircut I’ll subconsciously come to see that haircut as an indicator of that profession and recognize people claiming that profession but with a different haircut as an anomaly to be avoided.
Recent advances in AI generally try to replicate some of what the subconscious does. But they’re hampered in not having a conscious mind attached to them22 Efforts to emulate conscious minds were common from the 1960s through the 1990s, but failed to achieve that goal so thoroughly that the algorithms used then are rarely even called AI
anymore. and by our not actually wanting a mind, per se. We design and program computers so that they can do repetitive tasks quickly; the boredom, emotions, exhaustion, and related features that help make brains function are counterproductive to that goal, and I’m not aware of any serious effort to emulate them as part of a useful AI system.
Instead, we build stand-alone pattern recognizers, carefully pick what we want pattern
to mean in a given context, then feed them so many examples no human could ever look at them all. The huge number of examples helps compensate for the complete lack of understanding or intelligence, at the cost of being uncontrolled and expensive33 Expensive primarily in terms of the computer hardware and electricity needed to run the program that finds the patterns..
End result: systems that can recognize the patterns that are present in the data we show them, but recognize them indiscriminately without any actual understanding.
We know how to build decently-accurate pattern recognizers, and we have access to the Internet and its mindbogglingly-large number of examples. With those tools we can build system that can do many things that appear intelligent
despite having no underling intelligence.
We can build tools that can fill in incomplete content with new content that best fits patterns; for example
We can build tools that can find patterns between inputs and actions and use those to take the in-pattern action automatically; for example
We can build tools that can identify parts of an example that are least expected in a given context, mark them as likely in error, and suggest more pattern-fitting replacements.
All of this can look intelligent and useful, but it’s just subconscious-style pattern matching. Current news-making AI systems have no clue what they are doing, nor any part of their system that is capable of having a clue about anything. They’re modeled after subconscious minds with no conscious component attached. You can think about their operation as being like sleep-walking, like responding to a child while paying attention to your phone, like driving the car along a familiar route when you’re not paying attention, like trying to spell-check text in a language you don’t understand by comparing it to a word list you also don’t understand.
Because currently-popular AI approaches work on patterns, not understanding, if we want to control them we have to do so in terms of those patterns. Unless so controlled they’ll replicate whatever patterns are in their training data, and general amplify those patterns too, doing 100% of the time things the training data does 80% of the time. Because we need incomprehensibly large datasets to compensate for the limitations in current no-understanding AI algorithms, we generally pull that data from the Internet which has more misinformation and bad behaviors than we want to replicate.
To fix this, we apply a very limited kind of reasoning on top of the AI system. Typically, this looks like a set of different patterns, some to match and some to avoid matching. For example, suppose I want to generate text like people write online but don’t want to create hateful or abusive text. One way to do this is to train two different pattern recognizers: one that can generate text, another that can notice what kinds of text get flagged as hateful or abusive. I then run these together, trying to match the patterns in text generally while not triggering the abuse flagger.
While using AI systems to police or restrict other AI systems can work, it isn’t very reliable. There are always holes in the patterns44 Holes appear to be intrinsic to any model that lacks understanding. and we can always find those holes, either by clever design or by asking a different AI pattern tool to look for them.
Holes aside, there are more endemic problematic patterns in training data that AI does not yet have reliable approaches to solve; more on which in the next two posts.