Error 404: Why Common Sense is So Elusive in Artificial Intelligence

Common sense feels easy. As humans, we use it constantly throughout our daily lives without even thinking about it. Yet, when it comes to developing artificial intelligence with robust common sense capabilities, we run into a metaphorical “Error 404” message. The elusive nature of common sense poses a significant challenge in AI research.

In this comprehensive guide, we’ll explore why coding common sense into AI remains an unsolved challenge, the key barriers standing in the way, and promising approaches that may finally crack this AI frontier in the years ahead. By the end, you’ll have a deeper appreciation for how truly complex it is to create AI systems with the multidimensional understanding of the world that we humans intrinsically possess.

Why Is Common Sense So Important for AI?

Before diving into why common sense is so difficult for AI systems, it’s important to take a step back and understand why it’s such a critical capability in the first place.

At its core, common sense reasoning allows us to draw upon basic facts about the world around us in order to function and communicate effectively day-to-day. It encompasses diverse forms of reasoning about:

• Objects – Understanding what things are, their properties, and how they interact. For example, knowing that a cup can hold liquid or that a rock is hard.
• People – Interpreting human behavior, motivations, and mental states. For example, realizing that people get sad when scolded or that giving someone a gift makes them happy.
• Events – Predicting what will happen next based on an understanding of cause and effect. For example, realizing that dropping a glass will likely cause it to shatter.
• Language – Interpreting figurative language, ambiguity, sarcasm, and more based on social/cultural awareness. For example, understanding idioms like “break a leg” or that a joke is meant humorously, not literally.

This form of everyday reasoning comes instinctively to humans through years of lived experience in the world. We accumulate vast amounts of commonsense knowledge through direct interaction and observation.

In contrast, AI systems start from a blank slate. Without the right common sense capabilities built in, today’s AIs take instructions overly literal and fail to robustly generalize beyond their training data.

This severely limits their real-world usefulness. As AI researcher Ernest Davis puts it: “A system that has no common sense should not be trusted with important decisions about the real world.”

Moving forward, achieving human-like flexible intelligence will require closing this commonsense gap. That’s why ongoing research into areas like natural language processing, computer vision, and robotics increasingly involves integrating more sophisticated commonsense capabilities. The payoffs stand to be immense – but as we’ll explore next, the path to get there remains foggy.

The Key Barriers to Coding Common Sense

So with such tremendous potential benefits, why has endowing AI with common sense proven so difficult? There are a few fundamental challenges that help explain the bottleneck:

1. Sheer Vastness of Commonsense Knowledge

The scope of human common sense is almost unimaginably huge. From physical interactions to social dynamics, we effortlessly leverage countless facts and inferences about the world.

Consider this – researchers estimate the average American high school graduate knows at least 300,000 commonsense “facts” about the world. And that’s just a small fraction of the full scope of human commonsense knowledge.

This poses a monumental knowledge engineering challenge. Manually coding so many diverse facts into inflexible rules or a knowledge base is intractable.

Modern techniques like machine learning from big data provide some hope. But lacks of training data for rare concepts and difficulties capturing relational knowledge also pose hurdles to learning general common sense from data alone.

2. Contextual Flexibility and Nuance

Commonsense reasoning also requires flexibly adapting to context. For example, a rock is generally hard, but sandstone can actually be quite soft and easy to break apart.

Human common sense handles such nuance with ease. We rapidly integrate contextual cues like whether a rock sample is sandstone or granite. But such dynamic reasoning proves difficult for AI.

Likewise, social situations require navigating subtle cues and implicit assumptions that AI systems currently miss. Understanding sarcasm, for instance, requires reading between the lines based on tone, culture, and empathy.

This gets at the deeper challenge of representing commonsense concepts relationally rather than in isolation. Simple taxonomic knowledge alone fails to capture the fluid, adaptive nature of human reasoning.

3. Lack of Grounding in Human Experience

Finally, common sense intrinsically builds upon our experiences as embodied agents moving through the physical world. Developmental psychologists note how contextual experience grounds human cognition.

Yet most AI systems today lack such lived experience of objects, people, and events from which to derive grounded commonsense models. Some researchers argue sufficiently advanced AI could learn from scratch through virtual environments. But the fact remains – today’s AIs are missing this critical foundation.

Taken together, these realities explain why teaching machines common sense has proven such an intractable endeavor. Cracking this challenge will require breakthroughs across knowledge representation, reasoning, perception, and beyond.

Promising Frontiers to Look Out For

While common sense remains a hard unsolved problem, steady progress is occurring across fronts. Key directions where researchers hope to make headway include:

Commonsense Knowledge Bases

Manually constructing comprehensive commonsense databases is infeasible. But smaller-scale knowledge bases focused on targeted domains show promise for augmenting AI:

• ConceptNet – Knowledge graph containing over 21 million edges describing semantic relationships between words/phrases.
• ATOMIC – Collection of if-then relations about events and people’s intents/reactions.
• VisualGenome – Database of objects, attributes, and relationships in visual scenes.

By providing structured commonsense knowledge, these resources aim to fill gaps left by data alone. Combining them with large language models and reasoning techniques offers potential to scale up capabilities.

Self-Supervised and Multi-Modal Learning

Given vast data needs, self-supervised and multi-modal learning provide additional avenues:

• Language models like GPT-3 exploit self-supervision on vast text to acquire world knowledge. Fine-tuning them on commonsense tasks shows promise.
• Multi-modal learning from images, videos, and text could help ground understanding in sensory experiences.

However, these approaches also risk learning spurious correlations without explicit commonsense structure. Combining data-driven learning with structured knowledge appears key.

Neuro-Symbolic Models

Integrating neural learning with symbolic reasoning could provide the best of both worlds:

• Neuro-symbolic approaches aim to model relations and abstract reasoning while leveraging benefits of deep learning.
• They hope to reconcile flexible statistical learning with structured representation of concepts.

This direction remains nascent but responds to limitations of pure data-driven methods. It offers hope for learning broadly generalizable common sense.

Interactive Learning

Learning interactively through dialogue, experiential environments, and feedback provides another avenue:

• Dialogue-based learning allows correcting faulty assumptions and answering follow-up questions to refine common sense.
• Virtual environments enable controlled interaction to learn grounded knowledge through experience.

Such techniques could accelerate acquisition of multidimensional commonsense knowledge.

The Long Road Ahead

While steady progress is happening, we are still far from machines possessing the flexible situational understanding that humans take for granted. But with integrative approaches combining strengths across knowledge bases, learning paradigms, and reasoning, the breakthroughs could arrive sooner than we think.

What seems certain is that achieving strong AI will require unlocking common sense in some form. As AI pioneer Marvin Minsky famously stated, “in the long run, the only way to get intelligent computers is going to be to put the common sense in by hand.”

The hand of human researchers remains indispensable to making this dream a reality. But the seeds planted now through collaborative AI research could blossom sooner than we realize.

When the history books reflect back on this period, cracking common sense may very well be seen as the seminal breakthrough that unlocked artificial general intelligence. Only time will tell, but the quest continues onward!

Frequently Asked Questions About Common Sense in AI

What are some examples of common sense?

Some examples of everyday common sense include:

• Understanding objects and their properties – knowing water is wet, rocks are hard, glass breaks if dropped.
• Predicting basic physical interactions – realizing that a stack of plates might fall over if pushed.
• Interpreting social/emotional cues – recognizing people get sad when insulted or happy when given gifts.
• Parsing ambiguous or figurative language – understanding sarcasm, metaphors, humor.
• Adapting reasoning to context – knowing sandstone is softer than granite though both are types of rock.

Why is common sense difficult to represent algorithmically?

Several key properties of common sense make it hard to represent algorithmically:

Sheer vastness – The scope of human commonsense knowledge is too large to manually encode.

Contextual flexibility – Common sense adapts to nuanced contexts unlike rigid programs.

Grounded in experience – Common sense derives from lived sensory interactions unlike today’s abstract AIs.

Relational structure – Simple facts alone fail to capture the richness of flexible human reasoning.

These factors explain the persistent difficulties in teaching machines common sense.

How is common sense learned in humans?

Human common sense develops through:

• Experiential learning – Direct sensorimotor interactions with the everyday world.
• Social learning – Observing other people and accumulating shared cultural knowledge.
• Communication – Dialogue, storytelling, and information exchange with others.
• Introspection – Building conceptual representations by reflecting on experiences.

This multi-modal grounded learning occurs over years of development and remains hard to replicate in AI systems.

What are some promising areas of common sense research?

Some promising directions include:

• Structured commonsense knowledge bases – ConceptNet, ATOMIC
• Self-supervised multi-modal learning – GPT-3, imagery/video
• Neuro-symbolic models – Combining neural nets and symbolic reasoning
• Interactive learning – Dialogue agents and virtual environment simulation
• Transfer learning – Adapting knowledge across domains

Researchers hope combinations of these approaches may finally help crack common sense AI.

How long will it take to develop human-like common sense in AI?

Most researchers believe human-level common sense remains at least decades away. However, progress could accelerate with approaches combining strengths across data, knowledge bases, reasoning, and virtual interaction. Integrative methods building on current foundations offer hope for faster advances. But only time will tell how the challenges are ultimately solved.

The Bottom Line

At present, endowing machines with the flexible commonsense reasoning humans possess remains a frontier AI challenge. Coding the vast, nuanced, and relatonal nature of commonsense knowledge by hand is intractable.

Key barriers standing in the way include the sheer scope of everyday knowledge, the contextual fluidity of common sense, and its grounding in embodied experience still lacking in AI systems.

But steady progress is occurring – from curating structured knowledge bases to newer multi-modal learning paradigms. Combining these strengths through integrative neuro-symbolic and interactive learning techniques offers hope for finally replicating flexible humanlike common sense in artificial intelligence.

While the road ahead remains long, cracking open robust common sense may be the seminal breakthrough that finally unlocks artificial general intelligence. The fruits of this quest stand to be paradigm-shifting – even if widespread human-like common sense in machines remains years or decades down the road.