(Human) Domain Expertise in the Age of Deep Learning
It’s a burgeoning area of AI—but where is it headed?
It’s a bit of an understatement to say that Deep Learning has recently become a hot topic. Within a decade alone, the field has made significant strides on problems once thought to be impossible, including facial recognition, generating text that mimics human writing, making art, and playing games involving strategy and intuition.
Given the buzz surrounding the (seemingly unreasonable) effectiveness of these algorithms, it’s easy to get lost in the extremes of speculation and skepticism towards Deep Learning. Instead, I’d like to focus on the following issues.
First, what is the role of human expertise in a world where Deep Learning is becoming more prevalent as a problem-solving tool? Second, where is the field of Deep Learning headed, and how can (human) expertise help move it forward?
As we will soon see, these two questions are intimately related. But first, let’s start with the basics.
Deep vs. human learning
To get a better scope of the picture, it helps to get a common understanding of what we mean when we refer to Deep Learning.
Simply put, Deep Learning refers to a broad range of algorithms, very loosely inspired by the human brain.
These algorithms take the form of networks, known as Deep Neural Networks (DNNs), which iteratively improve as they encounter new examples. It’s important to note that the kind of learning we refer to when we talk about Deep Learning is a very narrow subset of what we as humans might consider learning.
Human learning often involves the ability to explain, generalize, and even teach concepts that we have come to understand, all of which encompass different forms and levels of learning. It turns out that a lot of the questions surrounding these variants of learning are still open problems in Deep Learning, ones where domain expertise may play a large role.
Explainability
While Deep Learning algorithms outperform other machine learning methods at a wide variety of tasks, they are often notoriously difficult to interpret. Often, these networks are so complex that it’s difficult to know how they made their decisions. Like your pesky third-grade math teacher, we want these networks to “show their work”.
This problem of understanding how and why a network arrives at its decisions is known as explainability. In machine learning literature, this term is often used in conjunction with concepts of interpretability, understandability, and trust. Explainability is important because it gives us a better reason to trust that these algorithms are doing what we want, and enables us to troubleshoot them when problems arise.
It’s important to note that explainability isn’t a one-way street. In the vast majority of cases, you want to explain your decisions to someone. That’s where expertise comes in. Just as a doctor knows what information should be included in a medical report or diagram to reach a diagnosis, domain experts know what components are necessary to make sound decisions in their domains.
These experts are not simply peripheral consultants; they are necessary players in designing and implementing good problem-solving algorithms. Experts know what information is relevant and irrelevant to making a decision, and this domain expertise is incredibly valuable in constraining the kinds of algorithms that are worth considering. At the end of the day, it will be the experts who will be able to see if a decision is made correctly, and what steps need to be taken to fix it.
Without experts, we cannot design good algorithms, period. It’s telling that when DeepMind decided to develop AlphaGo to defeat one of the reigning world champions at the game of Go, they regularly consulted with Go experts to design their algorithm⁵. This idea is so important it’s worth repeating again: domain expertise isn’t peripheral. It’s the whole damn picture.
Generalization
Another key aspect of learning is the ability to generalize. In the Machine Learning community, generalization refers to the ability to learn beyond just the examples you’ve already seen. Any college undergrad knows how to brute-force memorize sections of the textbook, but how well do they really know the ideas they’ve crammed into their heads the night before? The test lies in the ability to answer questions that test similar, but slightly different information to the concepts they’ve studied.
It’s no different for Deep Neural Networks (incidentally, there are a lot of similarities that can be drawn between DNNs and college undergrads, particularly in terms of decision-making efficacy, but I’m writing a Medium article and not a novel so I’ll leave it at that).
There are two main kinds of generalization: generalization within distribution, and out-of-distribution. The first refers to examples where the examples you use to test your network are similar to the ones you used to train it. If I train a network on, say, thousands of pictures of apples, it should be able to identify new apple even if it hasn’t seen that specific image before. DNNs are actually fairly good at this.
But while DNNs require tens of thousands of examples to identify an apple, humans and other intelligent animals can learn after just a handful of examples. This is known as few-shot learning (sometimes used in relation to zero-shot or one-shot learning), referring to the ability to learn or generalize over just a few examples.
The second kind of generalization is out-of-distribution learning. Ever notice how your athletic friend seems to be good at not only running but also football and swimming? Or how the math whiz in your class is also great at physics and chemistry? That’s because those concepts, while distinct, are related. The ability to take one skill and apply it to another domain is known in Machine Learning circles as transfer learning.
Like few-shot learning, the idea of transfer learning centers around the idea that it shouldn’t take thousands of examples to learn concepts, especially if you’ve already learned a similar task before. Importantly, learning a new task shouldn’t require that you unlearn the old one.
Again, human expertise is necessary to understand how to approach the design of these algorithms as well. When you talk to experts, they often tie distinct examples together, while also describing their differences: “Playing guitar is kind of like playing piano, except…” This is exactly the kind of thing that transfer learning seeks to mimic. Seeking expert opinions is the key to understanding if transfer learning is effective across the domains it seeks to transfer to.
Meta-learning
A deep (pardon the pun) question in the Deep Learning community is how to develop algorithms not only to learn but to learn how to learn. This idea of learning how to learn is known as meta-learning, a term just as confusing as it is mysterious.
The key behind meta-learning is this: often what separates an expert from a simpleton is the way that they practice. The best violinists in the world practice for hours on end, but they do so smartly — that’s what makes them the best. If we can develop effective algorithms that can practice smarter, then we can make significant strides towards more-intelligent agents. It’s not hard to see where the value of expertise comes here too. Different domains require different approaches toward a variety of definitions of success. To practice well as a pianist may look very different from practicing well as a swimmer, for example.
What “smart learning” looks like may vary from discipline to discipline, ultimately requiring the informed design with human experts at the lead.
Looking forward
As impressive as the accomplishments of Deep Learning are, it’s hard to imagine that any of these networks would have achieved their level of prowess without the experts that were consulted to design and structure them. Human experts, not AI agents, facilitated the curation of datasets, the design of reward functions, and the deployment of these algorithms. On top of that, it’s up to human experts to translate the outputs of these algorithms to decisions and insights in the real world. For now, humans are the agents.
In the future, as Deep Learning becomes more prevalent in industry applications, people outside the field will be faced with adapting their roles to accommodate these new and unfamiliar statistical tools. As domain experts, they can contribute meaningfully by considering the role they play in facilitating the ability to interpret, transfer, and teach these algorithms to learn and perform better within their domain of interest. They can interface with ML researchers and engineers by helping to narrow the search space of possible algorithms.
I like to think of the role of expertise as constraining the set of possible algorithms from infinity to finite, and the role of ML researchers as narrowing from finite to a few. I leave it as an exercise to the reader to determine which of the two is harder.
Human expertise isn’t dead, not by a long shot. In fact, in the age of Deep Learning, it may be more important than ever.
