I’ve believed for at least 25 years – and that’s provable in court – and probably longer that the most important aspect of AI is Natural Language Understanding (NLU), the bigger-brother of NLP (Natural Language Processing).

NLU is what AIs in movies do. And what Siri and Alexa try to do.

NLU AIs should ultimately understand human languages in context and in domain and with common sense. And not forget what you told them a few seconds ago. Or last year.

This text understanding part of the puzzle (we can separate the voice to text component out as a solved sub-problem) is proving the toughest (compared to vision and voice recognition and numerical analytics).

It’s because human languages are so rich in vocabulary and structure. And yet so apparently unstructured and flexible.

And – from a machine learning (ML) point of view – so unlabelled.

We don’t have much pre-exisiting data that is labelled with what the meaning is.

Because how would we even do that?

Understanding human languages involves not just syntax and semantics but . . understanding of the world. People. Cities. Animals. Common sense. Technologies. History. Goals. Emotions. World affairs. Science. Arts. Everything.

I also claim here that NLU is the most important AI branch.

It’s because blind and deaf people can be just as intelligent as other humans. Of course they can and are. It’s our understanding of language, reducible to text at it’s core, that therefore distinguishes us from other animals and insects and non-human-like AIs.

And if an AI can properly understand human language, in the unstructured form of paragraphs of text (not tables of ordered data) then it can go on to:

• read the whole internet
• answer questions
• ask questions
• follow instructions

Insect like AI’s just wont be able to do that.

In fact, controversially, I think we can move straight from machine learning to NLU without moving through an insect-like phase of AI. That is, without developing some sort of AGI or Artificial General Intelligence. Just build an NLU with some human-like goals and it will effectively be an AGI.

And I think we can do it with this generation of computing power.

We don’t even need to wonder, necessarily, about sentience and consciousness etc. Because we already know how useful such an AI would be from watching movies, irrespective of whether it ‘became sentient’. The Star Trek main computer is pretty useful without it wanting to leave the ship or demand better pay.

With or without a robotic body an NLU-capable AI is really handy.

Here’s my next claim (as an AI researcher): NLU will be cracked in the next 5 years to the extent that it will be almost indistinguishable from humans and easily pass Turing tests. And it will be ubiquitous.

Over the next few weeks I’ll discuss the Deep Learning (DL) technologies like CNNs, RNNs and Seq2seq and attention-based Transformer neural networks and the companies that are getting us there step by step, especially in the last four years.

These advanced AI technologies will enable automated customer service, automated research assistants, better web search, personalised education and text creating bots.

That work.

Just to name a few.

But much more than that.

AI’s equiped with NLU will be able to learn to do any job that we can do with our typing fingers.

Scary? Maybe. But it’ll be extremely useful and amplify our talents. Leave us to the genuine creative and finicky aspects, even in science and technology.

Let alone arts.