Session 1. Natural Language Processing

“Natural Language Understanding and Conversational AI”

Natural language processing (NLP) has made dramatic advances over the last three years, ranging from deep generative models for text-to-speech, such as WaveNet, through the extensive deployment of deep contextual language models, such as BERT. Pre-training with models like BERT has significantly raised the performance of almost all NLP tasks, allowed much better domain adaptation, and brought us human-level performance for tasks like answering straightforward factual questions. New neural language models have also brought much more fluent language generation. On the one hand, we should not be too impressed by these linguistic savants: Things like understanding the consequences of events in a story or performing common sense reasoning remain out of reach. But on the other hand, I will discuss how we now live in an era where there are many good commercial uses of NLP, with much of the heavy lifting already done in the construction of large but downloadable models. I present some of our work on understanding how these models learn to be so proficient, and how we can build new types of pre-trained models that are much more compute efficient. Finally, I turn to conversational agents, where neural models can produce accurate task-based dialog agents and more effective open domain social bots.

For more information, visit our websites: 
- Samsung Research: http://smsng.co/sr