We Got Access To GPT-3! | Gary, Walid and Leahy

In this special edition, Dr. Tim Scarfe, Yannic Kilcher and Dr. Keith Duggar speak with Professor Gary Marcus, Dr. Walid Saba and Connor Leahy about GPT-3.

We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations.

Do you think GPT-3 is a step towards AGI? Answer in the comments!

00:00:00 | Connor's take on LinkedIn

00:00:47 | Show teaser 

00:20:02 | Tim Introduction 

00:26:55 | First look at GPT-3, python sorting 

00:31:05 | Search strategy in LMs 

00:38:28 | Character analogies and Melanie Mitchell 

00:44:27 | Substitution cipher 

00:47:21 | Database prompt 

00:53:00 | Broader Impact Generation 

01:02:47 | Gary Marcus Interview (Robust.AI) 

01:29:11 | Connor Leahy Interview (Eleuther.AI) 

01:32:29 | Connor -- Tabular data 

01:33:41 | Connor -- other surprising examples? 

01:34:54 | Connor -- Is interpolated stuff new? 

01:37:43 | Connor -- structure of the brain / How GPT works 

01:41:21 | Connor -- Why cant GPT-3 reason? 

01:46:30 | Connor -- Missing information problem and ideas on our our brains work 

01:54:28 | Connor -- Topology of brain/models 

01:58:49 | Connor -- Hardware lottery / LSTM / Transformer 

02:01:41 | Connor -- NNs are just matrix program search 

02:10:32 | Connor -- Google -- information retrieval, the new paradigm, how to extract info from GPT-3, RL controller on top? 

02:19:38 | Connor -- Database example / "pattern matching is Turing complete" 

02:23:55 | Connor -- Did gpt3 understand? 

02:26:30 | Connor -- Are the GOFAI people right? 

02:27:40 | Walid Saba on GPT-3 

02:30:41 | Walid -- What is understanding and pattern recognition 

02:35:56 | Walid -- Chomsky would be happy 

02:42:13 | Walid -- Redefining success 

02:46:05 | Walid on Hinton 

02:47:34 | Walid on software 3.0 

02:53:11 | Keith -- We use machine learning because we cant write code to do the same thing 

02:59:36 | Keith -- What is pattern recognition and understanding 

03:14:06 | GPT-3 trials -- Turing Dialog 

03:15:35 | GPT-3 trials -- Mary Enjoyed a Sandwich 

03:16:19 | GPT-3 trials -- BBC has five offices in Germany. 

03:16:55 | GPT-3 trials -- Database prompt 

03:20:23 | GPT-3 trials -- Python 

03:20:31 | GPT-3 trials -- Patterns 

03:21:01 | GPT-3 trials -- Database again 

03:25:11 | GPT-3 trials -- GPT-3 experiment -- the trophy doesn’t fit in the suitcase 

03:27:32 | GPT-3 trials -- Scrambling words 

03:30:41 | GPT-3 trials -- PDF cleanup example (Gwern) 

03:35:03 | GPT-3 trials -- Word breaking and simple text patterns  

03:37:16 | GPT-3 trials -- Typing of entities  

03:38:30 | GPT-3 trials -- Basic Python append 

03:39:07 | GPT-3 trials -- Automatic programming?  

03:42:31 | GPT-3 trials -- Passive aggressive dialog input 

03:44:39 | GPT-3 trials -- symptoms of depression 

03:45:43 | GPT-3 trials -- Red shirts reasoning challenge 

03:49:59 | GPT-3 trials -- Binary encoding  

03:50:36 | Concluding statements from Walid, Tim and Yannic