Insights · February 13th, 2023
In the heat and fervor of the hype around generative AI systems like ChatGPT we need to look towards the wise thinkers around what’s happening with artificial intelligence and what is unfolding. Here we want to share Meta’s Yann LeCun’s thoughts on auto-regressive Large Language Models (LLMs).
Yann LeCun is a French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta.
LeCun’s ‘unwavering opinion on current (auto-regressive) Large Language Models (LLMs)’ is:
- They are useful as writing aids.
- They are “reactive” and don’t plan nor reason.
- They make stuff up or retrieve stuff approximately.
- That can be mitigated but not fixed by human feedback.
- Better systems will come.
- Current LLMs should be used as writing aids, not much more.
- Marrying them with tools such as search engines is highly non trivial.
- There *will* be better systems that are factual, non toxic, and controllable. They just won’t be auto-regressive LLMs.
- I have been consistent with the above while defending Galactica as a scientific writing aid.
- Warning folks that AR-LLMs make stuff up and should not be used to get factual advice.
- Warning that only a small superficial portion of human knowledge can ever be captured by LLMs.
- Being clear that better systems will be appearing, but they will be based on different principles. They will not be auto-regressive LLMs.
- Why do LLMs appear much better at generating code than generating general text? Because, unlike the real world, the universe that a program manipulates (the state of the variables) is limited, discrete, deterministic, and fully observable. The real world is none of that.
- Unlike what the most acerbic critics of Galactica have claimed that LLMs *are* being used as writing aids; they *will not* destroy the fabric of society by causing the mindless masses to believe their made-up nonsense; and, people will use them for what they are helpful with.
On futurist.com we have previously shared some posts that we created with the help of ChatGPT
Nikolas can provide an in-depth keynote into Generative Artificial Intelligence and the platforms that are changing the world – DeepMind’s Alpha Code (GoogleLab), OpenAI’s ChatGPT, GPT-3.5, DALL-E, MidJourney, Jasper and Stable Diffusion – click here to read more.
Nikolas Badminton is the Chief Futurist at futurist.com. He’s a world-renowned futurist speaker, consultant, author media producer, and executive advisor that has spoken to, and worked with, over 300 of the world’s most impactful organizations and governments. He helps shape the visions that shape impactful organizations, trillion-dollar companies, progressive governments, and 200+ billion dollar investment funds.
You can preorder ‘Facing Our Futures: How Foresight, Futures Design and Strategy Creates Prosperity and Growth’ at Amazon, Bloomsbury, Barnes and Noble and other fine purveyors of books. We’s also love it if you considered pre-ordering from your local, independent book store as well.