Meta’s Yann LeCun on auto-regressive Large Language Models (LLMs)

In the heat and fervor of the hype around generative AI systems like ChatGPT we need to look towards the wise thinkers around what’s happening with artificial intelligence and what is unfolding. Here we want to share Meta’s Yann LeCun’s thoughts on auto-regressive Large Language Models (LLMs).

Yann LeCun is a French computer scientist working primarily in the fields of machine learning, computer vision, mobile robotics and computational neuroscience. He is the Silver Professor of the Courant Institute of Mathematical Sciences at New York University and Vice-President, Chief AI Scientist at Meta. 

On Monday February 13th, 2023 he shared some useful thoughts on  Linkedin and we wanted to share here as well. See his post here.

LeCun’s ‘unwavering opinion on current (auto-regressive) Large Language Models (LLMs)’ is:

1. They are useful as writing aids.
2. They are “reactive” and don’t plan nor reason.
3. They make stuff up or retrieve stuff approximately.
4. That can be mitigated but not fixed by human feedback.
5. Better systems will come.
6. Current LLMs should be used as writing aids, not much more.
7. Marrying them with tools such as search engines is highly non trivial.
8. There *will* be better systems that are factual, non toxic, and controllable. They just won’t be auto-regressive LLMs.
9. I have been consistent with the above while defending Galactica as a scientific writing aid.
10. Warning folks that AR-LLMs make stuff up and should not be used to get factual advice.
11. Warning that only a small superficial portion of human knowledge can ever be captured by LLMs.
12. Being clear that better systems will be appearing, but they will be based on different principles. They will not be auto-regressive LLMs.
13. Why do LLMs appear much better at generating code than generating general text? Because, unlike the real world, the universe that a program manipulates (the state of the variables) is limited, discrete, deterministic, and fully observable. The real world is none of that.
14. Unlike what the most acerbic critics of Galactica have claimed that LLMs *are* being used as writing aids; they *will not* destroy the fabric of society by causing the mindless masses to believe their made-up nonsense; and, people will use them for what they are helpful with.

On futurist.com we have previously shared some posts that we created with the help of ChatGPT

• The Future of Humanity in 2023: Predictions, Implications, and Solutions (written by GPT-3)

• ChatGPT Discusses the Risks of Large language AI Models in Journalism

Nikolas can provide an in-depth keynote into Generative Artificial Intelligence and the platforms that are changing the world – DeepMind’s Alpha Code (GoogleLab), OpenAI’s ChatGPT, GPT-3.5, DALL-E, MidJourney, Jasper and Stable Diffusion – click here to read more.

_______________

Nikolas Badminton is the Chief Futurist at futurist.com. He’s a world-renowned futurist speaker, consultant, author media producer, and executive advisor that has spoken to, and worked with, over 300 of the world’s most impactful organizations and governments. He helps shape the visions that shape impactful organizations, trillion-dollar companies, progressive governments, and 200+ billion dollar investment funds.

You can preorder ‘Facing Our Futures: How Foresight, Futures Design and Strategy Creates Prosperity and Growth’ at Amazon, Bloomsbury, Barnes and Noble and other fine purveyors of books. We’s also love it if you considered pre-ordering from your local, independent book store as well.

Please contact futurist speaker and consultant Nikolas Badminton to discuss your engagement or event.