Authors: Derrick Harris, Matt Bornstein, and Guido Appenzeller

Original article: https://a16z.com/2023/05/25/ai-canon/

Translator: WaytoAGI (Way to AGI)

Part Two: Table of Contents: a16z's Recommended Advanced Classics

AI research is growing at an exponential rate. It's difficult even for AI experts to keep up with everything being published, let alone for beginners.

Therefore, in this article, we share a curated list of resources we trust for gaining a deeper understanding of modern AI. We call it the "AI Canon," because these papers, blog posts, courses, and guides have had an enormous impact on the field over the past few years.

We begin with a gentle introduction to Transformers and Latent Diffusion models, which are driving the current AI wave. Next, we dive into technical learning resources; practical guides to building large language models (LLMs); and analyses of the AI market. Finally, we provide a reference list of landmark research works, starting with Google's 2017 "Attention is All You Need"—the paper that introduced the Transformer model to the world and ushered in the era of generative AI.

Easy Introduction…

These articles require no specialized background knowledge and can help you quickly grasp the most important aspects of the modern AI wave.

• Software 2.0: Andrej Karpathy was one of the first to clearly explain (in 2017!) why the new AI wave truly matters. His argument is that AI is a new, powerful way to program computers. With the rapid improvement of large language models (LLMs), this argument has proven prescient and provides a good mental model for the possible progression of the AI market.

  • Translation: Software 2.0

• State of GPT: This is also by Karpathy, a very accessible explanation of how ChatGPT/GPT models generally work, how to use them, and the possible directions for research and development.

  • Translation: State of GPT

• What is ChatGPT doing … and why does it work?: Computer scientist and entrepreneur Stephen Wolfram provides a long but readable explanation of how modern AI models work from first principles. He follows the timeline from early neural networks to today's LLMs and ChatGPT.

  • Translation: What is ChatGPT doing … and why does it work?

• Transformers, explained: This article by Dale Markowitz is a shorter, more direct answer to the question "What is an LLM and how does it work?" It's a great way to ease into the topic and build an intuitive understanding of the technology. The article is about GPT-3 but still applies to newer models.

  • Translation: Transformers, explained

• How Stable Diffusion works: This is the computer vision counterpart to the previous article. Chris McCormick explains how Stable Diffusion works for non-experts, helping you build an intuitive understanding of the technology from the perspective of text-to-image models. If you want an even easier grasp, check out this comic from r/StableDiffusion.

  • Translation: How Stable Diffusion works

---

Foundational Learning: Neural Networks, Backpropagation, and Embeddings

These resources provide you with a foundational understanding of machine learning and AI core concepts, ranging from the basics of deep learning to university-level courses for AI experts.

Explanatory Resources

• Deep learning in a nutshell: core concepts: This is a four-part series from Nvidia that introduces the fundamentals of deep learning in practice as of 2015, making it a great resource for those just starting to learn AI.

  • Translation: Deep Learning (1) Core Concepts

  • Translation: Deep Learning (2) History and Training

  • Translation: Deep Learning (3) Sequence Learning

  • Translation: Deep Learning (4) Reinforcement Learning

• Practical deep learning for coders: A comprehensive, free course that explains AI fundamentals through practical examples and code.

• Word2vec explained: A simple introduction to embeddings and tokens, which are the building blocks of LLMs (and all language models).

  • Translation: Word2Vec Explained

• Yes you should understand backprop: A more in-depth article on backpropagation if you want to understand the details. To learn more, check out the Stanford CS231n lecture on YouTube.

  • Translation: Yes! You Should Understand Backpropagation

Courses

• Stanford CS229: Andrew Ng's introductory machine learning course, covering the fundamentals of machine learning.

• Stanford CS224N: Chris Manning's deep learning for natural language processing (NLP) course, covering NLP fundamentals through the introduction of first-generation LLMs.

---

Translation of Introductory Articles

📊 This is an embedded Feishu content (table/canvas, etc.), view the full version in the Feishu knowledge base

Community member interpretation:

Zhang Haigeng

https://mp.weixin.qq.com/s/cpLDPDbTjarU0_PpBK_RDQ