Module 5: Machine Learning and its Needs
Mandatory Watching - videos with quizzes - finish before 11 am, Tuesday, July 7
- 5.1 — Lecture: “Data is the New Oil”, 40 minutes (April 15, 2026)
- 5.2 — Lecture: Physical Needs for Neural Nets, 45 minutes (April 15, 2026)
- 5.3 — Art of the Problem: From Bacteria to Humans, 17 minutes (June 27, 2017)
- 5.4 — The Bitter Lesson, 9 minutes (April 7, 2019)
- 5.5 — Crash Course AI: Training Neural Networks, 12 minutes (Aug. 30, 2019)
- 5.6 — Hank Green: Why is Everyone So Wrong about AI Water Use?, 24 minutes (Dec. 8, 2025)
Written assignment - submit by end of Tuesday, July 7, give feedback by meeting on Thursday, July 9.
5. Deep Research and contextualize
Use ChatGPT and Gemini to make “Deep Research” reports on your assigned topic. Use Claude to help understand and contextualize them.
Further watching/reading
- The bitter lesson - the text of the brief note discussed in the video (March 13, 2019)
- Vox - Can AI Help Us Predict Extreme Weather?, 8 minutes (Feb. 21, 2024) - an example of the bitter lesson, where more data seems to beat physical theory
- Data Center maps
- NREL - current and future data centers in the US, scaled by power
- Datacenter Map - worldwide map, not scaled
- IEA - worldwide map, scaled by power
- Epoch AI - where many of my other charts in lecture 5.2 came from
- What’s in a data center
- More details on the physical needs of a data center, from Brian Potter at Construction Physics (June 20, 2024).
- How much computing power is in a data center?, from Brian Potter at Construction Physics (March 19, 2026)
- The history of The Netflix Prize, on Thrillist (July 7, 2017)
- Algorithmic Simplicity, THIS is Why Large Language Models Can Understand the World, 18 minutes (March 29, 2025) - discussion of why bigger models were originally expected to be worse, though they turn out to be better
- The limits of training data
- When AI’s Output is a Threat to AI Itself - NYTimes article (Aug. 25, 2024) about how the rise of AI-generated text on the internet is corrupting the input training data for future generations of AI
- Bloomberg Originals, How AI Got a Reality Check, 9 minutes (Dec. 20, 2024), about how training data is hitting its limits
- Welch Labs - AI can’t cross this line and we don’t know why, 24 minutes (Sept. 13, 2024), video about how the performance of various neural nets has depended specifically on amount of data and computing power and size of model
- Using AI Is Not Bad For The Environment, by Andy Masley (Jan. 13, 2025) - an in-depth analysis of how much energy and water it really uses
- Alison Gopnik and Ted Chiang on raising AIs like children, (Jan. 12, 2024) 55 minute podcast from the Center for Advanced Studies in Behavioral Sciences
- Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell, “On the Dangers of Stochastic Parrots” (March 1, 2021) - several of the authors lost their jobs at Google for writing this paper about the costs of building LLMs, and one only kept it by publishing under an obvious pseudonym