I was not aware of the “Free Lunch Theorem” (Wolpert & Macready, 1990s), but it is a powerful idea. In a nutshell, if you average all the problems, every learning algorithm has the same performance.
So if a model performs best for vision, it has to perform worse in something else. And another model has to perform worse in vision (or many).
Seems this is the reason why DL/NNs dominate in language/vision tasks.
I like the direction that Claude Code took in visualizing context usage. As these tools are being built, I think the work now needs to focus on control over how to clean up, inspect, and profile. Renderdoc-like maybe?
This weekend’s read isn’t about Artificial Intelligence, but about understanding the brain and our perception of time. Maybe it will spark some ideas to bring into AI.
I just finished AI Engineering by Chip Huyen. Probably the best resource I’ve seen that covers the full AI stack. People wondering how to shift their careers toward AI might find this very useful.
I’ve been learning AI for several years (coming from the games industry), and along the way, I curated a list of the tools, courses, books, papers, and models that actually helped me understand things.
