Index

This is ridiculous. Font descriptions mean nothing. We need bitter-lesson font classification.

Theory: people (partly) dislike deep learning because it feels like cheating, like Ozempic - it is "too easy" for what it gets you.

As Robin Hanson says, building the sheer variety of products we have is actually bad, because it increases unit costs. This is especially clear in laptops - there are far too many laptops with too little to distinguish them and too many nonsense minor issues. As such, I think we need a new streamlined and harmonized lineup of all laptops:

• Cheapest Possible Technically Functional Laptop
• Mediocre Office and Home Laptop (to be issued to most office workers and people who want to edit spreadsheets or emails and such)
• CEO Laptop (reasonably fast, expensive, big battery for CEO activities)
• Programmer Laptop (ThinkPad-like focused on CPU performance and reasonable portability)
• Gamer Laptop (16" Legion-like with middling battery life and decently high-powered CPU/GPU)
• Gamer Laptop (Big) (17"-18" desktop replacement)
• Technician Laptop (smallish thick and rugged laptop with many ports)
• Multimedia Laptop (Mediocre Office and Home Laptop with a nicer display and better graphics)

There would also be a version number updated whenever new components are available, of course. There can perhaps be two or three variants of each (with the same chassis, board, etc but different components) with different pricing, but no more.

Anyone optimistic about society adapting sanely to AGI should look at the uptake of IPv6.

Why do all three of the reasonably okay AI music tools (Udio, Suno, Riffusion) have fairly similar artifacts? Except for, I think, older versions of Udio, they all sound consistently off in some way I don't know enough music theory to explain, particularly in metal vocals and/or complex instrumentals. Do they all use the same autoencoders or something?

Street-Fighting Mathematics is not actually related to street fighting, but you should read it if you like estimating things. There is much power in being approximately right very fast, and it contains many clever tricks which are not immediately obvious but are very powerful. My favourite part so far is this exercise - you can uniquely (up to a dimensionless constant) identify this formula just from some ideas about what it should contain and a small linear algebra problem!

People are claiming (I don't know much RL) that DeepSeek-R1's training process is very simple (based on the paper: https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf) - a boring standardish (for LLMs) RL algorithm optimizing for reward on some ground-truth-verifiable tasks (they don't say which). So why did o1 not happen until late 2024 (public release) or late 2023 (rumours of Q*)? "Do RL on useful tasks" is a very obvious idea. I think the relevant algorithms are older than that.

The paper says that they tried applying it to smaller models and it didn't work nearly as well, so "base models were bad then" is a plausible explanation, but it's clearly not true - GPT-4-base is probably a generally better (if costlier) model than 4o, which o1 is based on (could be distillation from a secret bigger one though); and LLaMA-3.1-405B used a somewhat similar postttraining process and is about as good a base model, but is not competitive with o1 or R1. So I don't think it's that.

What's going on here? The process is simple-sounding but filled with pitfalls DeepSeek don't mention? What has changed between 2022/23 and now which means we have at least three decent long-CoT reasoning models around?

Religion has progressed, historically, from:

• there is a very large quantity of widely dispersed gods and you don't know about the vast majority of them
• there are quite a few gods, but a bounded amount
• there is exactly one god
• there are exactly zero gods

By extrapolation, we can conclude that the next step is that humanity has negative one god, i.e. is in theological debt and must build a god to continue. This is where the EY-style "aligned singleton" came from. But people are now moving toward "we need everyone to have pocket gods" because they are insane, in line with the pattern. The next step is of course "we need to build gods and put them in everything".