I don't care enough about interpretability, or this work in general, to spend much time looking at the "interpretable" Qwen neurons to possibly falsify this.
Writing
Read my opinions via the internet.
2026-09-12 / 4.05k words
Forward and reverse ML engineering on unfamiliar hardware by LLMs.
2026-02-23 / 2.52k words
Trying exotic parallel processing hardware which is dead for fairly good reasons.
2025-07-20 / 1.35k words
Also note-taking applications again, embeddings, and blog organization.
2025-05-27 / 1.83k words
The many technologies modern servers have which your desktop doesn't.
2024-02-25 / 3.61k words
How to run local AI slightly more cheaply than with a prebuilt system. Somewhat opinionated.
2025-05-18 / 1.86k words
How it's made (automatically) (in computer games with Minecraft-like recipe systems).
2022-02-24 / 973 words
Learn about how osmarks.net works internally! Spoiler warning if you wanted to reverse-engineer it yourself.
2025-03-27 / 654 words
Generalize land value tax to short names.
2025-03-02 / 4.13k words
The TAM for God is very large.
2025-01-24 / 4.19k words
Downloading and indexing everything* on Reddit on one computer.
2025-01-09 / 1.35k words
Computer algebra systems leave lots to the user and require task-specific manual design. Can we do better?
2024-10-06 / 2.99k words
Or: why most AI hardware startups are lying.
2024-10-06 / 1.08k words
As ever, AI safety becomes AI capabilities.
2020-06-11 / 5.03k words
A nonexhaustive list of media which I like and which you may also be interested in.
2023-08-28 / 2.59k words
Powerful search tools as externalized cognition, and how mine work.
2024-04-22 / 1.54k words
Absurd technical solutions for problems which did not particularly need solving are one of life's greatest joys.
2024-03-27 / 1.86k words
RSAPI and the rest of my infrastructure.
2023-09-24 / 1.64k words
This is, of course, all part of my evil plan to drive site activity through systematically generating (meta)political outrage.
2023-07-02 / 1.61k words
Why programming education isn't very good, and my thoughts on AI code generation.
2023-01-28 / 407 words
A common criticism of school is that it focuses overmuch on rote memorization. While I don't endorse school, I think this argument is wrong.
2022-05-14 / 462 words
RSS/Atom are protocols for Internet-based newsletter/feed services. They're surprisingly well-supported and you should consider using them.
2021-07-08 / 1.07k words
In which I get annoyed at yet more misguided UK government behaviour.
2020-05-20 / 582 words
Is solving Sudoku and similar puzzles by hand really useful in building computer science ability? We don't think so.
2017-08-16 / 940 words
We are not responsible if these tips cause your ship to implode/explode. Contains spoilers in vast quantities.
2018-08-14 / 686 words
Why I think that government programs telling everyone to "code" are pointless.
2018-06-01 / 737 words
My (probably unpopular in general but... actually likely fairly popular amongst this site's intended audience) opinions on smartphones today.
Microblog
Short-form observations.
discord / 2026-10-16 07:42:27
Those don't have the same privileged-basis thing going on (so this is not a good comparison), so I'm going to check some other things.
I don't actually care enough because it would be annoying.
discord / 2026-10-16 07:07:28
https://fixupx.com/allTheYud/status/2110837390780907689
Me: Mechanistic interpretability sounds like a good idea, yes. If we could learn something about how an AI was thinking, in enough detail to predict in advance what it would think even as the thought departed from human, that would be a bare beginning on the sort of understanding you'd needed to get started on AI alignment that wasn't just about hammering on an AI from the outside with SFT and RL. ︀︀AI guys: We found a neuron that lights up when the AI thinks of the Eiffel Tower! ︀︀Me: Cool. That does sound like it'd be a prerequisite to getting started on the important part eventually, someday. ︀︀AI guys: Oh NO! There are MULTIPLE neurons that light up when the AI thinks about the Eiffel Tower! And some of those neurons also participate in OTHER concepts! We're calling this unexpected development 'polysemanticity' and 'superposition'. ︀︀Me: That sure sounds like a thing that would happen, yes. I cannot say that I am impressed by you acting surprised abou…
FixupXhttps://fixupx.com/RatOrthodox/status/2110976835089809678
A Qwen3 neuron that tracks Colorado stuff was given as an example of a neuron that seems polysemantic but makes sense once you trace it. I had Claude check what else it tracks: Tornado’s Python HTTP client code, just as hard. Can’t wait to learn how Colorado and Tornado connect!
Quoting Brangus🔍⏹️ (@RatOrthodox) ︀ Current take: this is cool, genuine scientific progress, and it doesn’t currently give me much hope or change my model of how hard ambitious interpretability is. ︀︀ ︀︀Disclaimer: I didn’t read super closely, it’s very long. There’s a technical paper still to come which might change my mind some, but I predict not much. ︀︀ ︀︀Some of the claimed cool results from the post: ︀︀ ︀︀1) Some canonical examples of weird neurons have robust human-native interpretations. The famous cat + car neuron is a V-shape detector, and the famous car + dogs superposition circuit is generalized pose neurons plus a general feature visu…
FixupXSo apparently one of examples isn't even correct, unless you make some schizophrenic chain of connections, and you don't need to invoke the more complicated reasons this doesn't work. I compared the basis vectors of SigLIP embeddings against the SAE features from a previous project: https://datasets.osmarks.net/meme_features/ https://datasets.osmarks.net/big_sae/features/. I still feel that the SAE features are more obviously "monosemantic" if not perfectly.
discord / 2026-10-15 19:12:12
My thoughts on the claimed interpretability breakthrough: https://chatgpt.com/share/6ad12579-0b70-83eb-a9e4-93457fb20362
discord / 2026-10-13 08:30:00
https://x.com/pastoralcomical/status/1800656726124879904
(inside the torture sphere) wth this sucks. i hate the torture sphere (outside the torture sphere) ok now that i think about it, the torture sphere has a sort of “je ne sais quoi”
XExperiments
Various web projects I have put together over many years. Made with at least four different JS frameworks. Some of them are bad.
A game about... apioforms... by Heavpoot.
Collect Arbitrary Points and achievements by doing things on this website! See how many you have! Do nothing with them because you can't! This is the final form of gamification.
Automatic score keeper, designed for handling Monopoly money.
Colorizes the Alphabet, using highly advanced colorizational algorithms.
The Limitless Grid screensaver (kind of) implemented in a somewhat laggy pixel shader.
An unfinished attempt to replicate an Apple screensaver.
Survive as long as possible against emus and other wildlife. Contributed by Aidan.
Fly an ominous flying square around above some ground! Includes special relativity!
A somewhat unperformant generator for pleasant watercolor-y "fractalart" images. Ported from a Haskell implementation by "TomSmeets".
How bound are you to the habits of the past? Find out with this test.
Obligatory (John Conway's) Game of Life implementation.
It is pitch black (if you ignore all of the lighting). You are likely to be eaten by Heavpoot's terrible writing skills, and/or lacerated/shot/[REDACTED]. Vaguely inspired by the SCP Foundation.
Generates ideas. Terribly. Don't do them. These are not good ideas.
The exciting multiplayer game of incrementing and decrementing! No cheating.
Outdoing all other websites with INFINITE PAGES!
Tells you how late Joe's homework is.
Lorem Ipsum (latin-like placeholder text), eternally. Somehow people have left comments at the bottom anyway.
Instead of wasting time thinking of the best political opinion to hold, simply pick them pseudorandomly per day with this tool.
A Reverse Polish Notation (check wikipedia) calculator, version 2. Buggy and kind of unreliable. This updated version implements advanced features such as subtraction.
Reverse Polish Notation calculator, version 3 - with inbuilt docs, arbitrary-size rational numbers, utterly broken float/rational conversion and quite possibly Turing-completeness.
Reverse Polish Notation calculator, version 4 - increasingly esoteric and incomprehensible. Contributed by Aidan.
Your favourite* tic-tac-toe game in 3 dimensions, transplanted onto the main website! Technically this game is solved and always leads to player 1 winning with optimal play, but the AI is not good enough to do that without more compute!
More dimensions. More confusion. Somewhat worse performance. 4D Tic-Tac-Toe.
A basic implementation of the WFC procedural generation algorithm.
Type websocket URLs in the top bar and hit enter; type messages in the bottom bar, and also hit enter. Probably useful for some weirdly designed websocket services.
Dice-rolling webapp. Not very useful pending me writing a good parser.
Unholy horrors moved from the depths of my projects directory to your browser. Theoretically, this is a calculator. Good luck using it.
Get updates to the blog (not experiments) in your favourite RSS reader using the RSS feed.
View some of my projects at my git hosting.
Other blogs
View list
- Astral Codex Ten
- astrid dot tech
- benjojo blog
- BLDGBLOG
- BOFH
- Chips and Cheese
- citrons
- Construction Physics
- Dominic Cummings
- Drew DeVault
- Entropic Thoughts
- Factorio
- Fernando Borretti
- Graphcore Research
- Igor's Blog
- In the Pipeline
- Irrational Analysis
- Jay Carlson
- KGOnTech
- Libertarian Development
- Marginal Revolution
- Mariven
- Max Barry
- Metal Ivy
- Money Stuff
- Mythic Beasts
- Nanofactory Project
- Nick vs Networking
- Nicole Kornher-Stace
- Overcoming Bias
- phils web site
- Pylon of the Month
- Real World Tech
- rtl-sdr.com
- SemiAnalysis
- ServeTheHome
- Sixty Degrees North
- Stanichor
- Tales from the Void
- The Eldraeverse
- The Electric Grandma
- The Linchpin
- The Worlds of John Bierce
- Things Of Interest
- ToughSF
2026-10-14 / Marginal Revolution
By Max Hastings, an excellent book, I benefited from every page. One recurring theme is simply how hard the Germans fought on the way down, and how unusual that behavior was. Here is one interesting passage of many: In World War I, 150 German...
2026-10-14 / rtl-sdr.com
With AI-in-the-loop reverse engineering, things are heating up, and we're starting to see a gold rush of reverse-engineered Wi-Fi radios turning into software-defined radios. Just yesterday, we posted about h0m3us3r's discovery of how a Gemtek...
2026-10-13 / Money Stuff
Also indexed annuities, Martin Shkreli contempt, non-swap sports bets and retail options.
2026-10-13 / ServeTheHome
Cisco and NVIDIA team up announcing a new USFP form factor for 400G SerDes lanes. Cisco also has a few new Silicon One offerings The post New USFP Pluggable Optics Form Factor for Higher Speeds from Cisco and NVIDIA appeared first on ServeTheHome.
2026-10-13 / The Eldraeverse
Gentles all, for those of you who liked the tunnel carrier I described a while back, I have gone forth and struggled manfully with image-generation models to bring you a fully-realized rendering of Daiségai at sea, somewhere outside the Great Bay...
2026-10-12 / Entropic Thoughts
James Shore posits that each person-month spent writing code results in 10 days of maintenance in the first year, then 5 days every year after that. Naïvely interpreted, this is a threshold-based model for induced maintenance, and its interesting...
2026-10-12 / Overcoming Bias
Though three weeks ago I outlined a story of the long-term history of changes to moral authority, I decided to study this in a bit more detail.
2026-10-11 / Irrational Analysis
Too early and too late. Can it be resurrected from the dead?