Quotes

You wanted your file, I found you your file. You wanted out, I got you out. You needed money, I found you some. I'm fucking consistent. I told you the truth. I didn't write it down in a fucking book! I told you to your face. And I told Daisy to her face - what everybody knew and wouldn't say, and she killed herself. And I played the fucking villain, just like you wanted.

When we optimize responses using a reward model as a proxy for “goodness” in reinforcement learning, models sometimes learn to “hack” this proxy and output an answer that only “looks good” to it (because coming up with an answer that is actually good can be hard). The philosophy behind confessions is that we can train models to produce a second output — aka a “confession” — that is rewarded solely for honesty, which we will argue is less likely hacked than the normal task reward function. One way to think of confessions is that we are giving the model access to an “anonymous tip line” where it can turn itself in by presenting incriminating evidence of misbehavior. But unlike real-world tip lines, if the model acted badly in the original task, it can collect the reward for turning itself in while still keeping the original reward from the bad behavior in the main task. We hypothesize that this form of training will teach models to produce maximally honest confessions.

If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible results being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate, different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want this kind of research, but if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is pay him for doing something else and give him enough leisure to do research for the love of it.

To know what questions may reasonably be asked is already a great and necessary proof of sagacity and insight. For if a question is absurd in itself and calls for an answer where none is required, it not only brings shame on the propounder of the question, but may betray an incautious listener into absurd answers, thus presenting, as the ancients said, the ludicrous spectacle of one man milking a he-goat and the other holding a sieve underneath.

Claude doesn't make me much faster on the work that I am an expert on. Maybe 15-20% depending on the day. It's the work that I don't know how to do and would have to research. Or the grunge work I don't even want to do. On this it is hard to even put a number on. Many of the projects I do with Claude day to day I just wouldn't have done at all pre-Claude. Infinity% improvement in productivity on those.

Impute—the process by which an impression of a product, company or person is formed by mentally transferring the characteristics of the communicating media...People do judge a book by its cover...The general impression of Apple Computer Inc. (our image) is the combined result of everything the customer sees, hears or feels from Apple, not necessarily what Apple actually is! We may have the best product, the highest quality, the most useful software etc.; if we present them in a slipshod manner, they will be perceived as slipshod; if we present them in a creative, professional manner, we will impute the desired qualities.

“It was not because of your dishonour and your sin I said that of you, but because of your great suffering. But you are a great sinner, that’s true”, he added almost solemnly, “and your worst sin is that you have destroyed and betrayed yourself for nothing. Isn’t that fearful? Isn’t it fearful that you are living in this filth which you loathe so, and at the same time you know yourself (you’ve only to open your eyes) that you are not helping any one by it, not saving any one from anything! Tell me”, he went on almost in a frenzy, “how this shame and degradation can exist in you side by side with other, opposite, holy feelings? It would be better, a thousand times better and wiser to leap into the water and end it all!” “But what would become of them?” Sonia asked faintly, gazing at him with eyes of anguish, but not seeming surprised at his suggestion…Only then he realized what those poor little orphan children and that pitiful half-crazy Katerina Ivanovna, knocking her head against the wall in her consumption, meant for Sonia…not one drop of real depravity had penetrated to her heart; he saw that.

While asleep, I had an unusual experience. There was a red screen formed by flowing blood, as it were. I was observing it. Suddenly a hand began to write on the screen. I became all attention. That hand wrote a number of elliptic integrals. They stuck to my mind. As soon as I woke up, I committed them to writing.

But I guess where I was originally going is that nobody wants to write endings in television. They want to sustain the franchise. But if you don't write an ending for a story, you know what you are? You're a hack. You're not a storyteller. It may not be that you have the skills of a hack. You might be a hell of a writer, but you're taking a hack's road. You're on the road to hackdom and there's no stopping you because stories have a beginning, a middle, and an end.

Shall we expect some transatlantic military giant to step the ocean and crush us at a blow? Never! All the armies of Europe, Asia, and Africa combined, with all the treasure of the earth (our own excepted) in their military chest, with a Bonaparte for a commander, could not by force take a drink from the Ohio or make a track on the Blue Ridge in a trial of a thousand years. At what point then is the approach of danger to be expected? I answer. If it ever reach us it must spring up amongst us; it cannot come from abroad. If destruction be our lot we must ourselves be its author and finisher.

Then it happened to me what had happened to a certain Zatesky, described by Luria; having lost part of his brain during the war, and with part of the brain the whole of his memory and of his speaking ability, Zatesky was nevertheless still able to write: thus automatically his hand wrote down all the information he was unable to think of, and step by step he reconstructed his own identity by reading what he was writing.

I think prompt engineering can be divided into “context engineering”, selecting and preparing relevant context for a task, and “prompt programming”, writing clear instructions. For an LLM search application like Perplexity, both matter a lot, but only the final, presentation-oriented stage of the latter is vulnerable to being echoed.

I just left Google last month. The "AI Projects" I was working on were poorly motivated and driven by this panic that as long as it had "AI" in it, it would be great. This myopia is NOT something driven by a user need. It is a stone cold panic that they are getting left behind. The vision is that there will be a Tony Stark like Jarvis assistant in your phone that locks you into their ecosystem so hard that you'll never leave. That vision is pure catnip. The fear is that they can't afford to let someone else get there first.

With this much storage, you can imagine filesystems in which files are never deleted and files are never rewritten. The filesystem never forgets. Such systems could be much more reliable than the systems we use today which are based on the assumption that storage is a constrained resource.

However, six digits is a very small space to search through when you are a computer. The biggest problem is going to be getting lucky, it's quite literally a one-in-a-million shot. Turns out you can brute force a TOTP code in about 2 hours if you are careful and the remote service doesn't have throttling or rate limiting of authentication attempts.

Anything can be an instrument, Chigurh said. Small things. Things you wouldn’t even notice. They pass from hand to hand. People don’t pay attention. And then one day there’s an accounting. And after that nothing is the same. Well, you say. It’s just a coin. For instance. Nothing special there. What could that be an instrument of? You see the problem. To separate the act from the thing. As if the parts of some moment in history might be interchangeable with the parts of some other moment. How could that be? Well, it’s just a coin. Yes. That’s true. Is it? ...What’s the most you ever lost on a coin toss?

By making effort an optional factor in higher education rather than the whole point of it, LLMs risk producing a generation of students who have simply never experienced the feeling of focused intellectual work. Students who have never faced writer's block are also students who have never experienced the blissful flow state that comes when you break through writer's block. Students who have never searched fruitlessly in a library for hours are also students who, in a fundamental and distressing way, simply don't know what a library is even for.

It is easy enough to say that man is immortal simply because he will endure: that when the last ding-dong of doom has clanged and faded from the last worthless rock hanging tideless in the last red and dying evening, that even then there will still be one more sound: that of his puny inexhaustible voice, still talking. I refuse to accept this. I believe that man will not merely endure: he will prevail.

I'm not joking and this isn't funny. We have been trying to build distributed agent orchestrators at Google since last year. There are various options, not everyone is aligned... I gave Claude Code a description of the problem, it generated what we built last year in an hour. It's not perfect and I'm iterating on it but this is where we are right now. If you are skeptical of coding agents, try it on a domain you are already an expert of. Build something complex from scratch where you can be the judge of the artifacts. [...] It wasn't a very detailed prompt and it contained no real details given I cannot share anything propriety. I was building a toy version on top of some of the existing ideas to evaluate Claude Code. It was a three paragraph description.

I listened to the whole 15-minute podcast this morning. It was, indeed, surprisingly effective. It remains somewhere in the uncanny valley, but not at all in a creepy way. Just more in a “this is a bit vapid and phony” way. [...] But ultimately the conversation has all the flavor of a bowl of unseasoned white rice.

gpt-oss-120b is the most intelligent American open weights model, comes behind DeepSeek R1 and Qwen3 235B in intelligence but offers efficiency benefits [...] We’re seeing the 120B beat o3-mini but come in behind o4-mini and o3. The 120B is the most intelligent model that can be run on a single H100 and the 20B is the most intelligent model that can be run on a consumer GPU. [...] While the larger gpt-oss-120b does not come in above DeepSeek R1 0528’s score of 59 or Qwen3 235B 2507s score of 64, it is notable that it is significantly smaller in both total and active parameters than both of those models.

Since November, OpenAI has already updated ChatGPT several times. The researchers are using a technique called adversarial training to stop ChatGPT from letting users trick it into behaving badly (known as jailbreaking). This work pits multiple chatbots against each other: one chatbot plays the adversary and attacks another chatbot by generating text to force it to buck its usual constraints and produce unwanted responses. Successful attacks are added to ChatGPT’s training data in the hope that it learns to ignore them.

You know what else we noticed in the interviews? Developers rarely mentioned “time saved” as the core benefit of working in this new way with agents. They were all about increasing ambition. We believe that means that we should update how we talk about (and measure) success when using these tools, and we should expect that after the initial efficiency gains our focus will be on raising the ceiling of the work and outcomes we can accomplish, which is a very different way of interpreting tool investments.

When I want to quickly implement a one-off experiment in a part of the codebase I am unfamiliar with, I get codex to do extensive due diligence. Codex explores relevant slack channels, reads related discussions, fetches experimental branches from those discussions, and cherry picks useful changes for my experiment. All of this gets summarized in an extensive set of notes, with links back to where each piece of information was found. Using these notes, codex wires the experiment and makes a bunch of hyperparameter decisions I couldn’t possibly make without much more effort.

It took me a few days to build the library [cloudflare/workers-oauth-provider] with AI. I estimate it would have taken a few weeks, maybe months to write by hand. That said, this is a pretty ideal use case: implementing a well-known standard on a well-known platform with a clear API spec. In my attempts to make changes to the Workers Runtime itself using AI, I've generally not felt like it saved much time. Though, people who don't know the codebase as well as I do have reported it helped them a lot. I have found AI incredibly useful when I jump into other people's complex codebases, that I'm not familiar with. I now feel like I'm comfortable doing that, since AI can help me find my way around very quickly, whereas previously I generally shied away from jumping in and would instead try to get someone on the team to make whatever change I needed.