Quotes

We're adding the human touch, but that often requires a deep, developmental edit on a piece of writing. The grammar and word choice just sound weird. You're always cutting out flowery words like 'therefore' and 'nevertheless' that don't fit in casual writing. Plus, you have to fact-check the whole thing because AI just makes things up, which takes forever because it's not just big ideas. AI hallucinates these flippant little things in throwaway lines that you'd never notice. [...] It's tedious, horrible work, and they pay you next to nothing for it.

Nature hath furnished one part of the Earth, and man another. The treasures of time lie high, in Urnes, Coynes, and Monuments, scarce below the roots of some vegetables. Time hath endlesse rarities, and shows of all varieties; which reveals old things in heaven, makes new discoveries in earth, and even earth it self a discovery. That great Antiquity America lay buried for a thousand years; and a large part of the earth is still in the Urne unto us.

For [Natasha] Lyonne, the draw of AI isn’t speed or scale — it’s independence. “I’m not trying to run a tech company,” she told me. “It’s more that I’m a filmmaker who doesn’t want the tech people deciding the future of the medium.” She imagines a future in which indie filmmakers can use AI tools to reclaim authorship from studios and avoid the compromises that come with chasing funding in a broken system. “We need some sort of Dogme 95 for the AI era,” Lyonne said, referring to the stripped-down 1990s filmmaking movement started by Lars von Trier and Thomas Vinterberg, which sought to liberate cinema from an overreliance on technology. “If we could just wrangle this artist-first idea before it becomes industry standard to not do it that way, that’s something I would be interested in working on. Almost like we are not going to go quietly into the night.”

Anthropic declined to comment, but referred Bloomberg News to a five-hour podcast featuring Chief Executive Officer Dario Amodei that was released Monday. "People call them scaling laws. That's a misnomer," he said on the podcast. "They're not laws of the universe. They're empirical regularities. I am going to bet in favor of them continuing, but I'm not certain of that." [...] An Anthropic spokesperson said the language about Opus was removed from the website as part of a marketing decision to only show available and benchmarked models. Asked whether Opus 3.5 would still be coming out this year, the spokesperson pointed to Amodei’s podcast remarks. In the interview, the CEO said Anthropic still plans to release the model but repeatedly declined to commit to a timetable.

[...] The disappointing releases of both GPT-4.5 and Llama 4 have shown that if you don't train a model to reason with reinforcement learning, increasing its size no longer provides benefits. Reinforcement learning is limited only to domains where a reward can be assigned to the generation result. Until recently, these domains were math, logic, and code. Recently, these domains have also included factual question answering, where, to find an answer, the model must learn to execute several searches. This is how these "deep search" models have likely been trained. If your business idea isn't in these domains, now is the time to start building your business-specific dataset. The potential increase in generalist models' skills will no longer be a threat.

I wish it were possible...to invent a method of embalming drowned persons, in such a manner that they might be recalled to life at any period, however distant; for having a very ardent desire to see and observe the state of America a hundred years hence, I should prefer to an ordinary death, being immersed with a few friends in a cask of Madeira, until that time, then to be recalled to life by the solar warmth of my dear country! But...in all probability, we live in a century too little advanced, and too near the infancy of science, to see such an art brought in our time to its perfection...

We want AI to “just work” for you; we realize how complicated our model and product offerings have gotten. We hate the model picker as much as you do and want to return to magic unified intelligence. We will next ship GPT-4.5, the model we called Orion internally, as our last non-chain-of-thought model. After that, a top goal for us is to unify o-series models and GPT-series models by creating systems that can use all our tools, know when to think for a long time or not, and generally be useful for a very wide range of tasks. In both ChatGPT and our API, we will release GPT-5 as a system that integrates a lot of our technology, including o3. We will no longer ship o3 as a standalone model. When [asked about release dates for GPT 4.5 / GPT 5:] weeks / months

If you give feedback that isn't constructive your feedback is worthless. I know that sounds harsh but it is. If you give unconstructive feedback you might as well not be saying anything. If you just look at something and go "That's stupid" or "I don't like that" - that's worthless feedback, nobody can do anything with that. They're not going to start throwing darts against the wall until you say "Oh OK, I like that". You have to say something more.

A test of how seriously your firm is taking AI: when o-1 (& the new Gemini) came out this week, were there assigned folks who immediately ran the model through internal, validated, firm-specific benchmarks to see how useful it as? Did you update any plans or goals as a result? Or do you not have people (including non-technical people) assigned to test the new models? No internal benchmarks? No perspective on how AI will impact your business that you keep up-to-date? No one is going to be doing this for organizations, you need to do it yourself.

I know that what I see through the microscope is veridical because we made the grid to be just that way. I know that the process of manufacture is reliable, because we can check the results with the microscope. Moreover we can check the results with any kind of microscope, using any of a dozen unrelated physical processes to produce an image. Can we entertain the possibility that, all the same, this is some gigantic coincidence?

VII. ...The spray rainbows over the sloping lawns. With short jerks a robin runs up, stands motionless. The eucalyptus tree trunks glow in the light. The oaks perfect the shadow of May leaves. Only this. Only this is worth of praise: the day. ...VIII. And what if Pascal had not been saved and if those narrow hands in which we laid a cross are just he, entire, like a lifeless swallow in the dust, under the buzz of the poisonous-blue flies? And if they all, kneeling with poised palms, millions, billions of them, ended together with their illusion? I shall never agree. I will give them the crown. The human mind is splendid; lips, powerful, and the summons, so great, it must open Paradise. IX. They are so persistent, that give them a few stones and edible roots, and they will build the world.

Just because Java was once aimed at a set-top box OS that didn't support multiple address spaces, and just because process creation in Windows used to be slow as a dog, doesn't mean that multiple processes (with judicious use of IPC) aren't a much better approach to writing apps for multi-CPU boxes than threads.

Into thirty centuries born, At home in them all but the last, We meet ourselves at every turn, In the long country of the past. ...Ilium burns before our eyes For thirty centuries never put out And we walk the streets of Troy And breathe in the air its fabulous name The king, the courtier, and the rout Shall never perish in that flame; Old Priam shall become a boy For ever changed, for ever the same. ...This is the place of hope and fear, And faith that comes when hope is lost; Defeat and victory both are here. In this place where all’s to be.

I must hold it for the greatest calamity of our time, which lets nothing come to maturity, that one moment is consumed by the next, and the day spent in the day; so that a man is always living from hand to mouth, without having anything to show for it. Have we not already newspapers for every hour of the day! A good head could assuredly intercalate one or other of them. They publish abroad everything that every one does, or is busy with or meditating; nay, his very designs are thereby dragged into publicity. No one can rejoice or be sorry, but as a pastime for others; and so it goes on from house to house, from city to city, from kingdom to kingdom, and at last from one hemisphere to the other,—all in post haste.

It is probably more illuminating to go a little bit further back, to the Middle Ages. One of its characteristics was that "reasoning by analogy" was rampant; another characteristic was almost total intellectual stagnation, and we now see why the two go together. A reason for mentioning this is to point out that, by developing a keen ear for unwarranted analogies, one can detect a lot of medieval thinking today.

It is in the public good to have AI produce quality and credible (if ‘hallucinations’ can be overcome) output. It is in the public good that there be the creation of original quality, credible, and artistic content. It is not in the public good if quality, credible content is excluded from AI training and output OR if quality, credible content is not created.

In every group I speak to, from business executives to scientists, including a group of very accomplished people in Silicon Valley last night, much less than 20% of the crowd has even tried a GPT-4 class model. Less than 5% has spent the required 10 hours to know how they tick.

Deprivation seems a strange sort of gift. I find food in a couple hours of fishing each day, and I seek shelter in a rubber tent. How unnecessarily complicated my past life seems. For the first time, I clearly see a vast difference between human needs and human wants. Before this voyage, I always had what I needed—food, shelter, clothing, and companionship—yet I was often dissatisfied when I didn’t get everything I wanted, when people didn’t meet my expectations, when a goal was thwarted, or when I couldn’t acquire some material goody. My plight has given me a strange kind of wealth, the most important kind. I value each moment that is not spent in pain, desperation, hunger, thirst, or loneliness.

There was no point at all to the Book, which is what made it an especially dreaded form of penance. It contained twelve chapters. Like the scale used to measure earthquakes, these got exponentially worse as they went on, so Chapter Six was ten times as bad as Chapter Five, and so on. Chapter One was just a taste, meted out to delinquent children, and usually completed in an hour or two. Two meant at least one overnight stay, though any self-respecting troublemaker could bang it out in a day. Five typically meant a stay of several weeks. Any sentence of Chapter Six or higher could be appealed to the Primate and then to the Inquisition. Chapter Twelve amounted to a sentence of life at hard labor in solitary confinement; only three avout had finished it in 3690 years, and all of them were profoundly insane. Beyond about Six, the punishment could span years. Many chose to leave the concent rather than endure it. Those who stuck it out were changed when they emerged: subdued, and notably diminished. Which might sound crazy, because there was nothing to it other than copying out the required chapters, memorizing them, and then answering questions about them before a panel of hierarchs. But the contents of the Book had been crafted and refined over many centuries to be nonsensical, maddening, and pointless: flagrantly at first, more subtly as the chapters progressed. It was a maze without an exit, an equation that after weeks of toil reduced to 2 = 3. Chapter One was a page of nursery-rhymes salted with nonsense-words that almost rhymed—but not quite. Chapter Four was five pages of the digits of pi. Beyond that, however, there was no further randomness in the Book, since it was easy to memorize truly random things once you taught yourself a few tricks—and everyone who’d made it through Chapter Four knew the tricks. Much harder to memorize and to answer questions about were writings that almost but did not quite make sense; that had internal logic, but only to a point. Such things cropped up naturally in the mathic world from time to time—after all, not everyone had what it took to be a Saunt. After their authors had been humiliated and Thrown Back, these writings would be gone over by the Inquisition, and, if they were found to be the right kind of awful, made even more so, and folded into later and more wicked editions of the Book. To complete your sentence and be granted permission to walk out of your cell, you had to master them just as thoroughly as, say, a student of quantum mechanics must know group theory. The punishment lay in knowing that you were putting all of that effort into letting a kind of intellectual poison infiltrate your brain to its very roots. It was more humiliating than you might imagine, and after I’d been toiling on Chapter Five for a couple of weeks I had no difficulty in seeing how one who completed a sentence of, say, Chapter Nine would emerge permanently damaged.

Two things can be true simultaneously: (a) LLM provider cost economics are too negative to return positive ROI to investors, and (b) LLMs are useful for solving problems that are meaningful and high impact, albeit not to the AGI hype that would justify point (a). This particular combination creates a frustrating gray area that requires a nuance that an ideologically split social media can no longer support gracefully. [...] OpenAI collapsing would not cause the end of LLMs, because LLMs are useful today and there will always be a nonzero market demand for them: it’s a bell that can’t be unrung.

...well over half of the time you spend working on a project (on the order of 70 percent) is spent thinking, and no tool, no matter how advanced, can think for you. Consequently, even if a tool did everything except the thinking for you—if it wrote 100 percent of the code, wrote 100 percent of the documentation, did 100 percent of the testing, burned the CD-ROMs, put them in boxes, and mailed them to your customers—the best you could hope for would be a 30 percent improvement in productivity. In order to do better than that, you have to change the way you think.

Even then, what does “best” even mean? I think back then I used it a lot more just because I was writing for a food blog every day, and “best” gives you more clicks than “really good.” These days, I don’t really care about clicks, and so I very rarely say something is “best.” I generally go out of my way to say, “This is just what I felt like doing today.”

Recently Google Translate announced the ability to hear translations into English spoken via text-to-speech (TTS). Looking at the Firebug Net panel for where this TTS data was coming from, I saw that the speech audio is in MP3 format and is queried via a simple HTTP GET (REST) request: http://translate.google.com/translate_tts?q=text

The realization hit me [when the GPT-3 paper came out] that an important property of the field flipped. In ~2011, progress in AI felt constrained primarily by algorithms. We needed better ideas, better modeling, better approaches to make further progress. If you offered me a 10X bigger computer, I'm not sure what I would have even used it for. GPT-3 paper showed that there was this thing that would just become better on a large variety of practical tasks, if you only trained a bigger one. Better algorithms become a bonus, not a necessity for progress in AGI. Possibly not forever and going forward, but at least locally and for the time being, in a very practical sense. Today, if you gave me a 10X bigger computer I would know exactly what to do with it, and then I'd ask for more.

(This is the school in which we learn ...) What is the self amid this blaze? What am I now that I was then Which I shall suffer and act again, The theodicy I wrote in my high school days Restored all life from infancy, The children shouting are bright as they run (This is the school in which they learn ...) Ravished entirely in their passing play! (...that time is the fire in which they burn.)