Quotes

It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something. They don't care if the tokens happen to represent little text chunks. It could just as well be little image patches, audio chunks, action choices, molecules, or whatever. If you can reduce your problem to that of modeling token streams (for any arbitrary vocabulary of some set of discrete tokens), you can "throw an LLM at it".

The one crowded space in Father Perry's house was his bookshelves. I gradually came to understand that the marks on the pages were trapped words. Anyone could learn to decipher the symbols and turn the trapped words loose again into speech. The ink of the print trapped the thoughts; they could no more get away than a doomboo could get out of a pit. When the full realization of what this meant flooded over me, I experienced the same thrill and amazement as when I had my first glimpse of the bright lights of Konakry. I shivered with the intensity of my desire to learn to do this wondrous thing myself.

Draconian failure on error is not the answer problems of Postel's law. Draconian error handling creates an unstable equilibrium in Game Theory terms - it only lasts until one player breaks the rule. One non-Draconian XML5 implementation in key client product and the Draconian XML ranks would break. Well-specified error recovery is the right way to implement the liberal part of Postel's law.

National politics of snoopiness vs corporate ethic of not being evil aren’t directly compatible, and the solution here only works because (let’s face it) Tunisia is not a rising economic force. If you’re selling ads in China, you don’t get to pretend that the Great Firewall of China is a security issue.

From my perspective, it is crucial for Linux to have good support for Silverlight because I do not want Linux on the desktop to become a second class citizen ever again. [...] The core of the debate is whether Microsoft will succeed in establishing Silverlight as a RIA platform or not. You believe that without Moonlight they would not have a chance of success, and I believe that they would have regardless of us.

DAOs are, I think, one of the best illustrations of the problem with a lot of these Web3 projects: They are trying to find technological solutions that will somehow codify very complex social structures. A lot of them also seem to operate under the assumption that everyone is acting in good faith, and that project members’ interests will generally align—a baffling assumption given the amount of bad actors in the crypto space.

Llama 4 is making great progress in training. Llama 4 mini is done with pre-training and our reasoning models and larger model are looking good too. Our goal with Llama 3 was to make open source competitive with closed models, and our goal for Llama 4 is to lead. Llama 4 will be natively multimodal -- it's an omni-model -- and it will have agentic capabilities, so it's going to be novel and it's going to unlock a lot of new use cases.

V. WHAT THE THUNDER SAID After the torchlight red on sweaty faces After the frosty silence in the gardens After the agony in stony places The shouting and the crying Prison and palace and reverberation Of thunder of spring over distant mountains He who was living is now dead We who were living are now dying With a little patience

In 2021 we [the Mozilla engineering team] found “samesite=lax by default” isn’t shippable without what you call the “two minute twist” - you risk breaking a lot of websites. If you have that kind of two-minute exception, a lot of exploits that were supposed to be prevented remain possible. When we tried rolling it out, we had to deal with a lot of broken websites: Debugging cookie behavior in website backends is nontrivial from a browser. Firefox also had a prototype of what I believe is a better protection (including additional privacy benefits) already underway (called total cookie protection). Given all of this, we paused samesite lax by default development in favor of this.

If I am not mistaken, the heterogeneous pieces I have enumerated resemble Kafka; if I am not mistaken, not all of them resemble each other. This second fact is the more significant. In each of these texts we find Kafka’s idiosyncrasy to a greater or lesser degree, but if Kafka had never written a line, we would not perceive this quality; in other words, it would not exist. The poem, "Fears and Scruples" by Browning foretells Kafka’s work, but our reading of Kafka perceptibly sharpens and deflects our reading of the poem. Browning did not read it as we do now. In the critics’ vocabulary, the word 'precursor' is indispensable, but it should be cleansed of all connotation of polemics or rivalry. The fact is the every writer creates his own precursors. His work modifies our conception of the past, as it will modify the future. In this correlation the identity or plurality of the men involved is unimportant.

But [LLM assisted programming] does make me wonder whether the adoption of these tools will lead to a form of de-skilling. Not even that programmers will be less skilled, but that the job will drift from the perception and dynamics of a skilled trade to an unskilled trade, with the attendant change - decrease - in pay. Instead of hiring a team of engineers who try to write something of quality and try to load the mental model of what they're building into their heads, companies will just hire a lot of prompt engineers and, who knows, generate 5 versions of the application and A/B test them all across their users.

HTML has always been a conversation between browser makers, authors, standards wonks, and other people who just showed up and liked to talk about angle brackets. Most of the successful versions of HTML have been “retro-specs,” catching up to the world while simultaneously trying to nudge it in the right direction. Anyone who tells you that HTML should be kept “pure” (presumably by ignoring browser makers, or ignoring authors, or both) is simply misinformed. HTML has never been pure, and all attempts to purify it have been spectacular failures, matched only by the attempts to replace it.

In the past, these decisions were so consequential, they were basically one-way doors, in Amazon language. That’s why we call them ‘architectural decisions!’ You basically have to live with your choice of database, authentication, JavaScript UI framework, almost forever. But that’s changing with LLMs, because you can explore, investigate, and even prototype each one so quickly. Even technology migrations are becoming so much easier/cheaper/faster. These are all examples of increasing optionality.

E-Trade financial tried using a RSA fob as a second factor of authentication, but as of their 11/07/06 financial report their fraud losses continue to increase. That said, they considered this program a success because users indicated they feel safer and are more likely to provide assets.

...in the morning When the light drips through the shutters like the dew, I arise, I face the sunrise, And do the things my fathers learned to do. Stars in the purple dusk above the rooftops Pale in a saffron mist and seem to die, And I myself on a swiftly tilting planet Stand before a glass and tie my tie. ...It is morning. I stand by the mirror And tie my tie once more. While waves far off in a pale rose twilight Crash on a white sand shore. I stand by a mirror and comb my hair: How small and white my face!— The green earth tilts through a sphere of air And bathes in a flame of space. There are houses hanging above the stars And stars hung under a sea. . . And a sun far off in a shell of silence Dapples my walls for me. . .

In my opinion it is better to compare OpenIDs to credit cards. [...] Just as a credit card company may place limit on the level of guarantee, web sites are at liberty to restrict the OpenIDs it will recognize and accept. Just as many of us carry more than one credit card, we may have multiple OpenIDs and use them for different occasions. Just as some department store credit card is not accepted outside of that store, it is possible that IDs issued by some OpenID providers may not be accepted by some sites.

I think what you’ll find— I think what you’ll find is, Whatever it is we do substantively, There will be near-perfect clarity As to what it is. And it will be known, And it will be known to the Congress, And it will be known to you, Probably before we decide it, But it will be known.

Much like Gen X is sometimes the forgotten generation (or at least we feel that way), the generation of us who grew up with an internet that seemed an unalloyed good fall awkwardly into the middle between those who didn’t grow up with it, and those for whom there has always been the whiff of brimstone, greed, and ruin around the place.

Imagine a Simon Says style game where I present an article found on the web on a projector. Students research for two to three minutes, then respond by standing or staying seated to signal if they believe the article is true or fake. My students absolutely loved the game. Some refused to go to recess until I gave them another chance to figure out the next article I had queued.

Carl Hewitt recently remarked that the question what is an agent? is embarrassing for the agent-based computing community in just the same way that the question what is intelligence? is embarrassing for the mainstream AI community. The problem is that although the term is widely used, by many people working in closely related areas, it defies attempts to produce a single universally accepted definition. This need not necessarily be a problem: after all, if many people are successfully developing interesting and useful applications, then it hardly matters that they do not agree on potentially trivial terminological details. However, there is also the danger that unless the issue is discussed, 'agent' might become a 'noise' term, subject to both abuse and misuse, to the potential confusion of the research community.

Buzzwords describe what you already intuitively know. At once they snap the ‘kaleidoscopic flux of impressions’ in your mind into form, crystallizing them instantly allowing you to both organize your knowledge and recognize you share it with other. This rapid, mental crystallization is what I call the buzzword whiplash. It gives buzzwords more importance and velocity, more power, than they objectively should have. The potential energy stored within your mind is released by the buzzword whiplash. The buzzword is perceived as important partially because of what it describes but also because of the social and emotional weight felt when the buzzword recognizes your previously wordless experiences and demonstrates that those experiences are shared.

I am currently documenting a language called Seenku, spoken by fewer than 15,000 people in the rolling hills of southwestern Burkina Faso in West Africa. Like Chinese, it is a tonal language, meaning the pitch on which a word is pronounced can radically alter its meaning. For instance, tsu can mean “thatch” when pronounced with an extra low pitch, but “hippopotamus” when pronounced with falling pitch. In fact, pitch plays such a huge role in Seenku that it can be “spoken” through music alone, most notably on the traditional xylophone.

For some time, I’ve argued that a common conception of AI is misguided. This is the idea that AI systems like large language and vision models are individual intelligent agents, analogous to human agents. Instead, I’ve argued that these models are “cultural technologies” like writing, print, pictures, libraries, internet search engines, and Wikipedia. Cultural technologies allow humans to access the information that other humans have created in an effective and wide-ranging way, and they play an important role in increasing human capacities.

for those open source companies that still harbor magical beliefs, let me put this to you as directly as possible: cloud services providers are emphatically not going to license your proprietary software. I mean, you knew that, right? The whole premise with your proprietary license is that you are finding that there is no way to compete with the operational dominance of the cloud services providers; did you really believe that those same dominant cloud services providers can’t simply reimplement your LDAP integration or whatever? The cloud services providers are currently reproprietarizing all of computing — they are making their own CPUs for crying out loud! — reimplementing the bits of your software that they need in the name of the service that their customers want (and will pay for!) won’t even move the needle in terms of their effort.

It’s not gasps and blood and falling about—that isn’t what makes it death. It’s just a man failing to reappear, that’s all—now you see him, now you don’t, that’s the only thing that’s real: here one minute and gone the next and never coming back—an exit, unobtrusive and unannounced.