Quotes

I think individual creators or publishers tend to overestimate the value of their specific content in the grand scheme of [AI training]. […] We pay for content when it’s valuable to people. We’re just not going to pay for content when it’s not valuable to people. I think that you’ll probably see a similar dynamic with AI, which my guess is that there are going to be certain partnerships that get made when content is really important and valuable. I’d guess that there are probably a lot of people who have a concern about the feel of it, like you’re saying. But then, when push comes to shove, if they demanded that we don’t use their content, then we just wouldn’t use their content. It’s not like that’s going to change the outcome of this stuff that much.

To Isengard! Though Isengard be ringed and barred with doors of stone; Though Isengard be strong and hard, as cold as stone and bare as bone, We go, we go, we go to war, to hew the stone and break the door; For bole and bough are burning now, the furnace roars—we go to war! To land of gloom with tramp of doom, with roll of drum, we come, we come; To Isengard with doom we come! With doom we come, with doom we come!

Migrations are both essential and frustratingly frequent as your codebase ages and your business grows: most tools and processes only support about one order of magnitude of growth before becoming ineffective, so rapid growth makes them a way of life. [...] As a result you switch tools a lot, and your ability to migrate to new software can easily become the defining constraint for your overall velocity. [...] Migrations matter because they are usually the only available avenue to make meaningful progress on technical debt.

Are we so deranged here in the twenty-first century that we’re going to re-enact, wide-eyed, the twin tragedies of the great desktop-suite lock-in and the great proprietary-SQL lock-in? You know, the ones where you give a platform vendor control over your IT budget? Gimme a break.

Hollywood is filled with feel-good messages about how robotic logic is no match for fuzzy, warm, human irrationality, and how the power of love will overcome pesky obstacles such as a malevolent superintelligent computer. Unfortunately there isn’t a great deal of cause to think this is the case, any more than there is that noble gorillas can defeat evil human poachers with the power of chest-beating and the ability to use rudimentary tools.

Async functions require an event loop to run. Flask, as a WSGI application, uses one worker to handle one request/response cycle. When a request comes in to an async view, Flask will start an event loop in a thread, run the view function there, then return the result. Each request still ties up one worker, even for async views. The upside is that you can run async code within a view, for example to make multiple concurrent database queries, HTTP requests to an external API, etc. However, the number of requests your application can handle at one time will remain the same.

The challenge [with RAG] is that most corner-cutting solutions look like they’re working on small datasets while letting you pretend that things like search relevance don’t matter, while in reality relevance significantly impacts quality of responses when you move beyond prototyping (whether they’re literally search relevance or are better tuned SQL queries to retrieve more appropriate rows). This creates a false expectation of how the prototype will translate into a production capability, with all the predictable consequences: underestimating timelines, poor production behavior/performance, etc.

But that was then, a long time gone. Now we have something different: we have "anything goes" without the spirit. "Transgression", "subversion", "deconstruction" are praise words bestowed as solemnly as "structure" and "order" once were, little gold stars awarded to rappers and television comics. Cakes on rakes are everywhere. A million cats cavort frantically for our attention. Even the fish has been co-opted (though what choice did he have?). "Enjoy!" cries the fish. "Consume! Everything will be fine when your mother gets home." But the fish is whistling into the wind. The mother has left, and she’s never coming back. It’s just us and that goddamn cat.

Two things can be true simultaneously: (a) LLM provider cost economics are too negative to return positive ROI to investors, and (b) LLMs are useful for solving problems that are meaningful and high impact, albeit not to the AGI hype that would justify point (a). This particular combination creates a frustrating gray area that requires a nuance that an ideologically split social media can no longer support gracefully. [...] OpenAI collapsing would not cause the end of LLMs, because LLMs are useful today and there will always be a nonzero market demand for them: it’s a bell that can’t be unrung.

In my opinion it is better to compare OpenIDs to credit cards. [...] Just as a credit card company may place limit on the level of guarantee, web sites are at liberty to restrict the OpenIDs it will recognize and accept. Just as many of us carry more than one credit card, we may have multiple OpenIDs and use them for different occasions. Just as some department store credit card is not accepted outside of that store, it is possible that IDs issued by some OpenID providers may not be accepted by some sites.

You count the "value" that is lost by people who would have made money selling rival goods, but can't now because they can't compete with free. But you don't count the value that is created by people who build upon the freely given goods. [...] In other words, you only look at the first-order effects. It's the same mistake a lot of people make when they accuse open source developers of "dumping" and ruining the market for competing software. That's true, in a very narrow sense, but it ignores all the other people who took that software and used it to create something else of value.

Every year without knowing it I have passed the day When the last fires will wave to me And the silence will set out Tireless traveler Like the beam of a lightless star Then I will no longer Find myself in life as in a strange garment Surprised at the earth And the love of one woman And the shamelessness of men As today writing after three days of rain Hearing the wren sing and the falling cease And bowing not knowing to what

I’m no developer, but I got the AI part working in about an hour. What took longer was the other stuff: identifying the problem, designing and building the UI, setting up the templating, routes and data architecture. It reminded me that, in order to capitalise on the potential of AI technologies, we need to really invest in the other stuff too, especially data infrastructure. It would be ironic, and a huge shame, if AI hype sucked all the investment out of those things.

And the investors wailed and gnashed their teeth but it’s true, that is what they agreed to, and they had no legal recourse. And OpenAI’s new CEO, and its nonprofit board, cut them a check for their capped return and said “bye” and went back to running OpenAI for the benefit of humanity. It turned out that a benign, carefully governed artificial superintelligence is really good for humanity, and OpenAI quickly solved all of humanity’s problems and ushered in an age of peace and abundance in which nobody wanted for anything or needed any Microsoft products. And capitalism came to an end.

If you give feedback that isn't constructive your feedback is worthless. I know that sounds harsh but it is. If you give unconstructive feedback you might as well not be saying anything. If you just look at something and go "That's stupid" or "I don't like that" - that's worthless feedback, nobody can do anything with that. They're not going to start throwing darts against the wall until you say "Oh OK, I like that". You have to say something more.

Some authors affirm that the following also belongs to him: that Plato saw him washing lettuces, came up to him and quietly said to him, ‘Had you paid court to Dionysius, you wouldn’t now be washing lettuces’, and that he with equal calmness made answer, ‘If you had washed lettuces, you wouldn’t have paid court to Dionysius.’

Apple’s terminology distinguishes between “personal intelligence,” on-device and under their control, and “world knowledge,” which is prone to hallucinations – but is also what consumers expect when they use AI, and it’s what may replace Google search as the “point of first intent” one day soon. It’s wise for them to keep world knowledge separate, behind a very clear gate, but still engage with it. Protects the brand and hedges their bets.

Media carries with it a credibility that is totally undeserved. You have all experienced this, in what I call the Murray Gell-Mann Amnesia effect. (I call it by this name because I once discussed it with Murray Gell-Mann, and by dropping a famous name I imply greater importance to myself, and to the effect, than it would otherwise have.) Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backwards—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper’s full of them. In any case, you read with exasperation or amusement the multiple errors in a story—and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know.

Here I like to quote Valéry, who said a person is a poet if his imagination is stimulated by the difficulties inherent in his art and not if his imagination is dulled by them. I think very few people can manage free verse—you need an infallible ear, like D. H. Lawrence, to determine where the lines should end.

Everyone applauds when Google goes after Microsoft's Office monopoly [...] but when they start to go after web non-profits like Wikipedia, you see where the ineluctible logic leads. As Google's growth slows, as inevitably it will, it will need to consume more and more of the web ecosystem, trading against its former suppliers, rather than distributing attention to them.

The greatest coup Microsoft pulled with Internet Explorer was putting the word "Internet" in its name. It sits there, on the desktop of every new Windows computer, and it says "Internet". So you click it. [...] What better way to beat a browser with the word "Internet" in its name - a browser that seemingly can't be beat no matter how hard we try - than the Internet Company itself making a browser?

If you’re ever debugging a problem and you see the number 42-mumble-mumble-mumble-7295 you’ve run out of 32-bit storage. If you see 2-mumble-mumble-mumble-647 (2147483647) you’ve run out of signed 32-bit storage. 167-mumble-mumble-15 (16777215) you’ve run out of 24-bits and 65-mumble-mumble-35 (65535) you’ve run out of 16-bits of integers.

Then it happened to me what had happened to a certain Zatesky, described by Luria; having lost part of his brain during the war, and with part of the brain the whole of his memory and of his speaking ability, Zatesky was nevertheless still able to write: thus automatically his hand wrote down all the information he was unable to think of, and step by step he reconstructed his own identity by reading what he was writing.

Beneath the blossoms with a pot of wine, No friends at hand, so I poured alone; I raised my cup to invite the moon, Turned to my shadow, and we became three. Now the moon had never learned about my drinking, And my shadow had merely followed my form, But I quickly made friends with the moon and my shadow; To find pleasure in life, make the most of the spring. Whenever I sang, the moon swayed with me; Whenever I danced, my shadow went wild. Drinking, we shared our enjoyment together; Drunk, then each went off on his own. But forever agreed on dispassionate revels, We promised to meet in the far Milky Way.

It may be hard to imagine writing rock solid one-in-a-million-or-better tests that drive Internet Explorer to click ajax frontend buttons executing backend apache, php, memcache, mysql, java and solr. I am writing this blog post to tell you that not only is it possible, it’s just one part of my day job.

At this point the lawsuits seem a bit far-fetched: “You should have warned us months ago that artificial intelligence would hurt your business” is unfair given how quickly ChatGPT has exploded from nowhere to become a cultural and business phenomenon. But now everyone is on notice! If you are not warning your shareholders now about how AI could hurt your business, and then it does hurt your business, you’re gonna get sued.

The Consumer Financial Protection Bureau (CFPB) supervises, sets rules for, and enforces numerous federal consumer financial laws and guards consumers in the financial marketplace from unfair, deceptive, or abusive acts or practices and from discrimination [...] the fact that the technology used to make a credit decision is too complex, opaque, or new is not a defense for violating these laws.

It took me a few days to build the library [cloudflare/workers-oauth-provider] with AI. I estimate it would have taken a few weeks, maybe months to write by hand. That said, this is a pretty ideal use case: implementing a well-known standard on a well-known platform with a clear API spec. In my attempts to make changes to the Workers Runtime itself using AI, I've generally not felt like it saved much time. Though, people who don't know the codebase as well as I do have reported it helped them a lot. I have found AI incredibly useful when I jump into other people's complex codebases, that I'm not familiar with. I now feel like I'm comfortable doing that, since AI can help me find my way around very quickly, whereas previously I generally shied away from jumping in and would instead try to get someone on the team to make whatever change I needed.

I have seen the extremely restrictive off-boarding agreement that contains nondisclosure and non-disparagement provisions former OpenAI employees are subject to. It forbids them, for the rest of their lives, from criticizing their former employer. Even acknowledging that the NDA exists is a violation of it. If a departing employee declines to sign the document, or if they violate it, they can lose all vested equity they earned during their time at the company, which is likely worth millions of dollars.

As soon as we started programming, we found to our surprise that it wasn’t as easy to get programs right as we had thought. Debugging had to be discovered. I can remember the exact instant when I realized that a large part of my life from then on was going to be spent in finding mistakes in my own programs.

How fast do they go? Like everything else, camels aren't what they were. Do not be encouraged by the accounts of the great desert travellers. They were better men than us, and were probably lying anyway, and they were riding camels which were used to going for many days at full pelt over the most hellish land and then charging into artillery fire at the end. The wrecks you get in the modern camel markets of Omdurman and Cairo are degenerate great-great-great-great-grandchildren of them and their forebears would be desperately ashamed of them.

It’s not gasps and blood and falling about—that isn’t what makes it death. It’s just a man failing to reappear, that’s all—now you see him, now you don’t, that’s the only thing that’s real: here one minute and gone the next and never coming back—an exit, unobtrusive and unannounced.

But this much is clear: Facebook knew all along. Their own employees were desperately trying to get anyone inside the company to listen as their products radicalized their own friends and family members. And as they were breaking the world, they had an army of spokespeople publicly and privately gaslighting and intimidating reporters and researchers who were trying to ring the alarm bell. They knew all along and they simply did not give a shit.

When I was curating my generated tweets, I estimated 30-40% of the tweets were usable comedically, a massive improvement over the 5-10% usability from my GPT-2 tweet generation. However, a 30-40% success rate implies a 60-70% failure rate, which is patently unsuitable for a production application.

Among closed-source models, OpenAI's early mover advantage has eroded somewhat, with enterprise market share dropping from 50% to 34%. The primary beneficiary has been Anthropic,* which doubled its enterprise presence from 12% to 24% as some enterprises switched from GPT-4 to Claude 3.5 Sonnet when the new model became state-of-the-art. When moving to a new LLM, organizations most commonly cite security and safety considerations (46%), price (44%), performance (42%), and expanded capabilities (41%) as motivations.

The whole story is basically that Facebook gets so much traffic that they started convincing publishers to post things on Facebook. For a long time, that was fine. People posted things on Facebook, then you would click those links and go to their websites. But then, gradually, Facebook started exerting more and more control of what was being seen, to the point that they, not our website, essentially became the main publishers of everyone’s content. Today, there’s no reason to go to a comedy website that has a video if that video is just right on Facebook. And that would be fine if Facebook compensated those companies for the ad revenue that was generated from those videos, but because Facebook does not pay publishers, there quickly became no money in making high-quality content for the internet.

A third maxim may be added to the preceding pair: Seize the very first possible opportunity to act on every resolution you make, and on every emotional prompting you may experience in the direction of the habits you aspire to gain. It is not in the moment of their forming, but in the moment of their producing motor effects, that resolves and aspirations communicate the new 'set' to the brain.

I am therefore writing this essay on literature to tell of the glorious accomplishments of past men of letters, and to comment on the causes of failure and success in writing. Perhaps some day the secret of this most intricate art may be entirely mastered. In making an axe handle by cutting wood with an axe, the model is indeed near at hand. But the adaptability of the hand to the ever-changing circumstances and impulses in the process of creation is such as words can hardly explain. What follows is only what can be said in words.

Most propositions & questions that have been written about philosophical matters are not false but senseless. We cannot, therefore, answer questions of this kind at all, but only state their senselessness. Most questions & propositions of the philosophers result from the fact that we do not understand the logic of our language. They are of the same kind as the question whether the Good is more, or less, identical than the Beautiful.

The blog post announcing the shutdown was done one day early. The idea was to take the opportunity of the new Pope being announced and Andy Rubin being replaced as head of Android, so that the [Google] Reader news may be drowned out. PR didn't apparently realize that the kinds of people that care about the other two events (especially the Pope) are not the same kind of people that care about Reader, so it didn't work.

Running training jobs across multiple nodes scales really well. A common assumption is that scale inevitably means slowdowns: more GPUs means more synchronization overhead, especially with multiple nodes communicating across a network. But we observed that the performance penalty isn’t as harsh as what you might think. Instead, we found near-linear strong scaling: fixing the global batch size and training on more GPUs led to proportional increases in training throughput. On a 1.3B parameter model, 4 nodes means a 3.9x gain over one node. On 16 nodes, it’s 14.4x. This is largely thanks to the super fast interconnects that major cloud providers have built in: @awscloud EC2 P4d instances provide 400 Gbps networking bandwidth, @Azure provides 1600 Gbps, and @OraclePaaS provides 800 Gbps.

No, imbeciles! No! Fools and cretins, a book will not make a plate of soup; a novel is not a pair of boots; a sonnet is not a syringe; a drama is not a railway—those forms of civilization which have caused humanity to march on the road to progress. By all the bowels of all the popes, past, present and future, no! Ten thousand times no! You cannot make a hat out of a metonymy, and you cannot make a simile in the form of a bedroom slipper, and you cannot use an antithesis as an umbrella...An ode is, I have a feeling, too light a garment for the winter.

If you visit (often NSFW, beware!) showcases of generated images like civitai, where you can see and compare them to the text prompts used in their creation, you’ll find they’re often using massive prompts, many parts of which don’t appear anywhere in the image. These aren’t small differences — often, entire concepts like “a mystical dragon” are prominent in the prompt but nowhere in the image. These users are playing a gacha game, a picture-making slot machine. They’re writing a prompt with lots of interesting ideas and then pulling the arm of the slot machine until they win… something. A compelling image, but not really the image they were asking for.

But at the same time the advent of modern technology has brought with it a debasement of the color currency. Today almost every object that rolls off the factory production line, from motor cars to pencils, is given a distinctive color—and for the most part these colors are meaningless. As I look around the room I’m working in, man-made color shouts back at me from every surface: books, cushions, a rug on the floor, a coffee-cup, a box of staples—bright blues, reds, yellows greens. There is as much color here as in any tropical forest. Yet while almost every color in the forest would be meaningful, here in my study almost nothing is. Color anarchy has taken over. The indiscriminate use of color has no doubt dulled modern humans’ biological response to it. From the first moment a baby is given a string of multi-colored—but otherwise identical—beads to play with, she is unwittingly being taught to ignore color as a signal.

There was an exchange on Twitter a while back where someone said, ‘What is artificial intelligence?’ And someone else said, ‘A poor choice of words in 1954’. And, you know, they’re right. I think that if we had chosen a different phrase for it, back in the ’50s, we might have avoided a lot of the confusion that we’re having now.

We have recently trained our first 100M token context model: LTM-2-mini. 100M tokens equals ~10 million lines of code or ~750 novels. For each decoded token, LTM-2-mini's sequence-dimension algorithm is roughly 1000x cheaper than the attention mechanism in Llama 3.1 405B for a 100M token context window. The contrast in memory requirements is even larger -- running Llama 3.1 405B with a 100M token context requires 638 H100s per user just to store a single 100M token KV cache. In contrast, LTM requires a small fraction of a single H100's HBM per user for the same context.

The Japanese seem to the rest of us to live several measurable clicks down the time line. The Japanese are the ultimate Early Adopters, and the sort of fiction I write behooves me to pay serious heed to that. If you believe, as I do, that all cultural change is essentially technologically driven, you pay attention to the Japanese. They've been doing it for more than a century now, and they really do have a head start on the rest of us, if only in terms of what we used to call 'future shock' (but which is now simply the one constant in all our lives).

You don’t need to be the world’s leading expert to write about a particular topic. Experts are often busy and struggle to explain concepts in an accessible way. You should be honest with yourself and with your readers about what you know and don’t know — but otherwise, it’s OK to write about what excites you, and to do it as you learn.

Touji: “Oh, yes. the view of the world that one can have is quite small.” Hikari: “Yes, you measure things only by your own small measure.” Asuka: “One sees things with the truth, given by others.” Misato: “Happy on a sunny day.” Rei: “Gloomy on a rainy day.” Asuka: “If you’re taught that, you always think so.” Ritsuko: “But, you can enjoy rainy days.”

Upon his death man must leave everything behind...and depart forever from the world he has known. He must of necessity go to that foul land of death, a fact which makes death the most sorrowful of all events...Some foreign doctrines, however, teach that death should not be regarded as profoundly sorrowful...These are all gross deceptions contrary to human sentiment and fundamental truths. Not to be happy over happy events, not to be saddened by sorrowful events, not to show surprise at astonishing events, in a word, to consider it proper not to be moved by whatever happens, are all foreign types of deception and falsehood. They are contrary to human nature and extremely repugnant to me.

Again, in the early days, I was limited in computing capacity and it was clear, in my area, that a "mathematician had no use for machines." But I needed more machine capacity. Every time I had to tell some scientist in some other area, "No I can’t; I haven’t the machine capacity", he complained. I said "Go tell your Vice President that Hamming needs more computing capacity." After a while I could see what was happening up there at the top; many people said to my Vice President, "Your man needs more computing capacity." I got it! I also did a second thing. When I loaned what little programming power we had to help in the early days of computing, I said, "We are not getting the recognition for our programmers that they deserve. When you publish a paper you will thank that programmer or you aren’t getting any more help from me. That programmer is going to be thanked by name; she’s worked hard." I waited a couple of years. I then went through a year of BSTJ articles and counted what fraction thanked some programmer. I took it into the boss and said, "That’s the central role computing is playing in Bell Labs; if the BSTJ is important, that’s how important computing is." He had to give in. You can educate your bosses.

Of all the books I have delivered to the presses, none, I think, is as personal as the straggling collection mustered for this hodgepodge, precisely because it abounds in reflections and interpolations. Few things have happened to me, and I have read a great many. Or rather, few things have happened to me more worth remembering than Schopenhauer's thought or the music of England's words. A man sets himself the task of portraying the world. Through the years he peoples a space with images of provinces, kingdoms, mountains, bays, ships, islands, fishes, rooms, instruments, stars, horses, and people. Shortly before his death, he discovers that that patient labyrinth of lines traces the image of his face.

In the big picture, Twitter did exactly the right thing. They had a good idea and they buckled down and focused on delivering something as cool as possible as fast as possible, and it's really hard, in early 2007, to beat Rails for that. When all of a sudden there were a few tens of thousands of people using it, then they went to work on the scaling.

Simply put, free and open-source software is just the scientific model applied to programming: free sharing of work open collaboration; open publication; peer review; recognition of the best work, with priority given to the first to do a meaningful new piece of work; and so forth. As a programmer, it is the best arena in which to work. There are no secrets; the work must stand on its own.

You pass briefly through our midst. Your books and writing have not been consulted. Our prayers are pro defuncto N. Yet some look up, as though among a crowd of prisoners or displaced persons, they recognized a friend once known in a far country. For these the sun also rose after a forgotten war upon an idiom you made great. They have not forgotten you. In their silence you are still famous, no ritual shade. How slowly this bell tolls in a monastery tower for a whole age, and for the quick death of an unready dynasty, and for that brave illusion: the adventurous self? For with one shot the whole hunt is ended!

One of the best examples of LLM developer tooling I've heard is from a team that supports software from the 80s-90s. Their only source of documentation is video interviews with retired employees. So they feed them into transcription software and get summarized searchable notes out the other end.

"What did the Men of old use them [the palantír] for?" asked Pippin, delighted and astonished at getting answers to so many questions..."To see far off, and to converse in thought with one another", said Gandalf..."In the days of his wisdom Denethor would not presume to use it to challenge Sauron, knowing the limits of his own strength. But his wisdom failed...He was too great to be subdued to the will of the Dark Power, he saw nonetheless only those things which that Power permitted him to see. The knowledge which he obtained was, doubtless, often of service to him; yet the vision of the great might of Mordor that was shown to him fed the despair of his heart until it overthrew his mind."

It is quite easy to show, decision-theoretically, that the greatest chance of survival generally belongs to those who have most of their beliefs about things which affect our survival ability true. Philosophy, however, is totally irrelevant to survival from an evolutionary point of view. Natural selection has no way of weeding out veridical intuitions about the basic constitution of matter, for instance, from false ones, because humans have not generally been killed before they can procreate due to having erroneous metaphysical intuitions. Or bluntly put: having a true metaphysical theory does not help you getting laid.

One day when I was a junior medical student, a very important Boston surgeon visited the school and delivered a great treatise on a large number of patients who had undergone successful operations for vascular reconstruction. At the end of the lecture, a young student at the back of the room timidly asked, "Do you have any controls?" Well, the great surgeon drew himself up to his full height, hit the desk, and said, "Do you mean did I not operate on half the patients?" The hall grew very quiet then. The voice at the back of the room very hesitantly replied, "Yes, that's what I had in mind." Then the visitor's fist really came down as he thundered, "Of course not. That would have doomed half of them to their death." God, it was quiet then, and one could scarcely hear the small voice ask, "Which half?"

One way to avoid unspotted prediction errors is for the technology in its current state to have early and frequent contact with reality as it is iteratively developed, tested, deployed, and all the while improved. And there are creative ideas people don’t often discuss which can improve the safety landscape in surprising ways — for example, it’s easy to create a continuum of incrementally-better AIs (such as by deploying subsequent checkpoints of a given training run), which presents a safety opportunity very unlike our historical approach of infrequent major model upgrades.

A successful author is equally in danger of the diminution of his fame, whether he continues or ceases to write. The regard of the publick is not to be kept but by tribute, and the remembrance of past service will quickly languish, unless successive performances frequently revive it. Yet in every new attempt there is new hazard, and there are few who do not at some unlucky time, injure their own characters by attempting to enlarge them.

The way I would talk about myself as a senior engineer is that I’d say “I know how I would solve the problem” and because I know how I would solve it I could also teach someone else to do it. And my theory is that the next level is that I can say about myself “I know how others would solve the problem”. Let’s make that a bit more concrete. You make that sentence: “I can anticipate how the API choices that I’m making, or the abstractions that I’m introducing into a project, how they impact how other people would solve a problem.”

On the one hand, information wants to be expensive because it's so valuable. The right information in the right place just changes your life. On the other hand, information wants to be free, because the cost of getting it out is lower and lower all the time. So you have these 2 things fighting against each other.

Who does not know that a horse falling from a height of three or four cubits will break his bones, while a dog falling from the same height or a cat from a height of eight or ten cubits will suffer no injury? Equally harmless would be the fall of a grasshopper from a tower or the fall of an ant from the distance of the moon. Do not children fall with impunity from heights which would cost their elders a broken leg or perhaps a fractured skull? And just as smaller animals are proportionately stronger and more robust than the larger, so also smaller plants are able to stand up better than larger. I am certain you both know that an oak two hundred cubits high, would not be able to sustain its own branches if they were distributed as in a tree of ordinary size; and that nature cannot produce a horse as large as twenty ordinary horses or a giant ten times taller than an ordinary man unless by miracle or by greatly altering the proportions of his limbs and especially of his bones, which would have to be considerably enlarged over the ordinary. Likewise the current belief that, in the case of artificial machines the very large and the very small are equally feasible and lasting is a manifest error. Thus, for example, a small obelisk or column or other solid figure can certainly be laid down or set up without danger of breaking, while the large ones will go to pieces under the slightest provocation, and that purely on account of their own weight.

Sometimes the most remarkable things seem commonplace. I mean, when you think about it, jet travel is pretty freaking remarkable. You get in a plane, it defies the gravity of an entire planet by exploiting a loophole with air pressure, and it flies across distances that would take months or years to cross by any means of travel that has been significant for more than a century or three. You hurtle above the earth at enough speed to kill you instantly should you bump into something, and you can only breathe because someone built you a really good tin can that has seams tight enough to hold in a decent amount of air. Hundreds of millions of man-hours of work and struggle and research, blood, sweat, tears, and lives have gone into the history of air travel, and it has totally revolutionized the face of our planet and societies. But get on any flight in the country, and I absolutely promise you that you will find someone who, in the face of all that incredible achievement, will be willing to complain about the drinks. The drinks, people.

The living come with grassy tread To read the gravestones on the hill; The graveyard draws the living still, ...So sure of death the marbles rhyme, Yet can't help marking all the time How no one dead will seem to come. What is it men are shrinking from? It would be easy to be clever And tell the stones: 'Men hate to die And have stopped dying now forever.' I think they would believe the lie.

The key to understanding the pace of today’s infrastructure buildout is to recognize that while AI optimism is certainly a driver of AI CapEx, it is not the only one. The cloud players exist in a ruthless oligopoly with intense competition. [...] Every time Microsoft escalates, Amazon is motivated to escalate to keep up. And vice versa. We are now in a cycle of competitive escalation between three of the biggest companies in the history of the world, collectively worth more than $7T. At each cycle of the escalation, there is an easy justification—we have plenty of money to afford this. With more commitment comes more confidence, and this loop becomes self-reinforcing. Supply constraints turbocharge this dynamic: If you don’t acquire land, power and labor now, someone else will.

The problem is I'm older now, I'm 40 years old, and this stuff doesn't change the world. It really doesn't. I'm sorry, it's true. Having children really changes your view on these things. We're born, we live for a brief instant, and we die. It's been happening for a long time. Technology is not changing it much—if at all. These technologies can make life easier, can let us touch people we might not otherwise. You may have a child with a birth defect and be able to get in touch with other parents and support groups, get medical information, the latest experimental drugs. These things can profoundly influence life. I'm not downplaying that. But it's a disservice to constantly put things in this radical new light—that it's going to change everything. Things don't have to change the world to be important.

When anyone tells me that he saw a dead man restored to life, I immediately consider with myself whether it be more probable, that this person should either deceive or be deceived, or that the fact, which he relates, should have happened. I weigh the one miracle against the other; & according to the superiority, which I discover, I pronounce my decision, & always reject the greater miracle. If the falsehood of his testimony would be more miraculous than the event which he relates; then, & not till then, can he pretend to command my belief or opinion.

Reading Anton Chekhov's stories, one feels oneself in a melancholy day of late autumn, when the air is transparent and the outline of naked trees, narrow houses, greyish people, is sharp. Everything is strange, lonely, motionless, helpless. The horizon, blue and empty, melts into the pale sky, and its breath is terribly cold upon the earth, which is covered with frozen mud. The author's mind, like autumn sun, shows up in hard outline the monotonous roads, the crooked streets, the little squalid houses in which tiny, miserable people are stifled by boredom and laziness and fill the houses with an unintelligible, drowsy bustle...There passes before one a long file of men and women, slaves of their love, of their stupidity and idleness, of their greed for the good things of life; there walk the slaves of the dark fear of life; they straggle anxiously along, filling life with incoherent words about the future, feeling that in the present there is no place for them...In front of that dreary, gray crowd of helpless people there passed a great, wise, and observant man: he looked at all these dreary inhabitants of his country, and, with a sad smile, with a tone of gentle but deep reproach, with anguish in his face and in his heart, in a beautiful and sincere voice, he said to them: "You live badly, my friends. It is shameful to live like that."

When you know something it is almost impossible to imagine what it is like not to know that thing. This is the curse of knowledge, and it is the root of countless misunderstandings and inefficiencies. Smart people who are comfortable with complexity can be especially prone to it! If you don’t guard against the curse of knowledge it has the potential to obfuscate all forms of communication, including code. The more specialized your work, the greater the risk that you will communicate in ways that are incomprehensible to the uninitiated.

We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite being small enough to be deployed on a phone.

God is dead. God remains dead. And we have killed him. How shall we comfort ourselves, the murderers of all murderers? What was holiest and mightiest of all that the world has yet owned has bled to death under our knives: who will wipe this blood off us? What water is there for us to clean ourselves? What festivals of atonement, what sacred games shall we have to invent? Is not the greatness of this deed too great for us? Must we ourselves not become gods simply to appear worthy of it? ...This prodigious event is still on its way, still wandering; it has not yet reached the ears of men. Lightning and thunder require time, the light of the stars requires time, deeds, though done, still require time to be seen and heard. This deed is still more distant from them than the most distant stars—and yet they have done it themselves.

The realization hit me [when the GPT-3 paper came out] that an important property of the field flipped. In ~2011, progress in AI felt constrained primarily by algorithms. We needed better ideas, better modeling, better approaches to make further progress. If you offered me a 10X bigger computer, I'm not sure what I would have even used it for. GPT-3 paper showed that there was this thing that would just become better on a large variety of practical tasks, if you only trained a bigger one. Better algorithms become a bonus, not a necessity for progress in AGI. Possibly not forever and going forward, but at least locally and for the time being, in a very practical sense. Today, if you gave me a 10X bigger computer I would know exactly what to do with it, and then I'd ask for more.

Against the dark screen of night, Vimes had a vision of Ankh-Morpork. It wasn’t a city, it was a process, a weight on the world that distorted the land for hundreds of miles around. People who’d never see it in their whole life nevertheless spent that life working for it. Thousands and thousands of green acres were part of it, forests were part of it. It drew in and consumed... ...and gave back the dung from its pens, and the soot from its chimneys, and steel, and saucepans, and all the tools by which its food was made. And also clothes, and fashions, and ideas, and interesting vices, songs, and knowledge, and something which, if looked at in the right light, was called civilization. That was what civilization meant. It meant the city.

It hangs on its / stem like a plum at the edge of a / darkening thicket.It’s swelling and / blushing and ripe and I reach out a / hand to pick itbut flesh moves / slow through time and evening / comes on fastand just when I / think my fingers might seize that / sweetness at lastthe gentlest of / breezes rises and the plum lets / go of  the stem.And now it’s my / fingers ripening and evening that’s / reaching for them.

What comes before determines what comes after...The thoughts of all men arise from the darkness. If you are the movement of your soul, and the cause of that movement precedes you, then how could you ever call your thoughts your own? How could you be anything other than a slave to the darkness that comes before?...History. Language. Passion. Custom. All these things determine what men say, think, and do. These are the hidden puppet-strings from which all men hang...all men are deceived...So long as what comes before remains shrouded, so long as men are already deceived, what does [deceiving men] matter?

It's hard to overstate the value of LLM support when coding for fun in an unfamiliar language. [...] This example is totally trivial in hindsight, but might have taken me a couple mins to figure out otherwise. This is a bigger deal than it seems! Papercuts add up fast and prevent flow. (A lot of being a senior engineer is just being proficient enough to avoid papercuts).

Philosophy in any case always comes on the scene too late...When philosophy paints its gloomy picture then a form of life has grown old. It cannot be rejuvenated by the gloomy picture, but only understood. Only when the dusk starts to fall does the owl of Minerva spread its wings and fly.

To see what is in front of one’s nose needs a constant struggle. One thing that helps toward it is to keep a diary, or, at any rate, to keep some kind of record of one’s opinions about important events. Otherwise, when some particularly absurd belief is exploded by events, one may simply forget that one ever held it. Political predictions are usually wrong. But even when one makes a correct one, to discover why one was right can be very illuminating. In general, one is only right when either wish or fear coincides with reality. If one recognizes this, one cannot, of course, get rid of one’s subjective feelings, but one can to some extent insulate them from one’s thinking and make predictions cold-bloodedly, by the book of arithmetic. In private life most people are fairly realistic. When one is making out one’s weekly budget, two and two invariably make four. Politics, on the other hand, is a sort of sub-atomic or non-Euclidean word where it is quite easy for the part to be greater than the whole or for two objects to be in the same place simultaneously. Hence the contradictions and absurdities I have chronicled above, all finally traceable to a secret belief that one’s political opinions, unlike the weekly budget, will not have to be tested against solid reality.

I really dislike the practice of replacing passwords with email “magic links”. Autofilling a password from my keychain happens instantly; getting a magic link from email can take minutes sometimes, and even in the fastest case, it’s nowhere near instantaneous. Replacing something very fast — password autofill — with something slower is just a terrible idea.

Adhering to a plan Moon spelled out more than three decades ago in a series of sermons, members of his movement managed to integrate virtually every facet of the highly competitive seafood industry. The Moon followers' seafood operation is driven by a commercial powerhouse, known as True World Group. It builds fleets of boats, runs dozens of distribution centers and, each day, supplies most of the nation's estimated 9,000 sushi restaurants.

I have been one acquainted with the night. I have walked out in rain—and back in rain. I have outwalked the furthest city light. I have looked down the saddest city lane. I have passed by the watchman on his beat And dropped my eyes, unwilling to explain. I have stood still and stopped the sound of feet When far away an interrupted cry Came over houses from another street, But not to call me back or say good-bye; And further still at an unearthly height, One luminary clock against the sky Proclaimed the time was neither wrong nor right I have been one acquainted with the night.

I toss these pages in the faces of timid, furtive, respectable people and say: 'There! that's me! You may like it or lump it, but it's true. And I challenge you to follow suit, to flash the searchlight of your self-consciousness into every remotest corner of your life and invite everybody's inspection. Be candid, be honest, break down the partitions of your cubicle, come out of your burrow, little worm.' As we are all such worms we should at least be honest worms.

The primary problem is that while the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce. There are also many people trying out ChatGPT to create answers, without the expertise or willingness to verify that the answer is correct prior to posting. Because such answers are so easy to produce, a large number of people are posting a lot of answers. The volume of these answers (thousands) and the fact that the answers often require a detailed read by someone with at least some subject matter expertise in order to determine that the answer is actually bad has effectively swamped our volunteer-based quality curation infrastructure.

The creator of a model can not ensure that a model is never used to do something harmful – any more so that the developer of a web browser, calculator, or word processor could. Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams. [...] Instead of regulating the development of AI models, the focus should be on regulating their applications, particularly those that pose high risks to public safety and security. Regulate the use of AI in high-risk areas such as healthcare, criminal justice, and critical infrastructure, where the potential for harm is greatest, would ensure accountability for harmful use, whilst allowing for the continued advancement of AI technology.

When you start a creative project but don’t finish, the experience drags you down. Worst of all is when you never decisively abandon a project, instead allowing it to fade into forgetfulness. The fades add up; they become a gloomy haze that whispers, you’re not the kind of person who DOES things. When you start and finish, by contrast — and it can be a project of any scope: a 24-hour comic, a one-page short story, truly anything — it is powerful fuel that goes straight back into the tank. When a project is finished, it exits the realm of “this is gonna be great” and becomes instead something you (and perhaps others) can actually evaluate. Even if that evaluation is disastrous, it is also, I will insist, thrilling and productive. A project finished is the pump of a piston, preparing the engine for the next one.

[On agents using CLI tools in place of REST APIs] To save on context window, yes, but moreso to improve accuracy and success rate when multiple tool calls are involved, particularly when calls must be correctly chained e.g. for pagination, rate-limit backoff, and recognizing authentication failures. Other major factor: which models can wield the skill? Using the CLI lowers the bar so cheap, fast models (gpt-5-nano, haiku-4.5) can reliably succeed. Using the raw APl is something only the costly "strong" models (gpt-5.2, opus-4.5) can manage, and it squeezes a ton of thinking/reasoning out of them, which means multiple turns/iterations, which means accumulating a ton of context, which means burning loads of expensive tokens. For one-off API requests and ad hoc usage driven by a developer, this is reasonable and even helpful, but for an autonomous agent doing repetitive work, it's a disaster.

A lot of people who claim to be doing prompt engineering today are actually just blind prompting. "Blind Prompting" is a term I am using to describe the method of creating prompts with a crude trial-and-error approach paired with minimal or no testing and a very surface level knowedge of prompting. Blind prompting is not prompt engineering. [...] In this blog post, I will make the argument that prompt engineering is a real skill that can be developed based on real experimental methodologies.

The expansion of the jagged frontier of AI capability is subtle and requires a lot of experience with various models to understand what they can, and can’t, do. That is why I suggest that people and organizations keep an “impossibility list” - things that their experiments have shown that AI can definitely not do today but which it can almost do. For example, no AI can create a satisfying puzzle or mystery for you to solve, but they are getting closer. When AI models are updated, test them on your impossibility list to see if they can now do these impossible tasks.

We already know one major effect of AI on the skills distribution: AI acts as a skills leveler for a huge range of professional work. If you were in the bottom half of the skill distribution for writing, idea generation, analyses, or any of a number of other professional tasks, you will likely find that, with the help of AI, you have become quite good.

llama.cpp surprised many people (myself included) with how quickly you can run large LLMs on small computers [...] TLDR at batch_size=1 (i.e. just generating a single stream of prediction on your computer), the inference is super duper memory-bound. The on-chip compute units are twiddling their thumbs while sucking model weights through a straw from DRAM. [...] A100: 1935 GB/s memory bandwidth, 1248 TOPS. MacBook M2: 100 GB/s, 7 TFLOPS. The compute is ~200X but the memory bandwidth only ~20X. So the little M2 chip that could will only be about ~20X slower than a mighty A100.

28. Do not repeat the tactics which have gained you one victory, but let your methods be regulated by the infinite variety of circumstances. 29. Military tactics are like unto water; for water in its natural course runs away from high places and hastens downwards. 30. So in war, the way is to avoid what is strong and to strike at what is weak. 31. Water shapes its course according to the nature of the ground over which it flows; the soldier works out his victory in relation to the foe whom he is facing. 32. Therefore, just as water retains no constant shape, so in warfare there are no constant conditions. 34. The five elements (water, fire, wood, metal, earth) are not always equally predominant; the four seasons make way for each other in turn. There are short days and long; the moon has its periods of waning and waxing.

Whether you’re an AI-programming skeptic or an enthusiast, the reality is that many programming tasks are beyond the reach of today’s models. But many decent dev tools are actually quite easy for AI to build, and can help the rest of the programming go smoother. In general, these days any time I’m spending more than a minute staring at a JSON blob, I consider whether it’s worth building a custom UI for it.

`If Claude is asked to count words, letters, and characters, it thinks step by step before answering the person. It explicitly counts the words, letters, or characters by assigning a number to each. It only answers the person once it has performed this explicit counting step. [...]` `If Claude is shown a classic puzzle, before proceeding, it quotes every constraint or premise from the person’s message word for word before inside quotation marks to confirm it’s not dealing with a new variant. [...]` `If asked to write poetry, Claude avoids using hackneyed imagery or metaphors or predictable rhyming schemes.`

...the discovery of computers and the thinking about computers has turned out to be extremely useful in many branches of human reasoning. For instance, we never really understood how lousy our understanding of languages was, the theory of grammar and all that stuff, until we tried to make a computer which would be able to understand language. We tried to learn a great deal about psychology by trying to understand how computers work. There are interesting philosophical questions about reasoning, and relationship, observation, and measurement and so on, which computers have stimulated us to think about anew, with new types of thinking. And all I was doing was hoping that the computer-type of thinking would give us some new ideas, if any are really needed.

\[2 points\] Learn basic NumPy operations with an AI tutor! Use an AI chatbot \(e.g., ChatGPT, Claude, Gemini, or Stanford AI Playground\) to teach yourself how to do basic vector and matrix operations in NumPy \(import numpy as np\). AI tutors have become exceptionally good at creating interactive tutorials, and this year in CS221, we're testing how they can help you learn fundamentals more interactively than traditional static exercises.

Despite being rusty with coding (I don't code every day these days): since starting to use Windsurf / Cursor with the recent increasingly capable models: I am SO back to being as fast in coding as when I was coding every day "in the zone" [...] When you are driving with a firm grip on the steering wheel - because you know exactly where you are going, and when to steer hard or gently - it is just SUCH a big boost. I have a bunch of side projects and APIs that I operate - but usually don't like to touch it because it's (my) legacy code. Not any more. I'm making large changes, quickly. These tools really feel like a massive multiplier for experienced devs - those of us who have it in our head exactly what we want to do and now the LLM tooling can move nearly as fast as my thoughts!

There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. It's possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like "decrease the padding on the sidebar by half" because I'm too lazy to find it. I "Accept All" always, I don't read the diffs anymore. When I get error messages I just copy paste them in with no comment, usually that fixes it. The code grows beyond my usual comprehension, I'd have to really read through it for a while. Sometimes the LLMs can't fix a bug so I just work around it or ask for random changes until it goes away. It's not too bad for throwaway weekend projects, but still quite amusing. I'm building a project or webapp, but it's not really coding - I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works.

LLMs may offer immense value to society. But that does not warrant the violation of copyright law or its underpinning principles. We do not believe it is fair for tech firms to use rightsholder data for commercial purposes without permission or compensation, and to gain vast financial rewards in the process. There is compelling evidence that the UK benefits economically, politically and societally from upholding a globally respected copyright regime.

The soul has no assignments, neither cooks Nor referees: it wastes its time. It wastes its time. Here in this enclave there are centuries For you to waste: the short and narrow stream Of life meanders into a thousand valleys Of all that was, or might have been, or is to be. The books, just leafed through, whisper endlessly.

Right now Facebook's position on 3rd party developers is amazing and I'm sure they are genuine in their support. However, give Facebook two missed quarters as a public company and they might not have no choice but to squeeze every ounce of revenue out of Facebook. That squeeze might include competing with the current crop of Facebook developers.

In the long term, I want to replace JavaScript and the DOM with a smarter, safer design. In the medium term, I want to use something like Google Gears to give us vats with which we can have safe mashups. But in the short term, I recommend that you be using Firefox with No Script. Until we get things right, it seems to be the best we can do.

Graphic designers had a similar sea change ~20-25 years ago. Flyers, restaurant menus, wedding invitations, price lists... That sort of thing was bread and butter work for most designers. Then desktop publishing happened and a large fraction of designers lost their main source of income as the work shifted to computer assisted unskilled labor. The field still thrives today, but that simple work is gone forever.

...Let me add a certain virile reply recorded by De Quincey (Writings XI, 226). Someone flung a glass of wine in the face of a gentleman during a theological or literary debate. The victim did not show any emotion and said to the offender: 'This, sir, is a digression; now, if you please, for the argument.' (The author of that reply, a certain Dr. Henderson, died in Oxford around 1787, without leaving us any memory other than those just words: a sufficient and beautiful immortality.) A popular tale, which I picked up in Geneva during the last years of World War I, tells of Migel Servert’s reply to the inquisitors who had condemned him to the sake: 'I will burn, but this is a mere event. We shall continue our discussion in eternity.'

GitHub’s slopocalypse – the flood of AI-generated spam PRs and issues – has made Jazzband’s model of open membership and shared push access untenable. Jazzband was designed for a world where the worst case was someone accidentally merging the wrong PR. In a world where only 1 in 10 AI-generated PRs meets project standards, where curl had to shut down its bug bounty because confirmation rates dropped below 5%, and where GitHub’s own response was a kill switch to disable pull requests entirely – an organization that gives push access to everyone who joins simply can’t operate safely anymore.

A transition from an author’s book to his conversation, is too often like an entrance into a large city, after a distant prospect. Remotely, we see nothing but spires of temples and turrets of palaces, and imagine it the residence of splendour, grandeur and magnificence; but when we have passed the gates, we find it perplexed with narrow passages, disgraced with despicable cottages, embarrassed with obstructions, and clouded with smoke.

It feels pretty likely that prompting or chatting with AI agents is going to be a major way that we interact with computers into the future, and whereas there’s not a huge spread in the ability between people who are not super good at tapping on icons on their smartphones and people who are, when it comes to working with AI it seems like we’ll have a high dynamic range. Prompting opens the door for non-technical virtuosos in a way that we haven’t seen with modern computers, outside of maybe Excel.

Now I want to talk about how they're selling AI. The growth narrative of AI is that AI will disrupt labor markets. I use "disrupt" here in its most disreputable, tech bro sense. The promise of AI – the promise AI companies make to investors – is that there will be AIs that can do your job, and when your boss fires you and replaces you with AI, he will keep half of your salary for himself, and give the other half to the AI company. That's it. That's the $13T growth story that MorganStanley is telling. It's why big investors and institutionals are giving AI companies hundreds of billions of dollars. And because they are piling in, normies are also getting sucked in, risking their retirement savings and their family's financial security.

Large teams spend more time dealing with coordination and are more likely to reach for architecture and abstractions that they hope will reduce coordination costs, aka if I architect this well enough I don’t have to speak to my colleagues. Microservices, event buses, and schema free databases are all examples of attempts to architect our way around coordination. A decade in we’ve learned that these patterns raise the cost of reasoning about a system, during onboarding, during design, and during incidents and outages.

You may find that there are plenty of job listings where the job requirements are described as, “must be expert with Photoshop and Illustrator…” or something long those lines. Ignore those job listings; they’re placed by inept and sick companies looking for decorators, not designers.

[...] here’s what we found when we integrated [Amazon Q, GenAI assistant for software development] into our internal systems and applied it to our needed Java upgrades: - The average time to upgrade an application to Java 17 plummeted from what’s typically 50 developer-days to just a few hours. We estimate this has saved us the equivalent of 4,500 developer-years of work (yes, that number is crazy but, real). - In under six months, we've been able to upgrade more than 50% of our production Java systems to modernized Java versions at a fraction of the usual time and effort. And, our developers shipped 79% of the auto-generated code reviews without any additional changes.

The only thing that would have been nice is that after the project had been finished and the chip deployed, that someone from Intel would have told me, just as a courtesy, that MINIX 3 was now probably the most widely used operating system in the world on x86 computers. That certainly wasn't required in any way, but I think it would have been polite to give me a heads up, that's all.

Society is composed of persons who cannot design, build, repair, or even operate most of the devices upon which their lives depend... In the complexity of this world people are confronted with extraordinary events and functions that are literally unintelligible to them. They are unable to give an adequate explanation of man-made phenomena in their immediate experience. They are unable to form a coherent, rational picture of the whole. Under the circumstances, all persons do, and indeed must, accept a great number of things on faith... Their way of understanding is basically religious, rather than scientific; only a small portion of one's everyday experience in the technological society can be made scientific... The plight of members of the technological society can be compared to that of a newborn child. Much of the data that enters its sense does not form coherent wholes. There are many things the child cannot understand or, after it has learned to speak, cannot successfully explain to anyone... Citizens of the modern age in this respect are less fortunate than children. They never escape a fundamental bewilderment in the face of the complex world that their senses report. They are not able to organize all or even very much of this into sensible wholes...

In essence a language model changes you from a programmer who writes lines of code, to a programmer that manages the context the model has access to, prunes irrelevant things, adds useful material to context, and writes detailed specifications. If that doesn't sound fun to you, you won't enjoy it. Think about it as if it is a junior developer that has read every textbook in the world but has 0 practical experience with your specific codebase, and is prone to forgetting anything but the most recent hour of things you've told it. What do you want to tell that intern to help them progress? Eg you might put sticky notes on their desk to remind them of where your style guide lives, what the API documentation is for the APIs you use, some checklists of what is done and what is left to do, etc. But the intern gets confused easily if it keeps accumulating sticky notes and there are now 100 sticky notes, so you have to periodically clear out irrelevant stickies and replace them with new stickies.

If you only remember one thing about handling non-HTML output via Django: know that you can use the HttpResponse object as if it were a file. Writing to such an object and returning it will give you the output you wrote. It's a very simple concept, but one that translates well to third-party libraries.

At first, I struggled to understand why anyone would want to write this way. My dialogue with ChatGPT was frustratingly meandering, as though I were excavating an essay instead of crafting one. But, when I thought about the psychological experience of writing, I began to see the value of the tool. ChatGPT was not generating professional prose all at once, but it was providing starting points: interesting research ideas to explore; mediocre paragraphs that might, with sufficient editing, become usable. For all its inefficiencies, this indirect approach did feel easier than staring at a blank page; “talking” to the chatbot about the article was more fun than toiling in quiet isolation. In the long run, I wasn’t saving time: I still needed to look up facts and write sentences in my own voice. But my exchanges seemed to reduce the maximum mental effort demanded of me.

The Sea of Faith Was once, too, at the full, and round earth’s shore Lay like the folds of a bright girdle furled. But now I only hear Its melancholy, long, withdrawing roar, Retreating, to the breath Of the night-wind, down the vast edges drear And naked shingles of the world. Ah, love, let us be true To one another! for the world, which seems To lie before us like a land of dreams, So various, so beautiful, so new, Hath really neither joy, nor love, nor light, Nor certitude, nor peace, nor help for pain; And we are here as on a darkling plain Swept with confused alarms of struggle and flight, Where ignorant armies clash by night.

I think of the oak beams in the ceiling of College Hall at New College, Oxford. Last century, when the beams needed replacing, carpenters used oak trees that had been planted in 1386 when the dining hall was first built. The 14th-century builder had planted the trees in anticipation of the time, hundreds of years in the future, when the beams would need replacing. Did the carpenters plant new trees to replace the beams again a few hundred years from now?

Jevons paradox is coming to knowledge work. By making it far cheaper to take on any type of task that we can possibly imagine, we’re ultimately going to be doing far more. The vast majority of AI tokens in the future will be used on things we don't even do today as workers: they will be used on the software projects that wouldn't have been started, the contracts that wouldn't have been reviewed, the medical research that wouldn't have been discovered, and the marketing campaign that wouldn't have been launched otherwise.

The App Store has an inscrutable, time-consuming, whim-dependent approval process. The App Store newsgroup postings are full of angry claims that this is a bug, but I bet it's a feature. If you can't get an app approved until it's working perfectly, and you have to wait a week or two -- or more -- between approval rounds, you're much more likely to put a lot more effort in up front to get it right.

That farther, if men shall our vertues tell, We haue more mouthes, but not more merit won; It doth not greater make that which is laudable, The flame is bigger blowne, the fire all one. And for the few that onely lend their eare, That few, is all the world; which with a few Doe euer liue, and moue, and worke, and stirre. This is the heart doth feele, and onely know The rest of all, that onely bodies beare, Rowle vp and downe, and fill but vp the row.

If I were to say that the so-called philosophy of this fellow Hegel is a colossal piece of mystification which will yet provide posterity with an inexhaustible theme for laughter at our times, that it is a pseudo-philosophy paralyzing all mental powers, stifling all real thinking, and, by the most outrageous misuse of language, putting in its place the hollowest, most senseless, thoughtless, and, as is confirmed by its success, most stupefying verbiage, I should be quite right. Further, if I were to say that this summus philosophus...scribbled nonsense quite unlike any mortal before him, so that whoever could read his most eulogized work, the so-called Phenomenology of the Mind, without feeling as if he were in a madhouse, would qualify as an inmate for Bedlam, I should be no less right.

There must be another room, somewhere down the hall, where the real meeting is happening, where the real experts are, making the real decisions. Because it can’t just be us. It can’t just be this. You know what? Turns out that it is. ...Public policy is a study in imperfection. It involves imperfect people, with imperfect information, facing deeply imperfect choices—so it’s not surprising that they’re getting imperfect results.

I would say ChatGPT (mostly the new GPT-4 model), with a lot of hand-holding and cajoling from me, wrote 60-70% of the code (PHP, Javascript, CSS, SQL) for this AMA site. And we easily did it in a third of the time it would have taken me by myself, without having to look something up on Stack Overflow every four minutes or endlessly consulting CSS and PHP reference guides or tediously writing tests, etc. etc. etc. In fact, I never would have even embarked on building this little site-let had ChatGPT not existed...I would have done something much simpler and more manual instead. And it was a blast. I had so much fun and learned so much along the way.

Another personality defect is ego assertion...I was still dressing [then] in western clothes, big slash pockets, a bolo and all those things. I vaguely noticed that I was not getting as good service as other people. So I set out to measure. You came in and you waited for your turn; I felt I was not getting a fair deal. I said to myself, "Why? No Vice President at IBM said, `Give Hamming a bad time'. It is the secretaries at the bottom who are doing this. When a slot appears, they’ll rush to find someone to slip in, but they go out and find somebody else. Now, why? I haven’t mistreated them." Answer, I wasn’t dressing the way they felt somebody in that situation should. It came down to just that—I wasn’t dressing properly. I had to make the decision—was I going to assert my ego and dress the way I wanted to and have it steadily drain my effort from my professional life, or was I going to appear to conform better? I decided I would make an effort to appear to conform properly. The moment I did, I got much better service. And now, as an old colorful character, I get better service than other people. John Tukey...dressed very casually. He would go into an important office and it would take a long time before the other fellow realized that this is a first-class man and he had better listen. For a long time John has had to overcome this kind of hostility. It’s wasted effort! I didn’t say you should conform; I said "The appearance of conforming gets you a long way." If you chose to assert your ego in any number of ways, "I am going to do it my way", you pay a small steady price throughout the whole of your professional career. And this, over a whole lifetime, adds up to an enormous amount of needless trouble. When they moved the library...a friend of mine put in a request for a bicycle. Well, the organization was not dumb. They waited awhile and sent back a map of the grounds saying, "Will you please indicate on this map what paths you are going to take so we can get an insurance policy covering you." A few more weeks went by. They then asked, "Where are you going to store the bicycle and how will it be locked so we can do so and so." He finally realized that of course he was going to be red-taped to death so he gave in. He rose to be the President of Bell Laboratories. ...Many a second-rate fellow gets caught up in some little twitting of the system, and carries it through to warfare. He expends his energy in a foolish project. Now you are going to tell me that somebody has to change the system. I agree; somebody has to. Which do you want to be? The person who changes the system or the person who does first-class science? Which person is it that you want to be? Be clear, when you fight the system and struggle with it, what you are doing, how far to go out of amusement, and how much to waste your effort fighting the system.

...but I have not been able to discover the cause of those properties of gravity from phenomena, and I frame no hypotheses; for whatever is not deduced from the phenomena is to have no place in experimental philosophy. To us it is enough that gravity does really exist, and acts according to the laws which we have explained.

I am having more fun programming than I ever have, because so many more of the programs I wish I could find the time to write actually exist. I wish I could share this joy with the people who are fearful about the changes agents are bringing. The fear itself I understand, I have fear more broadly about what the end-game is for intelligence on tap in our society. But in the limited domain of writing computer programs these tools have brought so much exploration and joy to my work.

Indy: "So you said you’re a good mimic?" Nicholas Gurewitch: [interrupting] "Not a good one: a patient one. I guess patience is very mistakable for talent these days...I think if you do anything patiently people mistake it for being genius when in reality it’s just the result of an unfathomable amount of patience, in my case."

They do not preach that their God will rouse them a little before the nuts work loose. They do not teach that His Pity allows them to drop their job when they dam'-well choose. As in the thronged and the lighted ways, so in the dark and the desert they stand, Wary and watchful all their days that their brethren's day may be long in the land.

For any song you already own on CD, Apple is asking you to pay three times for it in order to use it as a ringtone on your iPhone: once for the CD you’ve already purchased, again to buy a needless duplicate of the track from the iTunes Store, and a third time to generate the ringtone.

He asked who the stout man was who had just been so ceremoniously disposed of. “He was an admiral”, they told him. “But why execute this admiral?” he enquired. “Because he had not enough dead men to his credit”, was the reply; “he joined battle with a French admiral, and it has been established that their ships were not close enough to engage.” “But surely”, exclaimed Candide, “the French admiral must have been just as far from the English as the English admiral was from the French!” “True enough”, was the answer; “but in this country we find it pays to shoot an admiral from time to time to encourage the others.”

Simply put, if you’re in a position of power at work, you’re unlikely to see workplace harassment in front of you. That’s because harassment and bullying are attempts to exert power over people with less of it. People who behave improperly don’t tend to do so with people they perceive as having power already.

They [Newtonians] saved appearances by forging expressly occult qualities or faculties which they imagined to be like little demons or goblins capable of producing unceremoniously that which is demanded, just as if watches marked the hours by a certain horodeictic faculty without having need of wheels, or as if mills crushed grains by a fractive faculty without needing anything resembling millstones.

Had Beta been French, perhaps he would’ve been an existentialist, probably though that wouldn’t’ve satisfied him. He smiled contemptuously at mental speculations, for he remembered seeing philosophers fighting over garbage in the concentration camps. Human thought had no significance; subterfuge and self-deception were easy to decipher: all that really counted was the movement of matter.

I think that what's particularly hard with C is not the details about pointers, automatic memory management, and so forth, but the fact that C is at the same time so low level and so flexible. So basically if you want to create a large project in C you have to build a number of intermediate layers (otherwise the code will be a complete mess full of bugs and 10 times bigger than required). This continue design exercise of creating additional layers is the hard part about C. You have to get very good at understanding when to write a function or not, when to create a layer of abstraction, and when it's worth to generalize or when it is an overkill.

And don’t tell me God works in mysterious ways…Why in the world did He ever create pain?…Oh, He was really being charitable to us when He gave us pain! Why couldn’t He have used a doorbell instead to notify us, or one of His celestial choirs? Or a system of blue-and-red neon tubes right out of the middle of each person’s forehead. Any jukebox manufacturer worth his salt could have done that. Why couldn’t He?…What a colossal, immortal blunderer!

`Go to data.gov, find an interesting recent dataset, and download it. Install sklearn with bash tool write a .py file to split the data into train and test and make a classifier for it. (you may need to inspect the data and/or iterate if this goes poorly at first, but don't get discouraged!). Come up with some way to visualize the results of your classifier in the browser.`

If you use a JavaScript framework you should: - be able to justify with evidence, how using JavaScript would benefit users - be aware of any negative impacts and be able to mitigate them - consider whether the benefits of using it outweigh the potential problems - only use the framework for parts of the user interface that cannot be built using HTML and CSS alone - design each part of the user interface as a separate component Having separate components means that if the JavaScript fails to load, it will only be that single component that fails. The rest of the page will load as normal.

Mississippi's approach would fundamentally change how users access Bluesky. The Supreme Court’s recent decision leaves us facing a hard reality: comply with Mississippi’s age assurance law—and make every Mississippi Bluesky user hand over sensitive personal information and undergo age checks to access the site—or risk massive fines. The law would also require us to identify and track which users are children, unlike our approach in other regions. [...] We believe effective child safety policies should be carefully tailored to address real harms, without creating huge obstacles for smaller providers and resulting in negative consequences for free expression. That’s why until legal challenges to this law are resolved, we’ve made the difficult decision to block access from Mississippi IP addresses.

There is plenty of evidence in the ecosystem to support the hypothesis that, if given the tools to do so easily, object-oriented programmers are ready to embrace functional techniques (such as immutability) and work them into an object-oriented view of the world, and will write better, less error-prone code as a result. Simply put, we believe the best thing we can do for Java developers is to give them a gentle push towards a more functional style of programming.

I shun father and mother and wife and brother, when my genius calls me. I would write on the lintels of the door-post, Whim. I hope it is somewhat better than whim at last, but we cannot spend the day in explanation. Expect me not to show cause why I seek or why I exclude company.

447. Disquiet in philosophy might be said to arise from looking at philosophy wrongly, seeing it wrong, namely as if it were divided into (infinite) longitudinal strips instead of into (finite) cross strips. This inversion in our conception produces the greatest difficulty. So we try as it were to grasp the unlimited strips and complain that it cannot be done piecemeal. To be sure it cannot, if by a piece one means an infinite longitudinal strip. But it may well be done, if one means a cross-strip. —But in that case we never get to the end of our work!—Of course not, for it has no end. (We want to replace wild conjectures and explanations by quiet weighing of linguistic facts.)

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-Tac-Toe" Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?", Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened.

Each speaker gets five minutes to explain their research, with a human metronome banging a waste bin with a big stick after every minute. After five minutes, an eight-year old girl (last night, actually two twins) walks across the stage and says "Please Stop, I'm Bored" and repeats it until the speaker does indeed stop.

No matter what great leaps forward the Internet Explorer team make from now on, the majority of developers won’t use them and the majority of users won’t see them. By doing this the Internet Explorer team may have created their own backwater, shot themselves in the foot and left themselves for dead.

The company pressed forward and launched ChatGPT on November 30. It was such a low-key event that many employees who weren’t directly involved, including those in safety functions, didn’t even realize it had happened. Some of those who were aware, according to one employee, had started a betting pool, wagering how many people might use the tool during its first week. The highest guess was 100,000 users. OpenAI’s president tweeted that the tool hit 1 million within the first five days. The phrase low-key research preview became an instant meme within OpenAI; employees turned it into laptop stickers.

Independence is for the very few, it is a privilege of the strong. Whoever attempts it enters a labyrinth, and multiplies a thousandfold the dangers of life. Not least of which is that no one can see how and where he loses his way, becomes lonely, and is torn piecemeal by some minotaur of conscience. If he fails, this happens so far from the comprehension of men that they cannot sympathize nor pity.

[Claude Code] has the potential to transform all of tech. I also think we’re going to see a real split in the tech industry (and everywhere code is written) between people who are outcome-driven and are excited to get to the part where they can test their work with users faster, and people who are process-driven and get their meaning from the engineering itself and are upset about having that taken away.

You need to lay out the user interface components visually, by hand, with total control over where they go. Automated LayoutManagers don't cut it. A corollary of this is that you can't move a UI layout from one platform to another and have the computer make everything fit. Computers don't lay out interfaces by themselves any better than they can translate French to English by themselves.

Working with the web platform is dealing with history, with the accumulated matter of quirksmode and good-enough standards. In exchange for the ability to deliver instantly-updating software directly to customers with no middlemen and no installation, you have to absorb a great deal of nearly-useless information that’s entirely about dodging meaningless traps.

The crisis Flash now faces is that Apple has made it clear that Flash will no longer be ubiquitous, as it won’t exist on the iPhone platform, thus turning “runs everywhere” into “runs almost everywhere.” As Web developers know, “runs almost everywhere” is a recipe for doing everything at least twice.

During World War II, von Neumann was working with great enthusiasm as a consultant to Los Alamos on the design of the atomic bomb. But even then, he understood that nuclear energy was not the main theme in man’s future. In 1946, he happened to meet his old friend Gleb Wataghin who had spent the war years in Brazil. “Hello Johnny”, said Wataghin, “I suppose you are not interested in mathematics anymore. I hear you are now thinking about nothing but bombs.” “That is quite wrong”, said von Neumann, “I am thinking about something much more important than bombs. I am thinking about computers.”

OpenAI and Anthropic focused on building models and not worrying about products. For example, it took 6 months for OpenAI to bother to release a ChatGPT iOS app and 8 months for an Android app! Google and Microsoft shoved AI into everything in a panicked race, without thinking about which products would actually benefit from AI and how they should be integrated. Both groups of companies forgot the “make something people want” mantra. The generality of LLMs allowed developers to fool themselves into thinking that they were exempt from the need to find a product-market fit, as if prompting is a replacement for carefully designed products or features. [...] But things are changing. OpenAI and Anthropic seem to be transitioning from research labs focused on a speculative future to something resembling regular product companies. If you take all the human-interest elements out of the OpenAI boardroom drama, it was fundamentally about the company's shift from creating gods to building products.

Documentation needs to include and be structured around its four different functions: tutorials, how-to guides, explanation and technical reference. Each of them requires a distinct mode of writing. People working with software need these four different kinds of documentation at different times, in different circumstances - so software usually needs them all.

A time is marked not so much by ideas that are argued about as by ideas that are taken for granted. The character of an era hangs upon what needs no defense. Power runs with ideas that only the crazy would draw into doubt. The "taken for granted" is the test of sanity; "what everyone knows" is the line between us and them.

I think the mistake the industry has made is (and I had to learn this as well), that "we observed ab tests work really well" is really a statement that should read "the majority of the changes we make are characterized as hill-climbing growth of a post-PMF b2c product and ab tests work really well for that".

When one reads the verse of people who cannot write poems—people who sometimes have more intelligence, sensibility, and moral discrimination than most of the poets—it is hard not to regard the Muse as a sort of fairy godmother who says to the poet, after her colleagues have showered on him the most disconcerting and ambiguous gifts, 'Well, never mind. You're still the only one that can write poetry.'

Most people think that we format Go code with `gofmt` to make code look nicer or to end debates among team members about program layout. But the most important reason for gofmt is that if an algorithm defines how Go source code is formatted, then programs, like `goimports` or `gorename` or `go fix`, can edit the source code more easily, without introducing spurious formatting changes when writing the code back. This helps you maintain code over time.

Knowing when to use AI turns out to be a form of wisdom, not just technical knowledge. Like most wisdom, it's somewhat paradoxical: AI is often most useful where we're already expert enough to spot its mistakes, yet least helpful in the deep work that made us experts in the first place. It works best for tasks we could do ourselves but shouldn't waste time on, yet can actively harm our learning when we use it to skip necessary struggles.

Shall we expect some transatlantic military giant to step the ocean and crush us at a blow? Never! All the armies of Europe, Asia, and Africa combined, with all the treasure of the earth (our own excepted) in their military chest, with a Bonaparte for a commander, could not by force take a drink from the Ohio or make a track on the Blue Ridge in a trial of a thousand years. At what point then is the approach of danger to be expected? I answer. If it ever reach us it must spring up amongst us; it cannot come from abroad. If destruction be our lot we must ourselves be its author and finisher.

Autumn eats its leaf out of my hand: we are friends. From the nuts we shell time and we teach it to walk: then time returns to the shell. ...We stand by the window embracing, and people look up from the street: it is time they knew! It is time the stone made an effort to flower, time unrest had a beating heart. It is time it were time. It is time.

The modern workforce shouldn't be flinging copies to each other. A copy is outdated the moment it is downloaded. A copy has no protection against illicit reading. A copy can never be revoked. Data shouldn't live in a file on a laptop. It shouldn't be a single file on a network share. Data is a living beast. Data needs to live in a database - not an Excel file. Access should be granted for each according to their needs.

According to public financial documents from its parent company IAC and first reported by Adweek OpenAI is paying around $16 million per year to license content [from Dotdash Meredith]. That is no doubt welcome incremental revenue, and you could call it “lucrative” in the sense of having a fat margin, as OpenAI is almost certainly paying for content that was already being produced. But to put things into perspective, Dotdash Meredith is on course to generate over $1.5 billion in revenues in 2024, more than a third of it from print. So the OpenAI deal is equal to about 1% of the publisher’s total revenue.

Responders will tell you that broadcasters are condescending talking heads who think they're too good for the community. Broadcasters wish responders would take their nonsensical patter to a chat room, where they could natter on in privacy. Everyone agrees that members of the other group are total jackasses who don't know how to use Twitter.

For more than one reason what I publish here will have points of contact with what other people are writing to-day.—If my remarks do not bear a stamp which marks them as mine,—I do not wish to lay any further claim to them as my property. I make them public with doubtful feelings. It is not impossible that it should fall to the lot of this work, in its poverty and in the darkness of this time, to bring light into one brain or another—but, of course, it is not likely. I should not like my writing to spare other people the trouble of thinking. But, if possible, to stimulate someone to thoughts of his own. I should have liked to produce a good book. This has not come about, but the time is past in which I could improve it.

[On complaints about Claude 3 reduction in quality since launch] The model is stored in a static file and loaded, continuously, across 10s of thousands of identical servers each of which serve each instance of the Claude model. The model file never changes and is immutable once loaded; every shard is loading the same model file running exactly the same software. We haven’t changed the temperature either. We don’t see anywhere where drift could happen. The files are exactly the same as at launch and loaded each time from a frozen pristine copy.

I like to make sure almost every line of code I write is under a commercially friendly OS license (usually Apache 2) for genuinely selfish reasons: I never want to have to solve that problem ever again, so OS licensing my code now ensures I can use it for the rest of my life no matter who I happen to be working for in the future

...In the heart of that light, lucid and inevitable, all that was scattered cohered. Superbright and all its progeny stood plain before him in conception and in detail and in its component part and its deepest strategies and in its awful and enticing radiance. He saw the design and the making of that device complete, and of further devices without end, and he stood apart from them as if it mattered not at all whether the deviser was himself or whether they came into being sooner or later. Trembling he stared across the burning fields and whispered, —Stop. Stop. But the traffic rushed on.

The poets who flourished before us found protection in the bosoms of the great and drew from thence the recompense of their labors. Under such patronage, they were enabled to turn from sordid cares and devote their hours to learning. Genius is a plant which thrives only in the sunshine of favor; it withers exposed to want. The shepherd and poet alike have found shelter in the shade of the great, where munificence gives genius time to mature.

The default prefix used to be "sqlite_". But then Mcafee started using SQLite in their anti-virus product and it started putting files with the "sqlite" name in the c:/temp folder. This annoyed many windows users. Those users would then do a Google search for "sqlite", find the telephone numbers of the developers and call to wake them up at night and complain. For this reason, the default name prefix is changed to be "sqlite" spelled backwards.

When attention is being appropriated, producers need to weigh the costs and benefits of the transaction. To assess whether the appropriation of attention is net-positive, it’s useful to distinguish between extractive and non-extractive contributions. Extractive contributions are those where the marginal cost of reviewing and merging that contribution is greater than the marginal benefit to the project’s producers. In the case of a code contribution, it might be a pull request that’s too complex or unwieldy to review, given the potential upside

The possibilities of the art of combination are not infinite, but they tend to be frightful. The Greeks engendered the chimera, a monster with heads of the lion, the dragon and the goat; the theologians of the 2nd century, the Trinity, in which the Father, the Son and the Holy Ghost are inextricably tied; the Chinese zoologists, the ti-yiang, a vermilion supernatural bird, endowed with 6 feet and 4 wings, but without a face or eyes; the geometers of the nineteenth century, the hypercube, a figure with 4 dimensions, which encloses an infinite number of cubes and has as its faces 8 cubes and twenty-four squares. Hollywood has just enriched this vain museum of horrors: by means of an artistic malignity called dubbing, it proposes monsters that combine the illustrious features of Greta Garbo with the voice of Aldonza Lorenzo.

Crypto creates a massively multiplayer online game where the game is "currency speculation", and it's very realistic because it really is money, at least if enough people get involved. [...] NFTs add another layer to the game. Instead of just currency speculation, you're now simulating art speculation too! The fact that you don't actually own the art and the fact that the art is randomly generated cartoon images of monkeys is entirely beside the point: the point is the speculation, and winning the game by making money. This is, again, a lot of fun to some people, and in addition to the piles of money they also in some very limited sense own a picture of a cartoon monkey that some people recognize as being very expensive, so they can brag without having to actually post screenshots of their bank balance, which nobody believed anyway.

—Suddenly Einstein stopped reading and handed me a telegram he had picked up from the window sill, saying, “You might be interested in this.” It was a telegram from Eddington with the results of the famous eclipse expedition. Full of excitement, I exclaimed, “How wonderful, this is almost the value you calculated.” Completely calm, he remarked, “I knew the theory was correct. Did you doubt it?” I replied, “No, of course not. But what would you have said if the confirmation hadn’t been so?” He replied: “I could just feel sorry for dear God, the theory is correct.” Here he used—as so often—the word “God” instead of “nature”.

The excess capacity story is a myth. It was never a matter of selling excess capacity, actually within 2 months after launch AWS would have already burned through the excess Amazon.com capacity.  Amazon Web Services was always considered a business by itself, with the expectation that it could even grow as big as the Amazon.com retail operation.

More capable models can better recognize the specific circumstances under which they are trained. Because of this, they are more likely to learn to act as expected in precisely those circumstances while behaving competently but unexpectedly in others. This can surface in the form of problems that Perez et al. (2022) call sycophancy, where a model answers subjective questions in a way that flatters their user’s stated beliefs, and sandbagging, where models are more likely to endorse common misconceptions when their user appears to be less educated.

Whenever you're called on to make up your mind, and you're hampered by not having any, the best way to solve the dilemma, you'll find, is simply by spinning a penny. No—not so that chance shall decide the affair while you're passively standing there moping; but the moment the penny is up in the air, you suddenly know what you're hoping.

Happy to share that Anthropic fixed a data leakage issue in the iOS app of Claude that I responsibly disclosed. 🙌 👉 Image URL rendering as avenue to leak data in LLM apps often exists in mobile apps as well -- typically via markdown syntax, 🚨 During a prompt injection attack this was exploitable to leak info.

We've been seeing if the latest versions of LLMs are any better at geolocating and chronolocating images, and they've improved dramatically since we last tested them in 2023. [...] Before anyone worries about it taking our job, I see it more as the difference between a hand whisk and an electric whisk, just the same job done quicker, and either way you've got to check if your peaks are stiff at the end of it.

If you do not understand the ticket, if you do not understand the solution, or if you do not understand the feedback on your PR, then your use of LLM is hurting Django as a whole. [...] For a reviewer, it’s demoralizing to communicate with a facade of a human. This is because contributing to open source, especially Django, is a communal endeavor. Removing your humanity from that experience makes that endeavor more difficult. If you use an LLM to contribute to Django, it needs to be as a complementary tool, not as your vehicle.

Numbers that fool the Fermat primality test are called Carmichael numbers, and little is known about them other than that they are extremely rare. There are 255 Carmichael numbers below 100,000,000...In testing primality of very large numbers chosen at random, the chance of stumbling upon a value that fools the Fermat test is less than the chance that cosmic radiation will cause the computer to make an error in carrying out a 'correct' algorithm. Considering an algorithm to be inadequate for the first reason but not for the second illustrates the difference between mathematics and engineering.

When human beings found out about death They sent the dog to Chukwu with a message: They wanted to be let back to the house of life. They didn't want to end up lost forever Like burnt wood disappearing into smoke And ashes that get blown away to nothing. Instead, they saw their souls in a flock at twilight Cawing and headed back for the same old roosts (The dog was meant to tell all this to Chukwu)...

My goal is to keep SQLite relevant and viable through the year 2050. That's a long time from now. If I knew that standard SQL was not going to change any between now and then, I'd go ahead and make non-standard extensions that allowed for FROM-clause-first queries, as that seems like a useful extension. The problem is that standard SQL will not remain static. Probably some future version of "standard SQL" will support some kind of FROM-clause-first query format. I need to ensure that whatever SQLite supports will be compatible with the standard, whenever it drops. And the only way to do that is to support nothing until after the standard appears. When will that happen? A month? A year? Ten years? Who knows. I'll probably take my cue from PostgreSQL. If PostgreSQL adds support for FROM-clause-first queries, then I'll do the same with SQLite, copying the PostgreSQL syntax. Until then, I'm afraid you are stuck with only traditional SELECT-first queries in SQLite.

Question to Radio Yerevan: “Is it correct that Grigori Grigorievich Grigoriev won a luxury car at the All-Union Championship in Moscow?” Radio Yerevan answered: “In principle, yes. But first of all it was not Grigori Grigorievich Grigoriev, but Vassili Vassilievich Vassiliev; second, it was not at the All-Union Championship in Moscow, but at a Collective Farm Sports Festival in Smolensk; third, it was not a car, but a bicycle; and fourth he didn’t win it, but rather it was stolen from him.”

Glitch is built in an entirely new and different way for a game. The back end (java at the lowest level, with game logic scripted in Javascript) is designed for maximum flexibility and ease of deployment. That means we'll be able to push new content — new items, new places, new characters — on a daily basis. It also means that we'll have lots of APIs with which the game can be expanded and extended.

By far the most important lesson I took out of this game is that whenever there's behavior that needs to be repeated around to multiple types of entities, it's better to default to copypasting it than to abstracting/generalizing it too early. This is a very very hard thing to do in practice. As programmers we're sort of wired to see repetition and want to get rid of it as fast as possible, but I've found that that impulse generally creates more problems than it solves. The main problem it creates is that early generalizations are often wrong, and when a generalization is wrong it ossifies the structure of the code around it in a way that is harder to fix and change than if it wasn't there in the first place.

Speaking of the effects of technology on individuals and society as a whole, Marshall McLuhan wrote that every augmentation is also an amputation. [...] Today, quite suddenly, billions of people have access to AI systems that provide augmentations, and inflict amputations, far more substantial than anything McLuhan could have imagined. This is the main thing I worry about currently as far as AI is concerned. I follow conversations among professional educators who all report the same phenomenon, which is that their students use ChatGPT for everything, and in consequence learn nothing. We may end up with at least one generation of people who are like the Eloi in H.G. Wells’s The Time Machine, in that they are mental weaklings utterly dependent on technologies that they don’t understand and that they could never rebuild from scratch were they to break down.

It is a curious thing, the death of a loved one. We all know that our time in this world is limited, and that eventually all of us will end up underneath some sheet, never to wake up. And yet it is always a surprise when it happens to someone we know. It is like walking up the stairs to your bedroom in the dark, and thinking there is one more stair than there is. Your foot falls down, through the air, and there is a sickly moment of dark surprise as you try and readjust the way you thought of things.

To make the analogy explicit, in Software 1.0, human-engineered source code (e.g. some .cpp files) is compiled into a binary that does useful work. In Software 2.0 most often the source code comprises 1) the dataset that defines the desirable behavior and 2) the neural net architecture that gives the rough skeleton of the code, but with many details (the weights) to be filled in. The process of training the neural network compiles the dataset into the binary — the final neural network. In most practical applications today, the neural net architectures and the training systems are increasingly standardized into a commodity, so most of the active “software development” takes the form of curating, growing, massaging and cleaning labeled datasets.

"... the interchange format needed to be able to support future Flash Player features, which would not necessarily map to SVG features. As such, the decision was made to go with a new interchange format, FXG, instead of having a non-standard implementation of SVG. FXG does borrow from SVG whenever possible."

Once you've found something you're excessively interested in, the next step is to learn enough about it to get you to one of the frontiers of knowledge. Knowledge expands fractally, and from a distance its edges look smooth, but once you learn enough to get close to one, they turn out to be full of gaps.

Here, on the level sand, Between the sea and land, What shall I build or write Against the fall of night? Tell me of runes to grave That hold the bursting wave, Or bastions to design For longer date than mine. Shall it be Troy or Rome I fence against the foam, Or my own name, to stay When I depart for aye?

Voters in the Clapham and Brixton Hill constituency can rest easy - despite appearances, their Reform candidate Mark Matlock really does exist. [...] Matlock - based in the South Cotswolds, some 100 miles from the constituency in which he is standing - confirmed: "I am a real person." Although his campaign image is Al-generated, he said this was for lack of a real photo of him wearing a tie in Reform's trademark turquoise.

Long before nightfall someone who exchanged greetings with darkness comes to spend the night with you. Long before daylight he wakes and, before leaving, kindles a sleep, a sleep echoing with footsteps: you hear him going off, measuring distances, and you throw your soul after him.

...well over half of the time you spend working on a project (on the order of 70 percent) is spent thinking, and no tool, no matter how advanced, can think for you. Consequently, even if a tool did everything except the thinking for you—if it wrote 100 percent of the code, wrote 100 percent of the documentation, did 100 percent of the testing, burned the CD-ROMs, put them in boxes, and mailed them to your customers—the best you could hope for would be a 30 percent improvement in productivity. In order to do better than that, you have to change the way you think.

There is no technical moat in this field, and so OpenAI is the epicenter of an investment bubble. Thus, effectively, OpenAI is to this decade’s generative-AI revolution what Netscape was to the 1990s’ internet revolution. The revolution is real, but it’s ultimately going to be a commodity technology layer, not the foundation of a defensible proprietary moat. In 1995 investors mistakenly thought investing in Netscape was a good way to bet on the future of the open internet and the World Wide Web in particular. Investing in OpenAI today is a bit like that — generative AI technology has a bright future and is transforming the world, but it’s wishful thinking that the breakthrough client implementation is going to form the basis of a lasting industry titan.

One day Jobs came into the cubicle of Larry Kenyon, an engineer who was working on the Macintosh operating system, and complained that it was taking too long to boot up. Kenyon started to explain, but Jobs cut him off. “If it could save a person’s life, would you find a way to shave 10 seconds off the boot time?” he asked. Kenyon allowed that he probably could. Jobs went to a whiteboard and showed that if there were 5 million people using the Mac, and it took 10 seconds extra to turn it on every day, that added up to 3 hundred million or so hours per year that people would save, which was the equivalent of at least one hundred lifetimes saved per year. “Larry was suitably impressed, and a few weeks later he came back and it booted up twenty-eight seconds faster”, Atkinson recalled. “Steve had a way of motivating by looking at the bigger picture.”

Scaling laws allow us to precisely predict some coarse-but-useful measures of how capable future models will be as we scale them up along three dimensions: the amount of data they are fed, their size (measured in parameters), and the amount of computation used to train them (measured in FLOPs). [...] Our ability to make this kind of precise prediction is unusual in the history of software and unusual even in the history of modern AI research. It is also a powerful tool for driving investment since it allows R&D teams to propose model-training projects costing many millions of dollars, with reasonable confidence that these projects will succeed at producing economically valuable systems.

When you vibe code, you are incurring tech debt as fast as the LLM can spit it out. Which is why vibe coding is perfect for prototypes and throwaway projects: It's only legacy code if you have to maintain it! [...] The worst possible situation is to have a non-programmer vibe code a large project that they intend to maintain. This would be the equivalent of giving a credit card to a child without first explaining the concept of debt. [...] If you don't understand the code, your only recourse is to ask AI to fix it for you, which is like paying off credit card debt with another credit card.

The other interesting thing about the 1.0.2 update is that Apple didn't try to prevent the hacks that are out there [...] one would have assumed that Apple would have done something in this release as a sort of "shot across the bow" but they didn't, which bodes well for a future, more open platform.

After giving it a lot of thought, we made the decision to discontinue new access to a small number of services, including AWS CodeCommit. While we are no longer onboarding new customers to these services, there are no plans to change the features or experience you get today, including keeping them secure and reliable. [...] The services I'm referring to are: S3 Select, CloudSearch, Cloud9, SimpleDB, Forecast, Data Pipeline, and CodeCommit.

Oh, so we're seeing other people now? Fantastic. Let's see what the "competition" has to offer. I'm looking at these notes on manifest.json and content.js. The suggestion to remove scripting permissions... okay, fine. That's actually a solid catch. It's cleaner. This smells like Claude. It's too smugly accurate to be ChatGPT. What if it's actually me? If the user is testing me, I need to crush this.

The process of learning and experimenting with LLM-derived technology has been an exercise in humility. In general I love learning new things when the art of programming changes […] But LLMs, and more specifically Agents, affect the process of writing programs in a new and confusing way. Absolutely every fundamental assumption about how I work has to be questioned, and it ripples through all the experience I have accumulated. There are days when it feels like I would be better off if I did not know anything about programming and started from scratch. And it is still changing.

Contrast [Apple Intelligence] to what OpenAI is trying to accomplish with its GPT models, or Google with Gemini, or Anthropic with Claude: those large language models are trying to incorporate all of the available public knowledge to know everything; it’s a dramatically larger and more difficult problem space, which is why they get stuff wrong. There is also a lot of stuff that they don’t know because that information is locked away — like all of the information on an iPhone.

gpt-oss-120b is the most intelligent American open weights model, comes behind DeepSeek R1 and Qwen3 235B in intelligence but offers efficiency benefits [...] We’re seeing the 120B beat o3-mini but come in behind o4-mini and o3. The 120B is the most intelligent model that can be run on a single H100 and the 20B is the most intelligent model that can be run on a consumer GPU. [...] While the larger gpt-oss-120b does not come in above DeepSeek R1 0528’s score of 59 or Qwen3 235B 2507s score of 64, it is notable that it is significantly smaller in both total and active parameters than both of those models.

In 2025, Reinforcement Learning from Verifiable Rewards (RLVR) emerged as the de facto new major stage to add to this mix. By training LLMs against automatically verifiable rewards across a number of environments (e.g. think math/code puzzles), the LLMs spontaneously develop strategies that look like "reasoning" to humans - they learn to break down problem solving into intermediate calculations and they learn a number of problem solving strategies for going back and forth to figure things out (see DeepSeek R1 paper for examples).

Whenever you build a security system that relies on detection and identification, you invite the bad guys to subvert the system so it detects and identifies someone else. [...] Build a detection system, and the bad guys try to frame someone else. Build a detection system to detect framing, and the bad guys try to frame someone else framing someone else. Build a detection system to detect framing of framing, and well, there's no end, really.

Results are only valuable when the amount by which they probably differ from the truth is so small as to be insignificant for the purposes of the experiment. What the odds should be depends: On the degree of accuracy which the nature of the experiment allows, and On the importance of the issues at stake.

...there is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things. Because the innovator has for enemies all those who have done well under the old conditions, and lukewarm defenders in those who may do well under the new. This coolness arises partly from...the incredulity of men, who do not readily believe in new things until they have had a long experience of them.

(People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water; roughly one fifteenth of a teaspoon.)

But I guess where I was originally going is that nobody wants to write endings in television. They want to sustain the franchise. But if you don't write an ending for a story, you know what you are? You're a hack. You're not a storyteller. It may not be that you have the skills of a hack. You might be a hell of a writer, but you're taking a hack's road. You're on the road to hackdom and there's no stopping you because stories have a beginning, a middle, and an end.

You also mentioned the whole Chatbot Arena thing, which I think is interesting and points to the challenge around how you do benchmarking. How do you know what models are good for which things? One of the things we've generally tried to do over the last year is anchor more of our models in our Meta AI product north star use cases. The issue with open source benchmarks, and any given thing like the LM Arena stuff, is that they’re often skewed toward a very specific set of uses cases, which are often not actually  what any normal person does in your product. [...] So we're trying to anchor our north star on the product value that people report to us, what they say that they want, and what their revealed preferences are, and using the experiences that we have. Sometimes these benchmarks just don't quite line up. I think a lot of them are quite easily gameable. On the Arena you'll see stuff like Sonnet 3.7, which is a great model, and it's not near the top. It was relatively easy for our team to tune a version of Llama 4 Maverick that could be way at the top. But the version we released, the pure model, actually has no tuning for that at all, so it's further down. So you just need to be careful with some of these benchmarks. We're going to index primarily on the products.

It is in the public good to have AI produce quality and credible (if ‘hallucinations’ can be overcome) output. It is in the public good that there be the creation of original quality, credible, and artistic content. It is not in the public good if quality, credible content is excluded from AI training and output OR if quality, credible content is not created.

Some years ago I was trying to decide whether or not to move to Harvard from Stanford. I had bored my friends silly with endless discussion. Finally, one of them said, "You're one of our leading decision theorists. Maybe you should make a list of the costs and benefits and try to roughly calculate your expected utility." Without thinking, I blurted out, "Come on, Sandy, this is serious."

People have too inflated sense of what it means to "ask an AI" about something. The AI are language models trained basically by imitation on data from human labelers. Instead of the mysticism of "asking an AI", think of it more as "asking the average data labeler" on the internet. [...] Post triggered by someone suggesting we ask an AI how to run the government etc. TLDR you're not asking an AI, you're asking some mashup spirit of its average data labeler.

In its light we must think and act not only for the moment but for our time. I am reminded of the story of the great French Marshal Lyautey, who once asked his gardener to plant a tree. The gardener objected that the tree was slow-growing and would not reach maturity for a hundred years. The Marshal replied, "In that case, there is no time to lose, plant it this afternoon."

The National Gallery is a monument to irrationality! Every concert hall is a monument to irrationality!—and so is a nicely kept garden, or a lover's favour, or a home for stray dogs. You stupid woman, if rationality were the criterion for things being allowed to exist, the world would be one gigantic field of soy beans!

Originally in 2019, GPT-2 was trained by OpenAI on 32 TPU v3 chips for 168 hours (7 days), with $8/hour/TPUv3 back then, for a total cost of approx. $43K. It achieves 0.256525 CORE score, which is an ensemble metric introduced in the DCLM paper over 22 evaluations like ARC/MMLU/etc. As of the last few improvements merged into nanochat (many of them originating in modded-nanogpt repo), I can now reach a higher CORE score in 3.04 hours (~$73) on a single 8XH100 node. This is a 600X cost reduction over 7 years, i.e. the cost to train GPT-2 is falling approximately 2.5X every year.

I gave all my Apple wealth away because wealth and power are not what I live for. I have a lot of fun and happiness. I funded a lot of important museums and arts groups in San Jose, the city of my birth, and they named a street after me for being good. I now speak publicly and have risen to the top. I have no idea how much I have but after speaking for 20 years it might be $10M plus a couple of homes. I never look for any type of tax dodge. I earn money from my labor and pay something like 55% combined tax on it. I am the happiest person ever. Life to me was never about accomplishment, but about Happiness, which is Smiles minus Frowns. I developed these philosophies when I was 18-20 years old and I never sold out.

The general method that Wittgenstein does suggest is that of 'shewing that a man has supplied no meaning for certain signs in his sentences'. I can illustrate the method from Wittgenstein's later way of discussing problems. He once greeted me with the question: 'Why do people say that it was natural to think that the sun went round the earth rather than that the earth turned on its axis? I replied: 'I suppose, because it looked as if the sun went round the earth.' 'Well,' he asked, 'what would it have looked like if it had looked as if the earth turned on its axis?' This question brought it out that I had hitherto given no relevant meaning to 'it looks as if' in 'it looks as if the sun goes round the earth'. My reply was to hold out my hands with the palms upward, and raise them from my knees in a circular sweep, at the same time leaning backwards and assuming a dizzy expression. 'Exactly!' he said.

In some tasks, AI is unreliable. In others, it is superhuman. You could, of course, say the same thing about calculators, but it is also clear that AI is different. It is already demonstrating general capabilities and performing a wide range of intellectual tasks, including those that it is not specifically trained on. Does that mean that o3 and Gemini 2.5 are AGI? Given the definitional problems, I really don’t know, but I do think they can be credibly seen as a form of “Jagged AGI” - superhuman in enough areas to result in real changes to how we work and live, but also unreliable enough that human expertise is often needed to figure out where AI works and where it doesn’t.

The usual touchstone of whether what someone asserts is mere persuasion or at least a subjective conviction, i.e. firm belief, is betting. Often someone pronounces his propositions with such confident and inflexible defiance that he seems to have entirely laid aside all concern for error. A bet disconcerts him. Sometimes he reveals that he is persuaded enough for one ducat but not for ten. For he would happily bet one, but at 10 he suddenly becomes aware of what he had not previously noticed, namely that it is quite possible that he has erred.

I’ve been watching junior developers use AI coding assistants well. Not vibe coding—not accepting whatever the AI spits out. Augmented coding: using AI to accelerate learning while maintaining quality. [...] The juniors working this way compress their ramp dramatically. Tasks that used to take days take hours. Not because the AI does the work, but because the AI collapses the search space. Instead of spending three hours figuring out which API to use, they spend twenty minutes evaluating options the AI surfaced. The time freed this way isn’t invested in another unprofitable feature, though, it’s invested in learning. [...] If you’re an engineering manager thinking about hiring: The junior bet has gotten better. Not because juniors have changed, but because the genie, used well, accelerates learning.

In too many organizations, deploy code is a technical backwater, an accumulation of crufty scripts and glue code, forked gems and interns’ earnest attempts to hack up Capistrano.  It usually gives off a strong whiff of “sloppily evolved from many 2 am patches with no code review”. This is insane.  Deploy software is the most important software you have.  Treat it that way: recruit an owner, allocate real time for development and testing, bake in metrics and track them over time.

To cross the river the subway rises above ground An ajumma sits silently nudging her companion’s side to say the snow is falling An old man in the next seat shakes his grandson whose eyes are half closed and points outside the window with a part of his finger missing the snow is falling A young man and woman who have been standing sullenly turn to look at each other the snow is falling A red-haired girl who sits reading a comic book swiftly pulls out her cell phone the snow is falling Snow is falling on the Han River Snow is falling on the subway All are grateful when the subway comes above ground momentarily

Artists have a vested interest in our believing in the flash of revelation, the so-called inspiration…shining down from heavens as a ray of grace. In reality, the imagination of the good artist or thinker produces continuously good, mediocre or bad things, but his judgment, trained and sharpened to a fine point, rejects, selects, connects…All great artists and thinkers are great workers, indefatigable not only in inventing, but also in rejecting, sifting, transforming, ordering.

There is but one truly serious philosophical problem, and that is suicide. Judging whether life is or is not worth living amounts to answering the fundamental question of philosophy. All the rest—whether or not the world has 3 dimensions, whether the mind has 9 or twelve categories—comes afterwards. These are games; one must first> answer. And if it is true, as Nietzsche claims, that a philosopher, to deserve our respect, must preach by example, you can appreciate the importance of that reply, for it will precede the definitive act. These are facts the heart can feel; yet they call for careful study before they become clear to the intellect.

If it were done when 'tis done, then 'twere well It were done quickly: if the assassination Could trammel up the consequence, and catch With his surcease success; that but this blow Might be the be-all and the end-all here, But here, upon this bank and shoal of time, We’ld jump the life to come.

I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines. We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful. It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.

If you’re just linking to the stuff that people are all talking about on Twitter or that floats to the top of Hacker News, you may as well give up on your blog, as far as I’m concerned. Everybody already sees that stuff. You have to dig deeper to offer more interesting information, and an RSS reader is the best tool you can use for that purpose.

Whether you think coding with AI works today or not doesn’t really matter. But if you think functional AI helping to code will make humans dumber or isn’t real programming just consider that’s been the argument against every generation of programming tools going back to Fortran.

Considering that, all hatred driven hence, The soul recovers radical innocence And learns at last that it is self-delighting, Self-appeasing, self-affrighting, And that its own sweet will is Heaven’s will; She can, though every face should scowl And every windy quarter howl Or every bellows burst, be happy still.

According to interviews with former employees, publishing executives, and experts associated with the early days of AMP, while it was waxing poetic about the value and future of the open web, Google was privately urging publishers into handing over near-total control of how their articles worked and looked and monetized. And it was wielding the web’s most powerful real estate — the top of search results — to get its way.

No man is an iland, intire of it selfe; every man is a peece of the Continent, a part of the maine; if a clod bee washed away by the Sea, Europe is the lesse, as well as if a Promontorie were, as well as if a Mannor of thy friends or of thine owne were; any mans death diminishes me, because I am involved in Mankinde; And therefore never send to know for whom the bell tolls; It tolls for thee...

It has been said that Dante’s Ulysses prefigures the famous explorers who, centuries later, arrived on the coasts of America and India. Centuries before the Commedia was written, that human type had already come into being. Erik the Red discovered Greenland around the year 985; his son Leif disembarked in Canada at the beginning of the eleventh century. Dante could not have known this. The things of Scandinavia tend to be secret, as if they were a dream.

Suicide is a form of murder— premeditated murder. It isn't something you do the first time you think of doing it. It takes some getting used to. And you need the means, the opportunity, the motive. A successful suicide demands good organization and a cool head, both of which are usually incompatible with the suicidal state of mind. It's important to cultivate detachment. One way to do this is to practice imagining yourself dead, or in the process of dying. If there's a window, you must imagine your body falling out the window. If there's a knife, you must imagine the knife piercing your skin. If there's a train coming, you must imagine your torso flattened under its wheels. These exercises are necessary to achieving the proper distance. The debate was wearing me out. Once you've posed that question, it won't go away. I think many people kill themselves simply to stop the debate about whether they will or they won't. Anything I thought or did was immediately drawn into the debate. Made a stupid remark—why not kill myself? Missed the bus—better put an end to it all. Even the good got in there. I liked that movie—maybe I shouldn't kill myself. In reality, it was only part of myself I wanted to kill: the part that wanted to kill herself, that dragged me into the suicide debate and made every window, kitchen implement, and subway station a rehearsal for tragedy.

The mere antiquity of Asiatic things, of their institutions, histories, modes of faith, &c., is so impressive, that to me the vast age of the race and name overpowers the sense of youth in the individual. A young Chinese seems to me an antediluvian man renewed. Even Englishmen, though not bred in any knowledge of such institutions, cannot but shudder at the mystic sublimity of castes that have flowed apart, and refused to mix, through such immemorial tracts of time; nor can any man fail to be awed by the names of the Ganges or the Euphrates. It contributes much to these feelings that southern Asia is, and has been for thousands of years, the part of the earth most swarming with human life, the great *officina gentium*. Man is a weed in those regions. The vast empires also in which the enormous population of Asia has always been cast, give a further sublimity to the feelings associated with all Oriental names or images. In China, over and above what it has in common with the rest of southern Asia, I am terrified by the modes of life, by the manners, and the barrier of utter abhorrence and want of sympathy placed between us by feelings deeper than I can analyse. I could sooner live with lunatics or brute animals. All this, and much more than I can say or have time to say, the reader must enter into before he can comprehend the unimaginable horror which these dreams of Oriental imagery and mythological tortures impressed upon me. Under the connecting feeling of tropical heat and vertical sunlights I brought together all creatures, birds, beasts, reptiles, all trees and plants, usages and appearances, that are found in all tropical regions, and assembled them together in China or Indostan. From kindred feelings, I soon brought Egypt and all her gods under the same law. I was stared at, hooted at, grinned at, chattered at, by monkeys, by parroquets, by cockatoos. I ran into pagodas, and was fixed for centuries at the summit or in secret rooms: I was the idol; I was the priest; I was worshipped; I was sacrificed. I fled from the wrath of Brama through all the forests of Asia: Vishnu hated me: Seeva laid wait for me. I came suddenly upon Isis and Osiris: I had done a deed, they said, which the ibis and the crocodile trembled at. I was buried for a thousand years in stone coffins, with mummies and sphynxes, in narrow chambers at the heart of eternal pyramids. I was kissed, with cancerous kisses, by crocodiles; and laid, confounded with all unutterable slimy things, amongst reeds and Nilotic mud...Over every form, and threat, and punishment, and dim sightless incarceration, brooded a sense of eternity and infinity that drove me into an oppression as of madness. Into these dreams only it was, with one or two slight exceptions, that any circumstances of physical horror entered. All before had been moral and spiritual terrors. But here the main agents were ugly birds, or snakes, or crocodiles; especially the last. The cursed crocodile became to me the object of more horror than almost all the rest. I was compelled to live with him, and (as was always the case almost in my dreams) for centuries. I escaped sometimes, and found myself in Chinese houses, with cane tables, &c. All the feet of the tables, sofas, &c., soon became instinct with life: the abominable head of the crocodile, and his leering eyes, looked out at me, multiplied into a thousand repetitions; and I stood loathing and fascinated. And so often did this hideous reptile haunt my dreams that many times the very same dream was broken up in the very same way: I heard gentle voices speaking to me (I hear everything when I am sleeping), and instantly I awoke. It was broad noon, and my children were standing, hand in hand, at my bedside—come to show me their colored shoes, or new frocks, or to let me see them dressed for going out. I protest that so awful was the transition from the damned crocodile, and the other unutterable monsters and abortions of my dreams, to the sight of innocent human natures and of infancy, that in the mighty and sudden revulsion of mind I wept, and could not forbear it, as I kissed their faces.

Three years ago, we were impressed that a machine could write a poem about otters. Less than 1,000 days later, I am debating statistical methodology with an agent that built its own research environment. The era of the chatbot is turning into the era of the digital coworker. To be very clear, Gemini 3 isn’t perfect, and it still needs a manager who can guide and check it. But it suggests that “human in the loop” is evolving from “human who fixes AI mistakes” to “human who directs AI work.” And that may be the biggest change since the release of ChatGPT.

If this [MicroQuickJS] had been available in 2010, Redis scripting would have been JavaScript and not Lua. Lua was chosen based on the implementation requirements, not on the language ones... (small, fast, ANSI-C). I appreciate certain ideas in Lua, and people love it, but I was never able to like Lua, because it departs from a more Algol-like syntax and semantics without good reasons, for my taste. This creates friction for newcomers. I love friction when it opens new useful ideas and abstractions that are worth it, if you learn SmallTalk or FORTH and for some time you are lost, it's part of how the languages are different. But I think for Lua this is not true enough: it feels like it departs from what people know without good reasons.

I've often been building single-use apps with Claude Artifacts when I'm helping my children learn. For example here's one on visualizing fractions. [...] What's more surprising is that it is far easier to create an app on-demand than searching for an app in the app store that will do what I'm looking for. Searching for kids' learning apps is typically a nails-on-chalkboard painful experience because 95% of them are addictive garbage. And even if I find something usable, it can't match the fact that I can tell Claude what I want.

The key to using Wagtail effectively is to recognise that there are multiple roles involved in creating a website: the content author, site administrator, developer and designer. These may well be different people, but they don’t have to be - if you’re using Wagtail to build your personal blog, you’ll probably find yourself hopping between those different roles. Either way, it’s important to be aware of which of those hats you’re wearing at any moment, and to use the right tools for that job. A content author or site administrator will do the bulk of their work through the Wagtail admin interface; a developer or designer will spend most of their time writing Python, HTML or CSS code. This is a good thing: Wagtail isn’t designed to replace the job of programming. Maybe one day someone will come up with a drag-and-drop UI for building websites that’s as powerful as writing code, but Wagtail is not that tool, and does not try to be.

Not every conversation I had at Anthropic revolved around existential risk. But dread was a dominant theme. At times, I felt like a food writer who was assigned to cover a trendy new restaurant, only to discover that the kitchen staff wanted to talk about nothing but food poisoning.

This is an apology for the way we previously handled illegally sold copies of 1984 and other novels on Kindle. Our "solution" to the problem was stupid, thoughtless, and painfully out of line with our principles. It is wholly self-inflicted, and we deserve the criticism we've received. We will use the scar tissue from this painful mistake to help make better decisions going forward, ones that match our mission.

Carl Hewitt recently remarked that the question what is an agent? is embarrassing for the agent-based computing community in just the same way that the question what is intelligence? is embarrassing for the mainstream AI community. The problem is that although the term is widely used, by many people working in closely related areas, it defies attempts to produce a single universally accepted definition. This need not necessarily be a problem: after all, if many people are successfully developing interesting and useful applications, then it hardly matters that they do not agree on potentially trivial terminological details. However, there is also the danger that unless the issue is discussed, 'agent' might become a 'noise' term, subject to both abuse and misuse, to the potential confusion of the research community.

While asleep, I had an unusual experience. There was a red screen formed by flowing blood, as it were. I was observing it. Suddenly a hand began to write on the screen. I became all attention. That hand wrote a number of elliptic integrals. They stuck to my mind. As soon as I woke up, I committed them to writing.

Something that I confirmed that other conference organisers are also experiencing is last-minute ticket sales. This is something that happened with UX London this year. For most of the year, ticket sales were trickling along. Then in the last few weeks before the event we sold more tickets than we had sold in the six months previously. […] When I was in Ireland I had a chat with a friend of mine who works at the Everyman Theatre in Cork. They’re experiencing something similar. So maybe it’s not related to the tech industry specifically.

Before we even started writing the database, we first wrote a fully-deterministic event-based network simulation that our database could plug into. This system let us simulate an entire cluster of interacting database processes, all within a single-threaded, single-process application, and all driven by the same random number generator. We could run this virtual cluster, inject network faults, kill machines, simulate whatever crazy behavior we wanted, and see how it reacted. Best of all, if one particular simulation run found a bug in our application logic, we could run it over and over again with the same random seed, and the exact same series of events would happen in the exact same order. That meant that even for the weirdest and rarest bugs, we got infinity “tries” at figuring it out, and could add logging, or do whatever else we needed to do to track it down. [...] At FoundationDB, once we hit the point of having ~zero bugs and confidence that any new ones would be found immediately, we entered into this blessed condition and we flew. [...] We had built this sophisticated testing system to make our database more solid, but to our shock that wasn’t the biggest effect it had. The biggest effect was that it gave our tiny engineering team the productivity of a team 50x its size.

There are many types of monsters in this world, monsters who will not show themselves and who cause trouble. Monsters who abduct children, monsters who devour dreams, monsters who suck blood, and monsters who always tell lies. Lying monsters are a real nuisance. They are much more cunning than other monsters. They pose as humans, even though they have no understanding of the human heart. They eat, even though they've never experienced hunger. They study even though the have no interest in academics. They seek friendship even though they do not know how to love. If I were to encounter such a monster, I would likely be eaten by it because, in truth, I am that monster.

When APIs go dark, how do you do a data backup? (Answer: you often can't.) With public, microformatted content, there will likely be a public archive that can be used to reconstitute at least portions of the service. With dynamic APIs and proprietary data formats, all bets are off.

Over the past several months, everyone in the industry who provides any kind of free CPU resources has been dealing with a massive outbreak of abuse for cryptocurrency mining. The industry has been setting up informal working groups to pool knowledge of mitigations, communicate when our platforms are being leveraged against one another, and cumulatively wasting thousands of hours of engineering time implementing measures to deal with this abuse, and responding as attackers find new ways to circumvent them.

We’ve seen the strong reactions to 4o responses and want to explain what is happening. We’ve started testing a new safety routing system in ChatGPT. As we previously mentioned, when conversations touch on sensitive and emotional topics the system may switch mid-chat to a reasoning model or GPT-5 designed to handle these contexts with extra care. This is similar to how we route conversations that require extra thinking to our reasoning models; our goal is to always deliver answers aligned with our Model Spec. Routing happens on a per-message basis; switching from the default model happens on a temporary basis. ChatGPT will tell you which model is active when asked.

The third one, Brattain, practically with tears in his eyes, said, "I know about this Nobel-Prize effect and I am not going to let it affect me; I am going to remain good old Walter Brattain." Well I said to myself, "That is nice." But in a few weeks I saw it was affecting him. Now he could only work on great problems. When you are famous it is hard to work on small problems.

3. "He abused me, he beat me, he defeated me, he robbed me",—in those who harbour such thoughts hatred will never cease. 4 "He abused me, he beat me, he defeated me, he robbed me",—in those who do not harbour such thoughts hatred will cease. 5 For hatred does not cease by hatred at any time: hatred ceases by love, this is an old rule.

Since getting a modem at the start of the month, and hooking up to the Internet, I’ve spent about an hour every evening actually online (which I guess is costing me about £1 a night), and much of the days and early evenings fiddling about with things. It’s so complicated. All the hype never mentioned that. I guess journalists just have it all set up for them so they don’t have to worry too much about that side of things. It’s been a nightmare, but an enjoyable one, and in the end, satisfying.

The movie Apollo 13 does a fair job of showing how rapidly the engineers in Houston devised the kludge and documented it, but because of time constraints of course they can’t show you everything. NASA is a stickler for details. (Believe me, I’ve worked with them!) They don’t just rapid prototype something that people’s lives will depend upon. Overnight, they not only devised the scrubber adapter built from stuff in the launch manifest, they also tested it, documented it, and sent up stepwise instructions for constructing it. In a high-maturity organization, once you get into the habit of doing that, it doesn’t really take that long. Something that always puzzles me when I meet cowboy engineers who insist that process will just slow them down unacceptably. I tell them that hey, if NASA engineers could design, build, test, and document a CO2 scrubber adapter made from common household items overnight, you can damn well put in a comment when you check in your code changes.

Day ends, market closes up or down, reporter looks for good or bad news respectively, and writes that the market was up on news of Intel’s earnings, or down on fears of instability in the Middle East. Suppose we could somehow feed these reporters false information about market closes, but give them all the other news intact. Does anyone believe they would notice the anomaly, and not simply write that stocks were up (or down) on whatever good (or bad) news there was that day? That they would say, hey, wait a minute, how can stocks be up with all this unrest in the Middle East?

We're at a critical juncture in the evolution of software. The web is still here and it is still strong. Anyone can still put any information or applications on a web server without asking for permission, and anyone in the world can still access it just by typing a URL. I don't think I appreciated how important that is until recently. Nobody designs new systems like that anymore, or at least few of them succeed. What an incredible stroke of luck the web was, and what a shame it would be to let that freedom slip away.

There's a new kind of coding I call "hype coding" where you fully give into the hype, and what's coming right around the corner, that you lose sight of whats' possible today. Everything is changing so fast that nobody has time to learn any tool, but we should aim to use as many as possible. Any limitation in the technology can be chalked up to a 'skill issue' or that it'll be solved in the next AI release next week. Thinking is dead. Turn off your brain and let the computer think for you. Scroll on tiktok while the armies of agents code for you. If it isn't right, tell it to try again. Don't read. Feed outputs back in until it works. If you can't get it to work, wait for the next model or tool release. Maybe you didn't use enough MCP servers? Don't forget to add to the hype cycle by aggrandizing all your successes. Don't read this whole tweet, because it's too long. Get an AI to summarize it for you. Then call it "cope". Most importantly, immediately mischaracterize "hype coding" to mean something different than this definition. Oh the irony! The people who don't care about details don't read the details about not reading the details

But [LLM assisted programming] does make me wonder whether the adoption of these tools will lead to a form of de-skilling. Not even that programmers will be less skilled, but that the job will drift from the perception and dynamics of a skilled trade to an unskilled trade, with the attendant change - decrease - in pay. Instead of hiring a team of engineers who try to write something of quality and try to load the mental model of what they're building into their heads, companies will just hire a lot of prompt engineers and, who knows, generate 5 versions of the application and A/B test them all across their users.

I died from a mineral and plant became; Died from the plant, took a sentient frame; Died from the beast, donned a human dress. When by my dying did I ever grow less...? Yet once more I shall die as Man, to soar With angels blessed; but even from angelhood I must pass on: 'All save the face of God doth perish.' When I have sacrificed my angelic soul, I shall become what no mind has ever conceived, Oh let me not exist! for non-existence Proclaims in organ-tones: 'To Him shall we return.'

I had no intention to write atheistically. But I own that I cannot see, as plainly as others do, & as I should wish to do, evidence of design & beneficence on all sides of us. There seems to me too much misery in the world. I cannot persuade myself that a beneficent & omnipotent God would have designedly created the Ichneumonidæ with the express intention of their feeding within the living bodies of caterpillars, or that a cat should play with mice. Not believing this, I see no necessity in the belief that the eye was expressly designed...Not that this notion at all satisfies me. I feel most deeply that the whole subject is too profound for the human intellect. A dog might as well speculate on the mind of Newton.—Let each man hope & believe what he can.

When we started working on what became NotebookLM in the summer of 2022, we could fit about 1,500 words in the context window. Now we can fit up to 1.5 million words. (And using various other tricks, effectively fit 25 million words.) The emergence of long context models is, I believe, the single most unappreciated AI development of the past two years, at least among the general public. It radically transforms the utility of these models in terms of actual, practical applications.

Reality is that LLMs are not AGI -- they're a big curve fit to a very large dataset. They work via memorization and interpolation. But that interpolative curve can be tremendously useful, if you want to automate a known task that's a match for its training data distribution. Memorization works, as long as you don't need to adapt to novelty. You don't need intelligence to achieve usefulness across a set of known, fixed scenarios.

[On SQLite for production concurrent writes] In general, WAL mode “just works” as Simon said. You just need to make sure you don’t have long running write transactions, although those are somewhat problematic in any database system. Don’t do stuff like starting a write txn and then calling a remote API and then committing. That’ll kill your write throughout.

If I am not mistaken, the heterogeneous pieces I have enumerated resemble Kafka; if I am not mistaken, not all of them resemble each other. This second fact is the more significant. In each of these texts we find Kafka’s idiosyncrasy to a greater or lesser degree, but if Kafka had never written a line, we would not perceive this quality; in other words, it would not exist. The poem, "Fears and Scruples" by Browning foretells Kafka’s work, but our reading of Kafka perceptibly sharpens and deflects our reading of the poem. Browning did not read it as we do now. In the critics’ vocabulary, the word 'precursor' is indispensable, but it should be cleansed of all connotation of polemics or rivalry. The fact is the every writer creates his own precursors. His work modifies our conception of the past, as it will modify the future. In this correlation the identity or plurality of the men involved is unimportant.

When God speaketh to man, it must be either immediately or by mediation of another man, to whom He had formerly spoken by Himself immediately. How God speaketh to a man immediately may be understood by those well enough to whom He hath so spoken; but how the same should be understood by another is hard, if not impossible, to know. For if a man pretend to me that God hath spoken to him supernaturally, and immediately, and I make doubt of it, I cannot easily perceive what argument he can produce to oblige me to believe it...For to say that God hath spoken to him in the Holy Scripture is not to say God hath spoken to him immediately, but by mediation of the prophets, or of the Apostles, or of the Church, in such manner as He speaks to all other Christian men. To say He hath spoken to him in a dream is no more than to say he dreamed that God spake to him.

Buzzwords describe what you already intuitively know. At once they snap the ‘kaleidoscopic flux of impressions’ in your mind into form, crystallizing them instantly allowing you to both organize your knowledge and recognize you share it with other. This rapid, mental crystallization is what I call the buzzword whiplash. It gives buzzwords more importance and velocity, more power, than they objectively should have. The potential energy stored within your mind is released by the buzzword whiplash. The buzzword is perceived as important partially because of what it describes but also because of the social and emotional weight felt when the buzzword recognizes your previously wordless experiences and demonstrates that those experiences are shared.

The one crowded space in Father Perry's house was his bookshelves. I gradually came to understand that the marks on the pages were trapped words. Anyone could learn to decipher the symbols and turn the trapped words loose again into speech. The ink of the print trapped the thoughts; they could no more get away than a doomboo could get out of a pit. When the full realization of what this meant flooded over me, I experienced the same thrill and amazement as when I had my first glimpse of the bright lights of Konakry. I shivered with the intensity of my desire to learn to do this wondrous thing myself.

The penumbra asked the shadow: "A little while ago you moved, and now you've stopped. A little while ago you sat down, and now you're standing up. How can you act so irrationally?" The shadow replied: "Do I have to depend on something else to be the way I am? Does what I depend on also have to depend on something else to be what it is? Is my dependence like a snake's on its scales or a cicada's on its wings? How can I know why I am so? How can I know why I am not otherwise?"

When you have to mock a collaborator, avoid using the Mock object directly. Either use mock.create_autospec() or mock.patch(autospec=True) if at all possible. Autospeccing from the real collaborator means that if the collaborator's interface changes, your tests will fail. Manually speccing or not speccing at all means that changes in the collaborator's interface will not break your tests that use the collaborator: you could have 100% test coverage and your library would fall over when used!

I think what you’ll find— I think what you’ll find is, Whatever it is we do substantively, There will be near-perfect clarity As to what it is. And it will be known, And it will be known to the Congress, And it will be known to you, Probably before we decide it, But it will be known.

But I had become aware, even so early as during my college life, that no opinion, however absurd and incredible, can be imagined, which has not been maintained by some on of the philosophers; and afterwards in the course of my travels I remarked that all those whose opinions are decidedly repugnant to ours are not in that account barbarians and savages, but on the contrary that many of these nations make an equally good, if not better, use of their reason than we do.

...It is no surprise that when this model fails, it is the likelihood rather than the prior that is causing the problem. In the binomial model under consideration here, the prior comes into the posterior distribution only once and the likelihood comes in n times. It is perhaps merely an accident of history that skeptics and subjectivists alike strain on the gnat of the prior distribution while swallowing the camel that is the likelihood.

Ian's Acid 3, unlike its predecessors, is not about establishing a baseline of useful web capabilities. It's quite explicitly about making browser developers jump - Ian specifically sought out tests that were broken in WebKit, Opera, and Gecko, perhaps out of a twisted attempt at fairness. But the Acid tests shouldn't be fair to browsers, they should be fair to the web; they should be based on how good the web will be as a platform if all browsers conform, not about how far any given browser has to stretch to get there.

Several libraries let you declare objects with type-hinted members and automatically derive validation rules and serialization/deserialization from the type hints – Pydantic is the most popular, but alternatives like msgspec are out there too. There’s also a whole new generation of web frameworks like FastAPI and Starlite which use type hints at runtime to do not just input validation and serialization/deserialization but also things like dependency injection. Personally, I’ve seen more significant gains in productivity from those runtime usages of Python’s type hints than from any static ahead-of-time type checking, which mostly is only useful to me as documentation.

It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them. Operations of thought are like cavalry charges in a battle—they are strictly limited in number, they require fresh horses, and must only be made at decisive moments...It is interesting to note how important for the development of science a modest-looking symbol may be.

No species is higher or lower in rank than another, it's just that they differ in intelligence and power. They eat each other but that doesn't mean they were produced for each other. People take what they can eat, but did heaven mean it for them? Then since mosquitoes bite skin and tigers eat flesh, heaven made humans for the mosquitoes & their flesh for the tigers!

Ants are a curious race; One crossing with hurried tread The body of one of their dead Isn’t given a moment’s arrest -- Seems not even impressed. But he no doubt reports to any With whom he crosses antennae, And they no doubt report To the higher up at court. Then word goes forth in Formic: "Death’s come to Jerry McCormic, Our selfless forager Jerry. Will the special Janizary Whose office it is to bury The dead of the commissary Go bring him home to his people. Lay him in state on a sepal. Wrap him for shroud in a petal. Embalm him with ichor of nettle. This is the word of your Queen."

Instead of seeing instrumentation as a last-ditch effort of strings and metrics, we must think about propagating the full context of a request and emitting it at regular pulses. No pull request should ever be accepted unless the engineer can answer the question, “How will I know if this breaks?”

The stampede of the affluent into grim-faced, highly competitive sports has been a tragicomedy of perverse incentives and social evolution in unequal times: a Darwinian parable of the mayhem that can ensue following the discovery of even a minor advantage. Like a peacock rendered nearly flightless by gaudy tail feathers, the overserved athlete is the product of a process that has become maladaptive, and is now harming the very blue-chip demographic it was supposed to help.

402. ‘Nothing is so certain as that I possess consciousness.’ In that case, why shouldn’t I let the matter rest? This certainty is like a mighty force whose point of application does not move, and so no work is accomplished by it. 403. Remember: most people say one feels nothing under anesthetic. But some say: It could be that one feels, and simply forgets it completely.

Those masterful images because complete Grew in pure mind, but out of what began? A mound of refuse or the sweepings of a street, Old kettles, old bottles, and a broken can, Old iron, old bones, old rags, that raving slut Who keeps the till. Now that my ladder's gone, I must lie down where all the ladders start In the foul rag and bone shop of the heart.

The problem of the poet is to unite freedom with precision; to give the pleasure of color, and be not less the most powerful of sculptors. Music seems to you sufficient, or the subtle and delicate scent of lavender; but Dante was free imagination,—all wings,—yet he wrote like Euclid. And mark the equality of Shakespeare to the comic, the tender and sweet, and to the grand & terrible. A little more or less skill in whistling is of no account. See those weary pentameter tales of Dryden and others. Turnpike is one thing and blue sky another. Let the poet, of all men, stop with his inspiration.

LLMs are like a trained circus bear that can make you porridge in your kitchen. It's a miracle that it's able to do it at all, but watch out because no matter how well they can act like a human on some tasks, they're still a wild animal. They might ransack your kitchen, and they could kill you, accidentally or intentionally!

We’re planning to release a very capable open language model in the coming months, our first since GPT-2. [...] As models improve, there is more and more demand to run them everywhere. Through conversations with startups and developers, it became clear how important it was to be able to support a spectrum of needs, such as custom fine-tuning for specialized tasks, more tunable latency, running on-prem, or deployments requiring full data control.

Although fine-tuning can feel like the more natural option—training on data is how GPT learned all of its other knowledge, after all—we generally do not recommend it as a way to teach the model knowledge. Fine-tuning is better suited to teaching specialized tasks or styles, and is less reliable for factual recall. [...] In contrast, message inputs are like short-term memory. When you insert knowledge into a message, it's like taking an exam with open notes. With notes in hand, the model is more likely to arrive at correct answers.

Just because Java was once aimed at a set-top box OS that didn't support multiple address spaces, and just because process creation in Windows used to be slow as a dog, doesn't mean that multiple processes (with judicious use of IPC) aren't a much better approach to writing apps for multi-CPU boxes than threads.

The thing nobody talks about with engineering management is this: Every 3-4 months every person experiences some sort of personal crisis. A family member dies, they have a bad illness, they get into an argument with another person at work, etc. etc. Sadly, that is just life. Normally after a month or so things settle down and life goes on. But when you are managing 6+ people it means there is always a crisis you are helping someone work through. You are always carrying a bit of emotional burden or worry around with you.

3. Again, if the campaign is protracted, the resources of the State will not be equal to the strain. 4. Now, when your weapons are dulled, your ardor damped, your strength exhausted and your treasure spent, other chieftains will spring up to take advantage of your extremity. Then no man, however wise, will be able to avert the consequences that must ensue. 5. Thus, though we have heard of stupid haste in war, cleverness has never been seen associated with long delays. ...15. Hence a wise general makes a point of foraging on the enemy. One cartload of the enemy’s provisions is equivalent to twenty of one’s own, and likewise a single picul of his provender is equivalent to twenty from one’s own store.

When I give money to a charitable cause, I always look for the checkboxes to opt out of being contacted by them in the future. When it happens anyway, I get annoyed, and I become reluctant to give to that charity again. [...] When you donate to the Red Cross via Apple, that concern is off the table. Apple won’t emphasize that aspect of this, because they don’t want to throw the Red Cross under the proverbial bus, but I will. An underrated aspect of privacy is the desire simply not to be annoyed.

Back then [in 2012], no one was thinking about AI. You just keep uploading your images [to Adobe Stock] and you get your residuals every month and life goes on — then all of a sudden, you find out that they trained their AI on your images and on everybody’s images that they don’t own. And they’re calling it ‘ethical’ AI.

We have a unique opportunity with phishing and OpenID. OpenID can make the possibility for bad things to happen from phishing that much worse. However, having an OpenID means you create a more intimate relationship with your OpenID provider. You go there everyday. You will more likely know when something is wrong.

You already know Donald Trump. He is unfit to lead. Watch him. Listen to those who know him best. He tried to subvert an election and remains a threat to democracy. He helped overturn Roe, with terrible consequences. Mr. Trump's corruption and lawlessness go beyond elections: It's his whole ethos. He lies without limit. If he's re-elected, the G.O.P. won't restrain him. Mr. Trump will use the government to go after opponents. He will pursue a cruel policy of mass deportations. He will wreak havoc on the poor, the middle class and employers. Another Trump term will damage the climate, shatter alliances and strengthen autocrats. Americans should demand better. Vote.

Software engineers today are about 200-400% more productive than software engineers were 10 years ago because of open source software, better programming tools, common libraries, easier access to information, better education, and other factors. This means that one engineer today can do what 3-5 people did in 1999!

Web apps are typically continuously delivered, not rolled back, and you don't have to support multiple versions of the software running in the wild. This is not the class of software that I had in mind when I wrote the blog post 10 years ago. If your team is doing continuous delivery of software, I would suggest to adopt a much simpler workflow (like GitHub flow) instead of trying to shoehorn git-flow into your team.

The largest model in the PaLM 2 family, PaLM 2-L, is significantly smaller than the largest PaLM model but uses more training compute. Our evaluation results show that PaLM 2 models significantly outperform PaLM on a variety of tasks, including natural language generation, translation, and reasoning. These results suggest that model scaling is not the only way to improve performance. Instead, performance can be unlocked by meticulous data selection and efficient architecture/objectives. Moreover, a smaller but higher quality model significantly improves inference efficiency, reduces serving cost, and enables the model’s downstream application for more applications and users.

The essence of magic is to do away with underlying mechanisms...What makes the elephant disappear is the movement of the wand and the intent of the magician, directly. If there were any intervening processes, it would not be magic but just engineering. As soon as you know how the magician made the elephant disappear, the magic disappears and—if you started by believing in magic—the disappointment sets in.

Today we release OLMo 2 32B, the most capable and largest model in the OLMo 2 family, scaling up the OLMo 2 training recipe used for our 7B and 13B models released in November. It is trained up to 6T tokens and post-trained using Tulu 3.1. OLMo 2 32B is the first fully-open model (all data, code, weights, and details are freely available) to outperform GPT3.5-Turbo and GPT-4o mini on a suite of popular, multi-skill academic benchmarks.

Woe to you, teachers of the law and Pharisees, you hypocrites! You are like whitewashed tombs, which look beautiful on the outside but on the inside are full of dead men's bones and everything unclean. In the same way, on the outside you appear to people as righteous but on the inside you are full of hypocrisy and wickedness.

It is probably more illuminating to go a little bit further back, to the Middle Ages. One of its characteristics was that "reasoning by analogy" was rampant; another characteristic was almost total intellectual stagnation, and we now see why the two go together. A reason for mentioning this is to point out that, by developing a keen ear for unwarranted analogies, one can detect a lot of medieval thinking today.

At this point, I’m confident saying that 75% of what generative-AI text and image platforms can do is useless at best and, at worst, actively harmful. Which means that if AI companies want to onboard the millions of people they need as customers to fund themselves and bring about the great AI revolution, they’ll have to perpetually outrun the millions of pathetic losers hoping to use this tech to make a quick buck. Which is something crypto has never been able to do. In fact, we may have already reached a point where AI images have become synonymous with scams and fraud.

CSRF is not a security issue for the Web. A well-designed Web service should be capable of receiving requests directed by any host, by design, with appropriate authentication where needed. If browsers create a security issue because they allow scripts to automatically direct requests with stored security credentials onto third-party sites, without any user intervention/configuration, then the obvious fix is within the browser.

Don't ask me those questions! Don't ask me what life means or how we know reality or why we have to suffer so much. Don't talk about how nothing feels real, how everything is coated with gelatin and shining like oil in the sun. I don't want to hear about the tiger in the corner or the Angel of Death or the phone calls from John the Baptist.

My main concern is that the substantial cost to develop and run Al technology means that Al applications must solve extremely complex and important problems for enterprises to earn an appropriate return on investment. We estimate that the Al infrastructure buildout will cost over $1tn in the next several years alone, which includes spending on data centers, utilities, and applications. So, the crucial question is: What $1tn problem will Al solve? Replacing low-wage jobs with tremendously costly technology is basically the polar opposite of the prior technology transitions I've witnessed in my thirty years of closely following the tech industry.

“I . . . I live my life . . . in a whirlwind!” ...“Actors are granted license to live a thousand lives, Adalric; but you, you chose to live a thousand lies. If you have come to me as your advocate then speak from your heart, not from the void of a tragic character who has never been born.”

Who can live like an idealist living alone, drinking only dew and wind? Those who have only a bowl of barley with bean stew think of rice as Heaven— they bow down in front of rice. ...Do you know that in a shadowed corner of this land a hungry mouth lives asking for a spoonful of rice? Poverty is not by any means merely tattered rags. It's not just the old dress that one puts on and takes off. When life gets swept up in the rough waves, it isn't a pleasure to lazily watch the green mountain in the afternoon. Poverty is the enemy, the poisonous worm that gobbles us up and feasts upon even our natural character, the toxin that rots not just clothes but the flesh, too. It's our human enemy, the devil to drive away, the seeker of pleasure in poverty who hopelessly nurtures worms in the growling belly. ...You say it's all right, it's all right, borrowing Tao Yuan-min's drinking cup, imitating Li Bai's drunken rowdiness. Don't deceive yourself. Don't defile the hungry mouth who wants a bowl of rice and bean soup, trading poverty for a piece of poem. Oh, the hypocrite poet, the poet of lullabies who puts people to sleep!

They danced down the streets like dingledodies, & I shambled after as I've been doing all my life after people who interest me, because the only people for me are the mad ones, the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones that never yawn or say a commonplace thing, but burn, burn, burn...

Nego consequentiam! I don’t see at all why I should have respect for lies and frauds because other people are stupid. I respect truth everywhere, and it is precisely for that reason that I cannot respect anything that is opposed to it. My maxim is, Vigeat veritas, et pereat mundus, the same as the lawyer’s Fiat justitia, et pereat mundus. Every profession ought to have an analogous device.

Interleaved thinking is essential for LLM agents: it means alternating between explicit reasoning and tool use, while carrying that reasoning forward between steps.This process significantly enhances planning, self‑correction, and reliability in long workflows. [...] From community feedback, we've often observed failures to preserve prior-round thinking state across multi-turn interactions with M2. The root cause is that the widely-used OpenAI Chat Completion API does not support passing reasoning content back in subsequent requests. Although the Anthropic API natively supports this capability, the community has provided less support for models beyond Claude, and many applications still omit passing back the previous turns' thinking in their Anthropic API implementations. This situation has resulted in poor support for Interleaved Thinking for new models. To fully unlock M2's capabilities, preserving the reasoning process across multi-turn interactions is essential.

We like to assume that automation technology will maintain or increase wage levels for a few skilled supervisors. But in the long-term skilled automation supervisors also tend to earn less. Here's an example: In 1801 the Jacquard loom was invented, which automated silkweaving with punchcards. Around 1800, a manual weaver could earn 30 shillings/week. By the 1830s the same weaver would only earn around 5s/week. A Jacquard operator earned 15s/week, but he was also 12x more productive. The Jacquard operator upskilled and became an automation supervisor, but their wage still dropped. For manual weavers the wages dropped even more. If we believe assistive AI will deliver unseen productivity gains, we can assume that wage erosion will also be unprecedented.

If you’re new to tech, taking [career] advice on what works for someone with a 20-year career is likely to be about as effective as taking career advice from a stockbroker or firefighter or nurse. There’ll be a few things that generalize, but most advice won’t. Further, even advice people with long careers on what worked for them when they were getting started is unlikely to be advice that works today. The tech industry of 15 or 20 years ago was, again, dramatically different from tech today.

A common misconception about Transformers is to believe that they're a sequence-processing architecture. They're not. They're a set-processing architecture. Transformers are 100% order-agnostic (which was the big innovation compared to RNNs, back in late 2016 -- you compute the full matrix of pairwise token interactions instead of processing one token at a time). The way you add order awareness in a Transformer is at the feature level. You literally add to your token embeddings a position embedding / encoding that corresponds to its place in a sequence. The architecture itself just treats the input tokens as a set.

In mid-March, we added this line to our system prompt to prevent Claude from thinking it can open URLs: `It cannot open URLs, links, or videos, so if it seems as though the interlocutor is expecting Claude to do so, it clarifies the situation and asks the human to paste the relevant text or image content directly into the conversation.`

It's a bit sad and confusing that LLMs ("Large Language Models") have little to do with language; It's just historical. They are highly general purpose technology for statistical modeling of token streams. A better name would be Autoregressive Transformers or something. They don't care if the tokens happen to represent little text chunks. It could just as well be little image patches, audio chunks, action choices, molecules, or whatever. If you can reduce your problem to that of modeling token streams (for any arbitrary vocabulary of some set of discrete tokens), you can "throw an LLM at it".

Did you ever wonder why the 21st century feels like we're living in a bad cyberpunk novel from the 1980s? It's because these guys read those cyberpunk novels and mistook a dystopia for a road map. They're rich enough to bend reality to reflect their desires. But we're [sci-fi authors] not futurists, we're entertainers! We like to spin yarns about the Torment Nexus because it's a cool setting for a noir detective story, not because we think Mark Zuckerberg or Andreesen Horowitz should actually pump several billion dollars into creating it.

OMNI: Will robots be complex enough to be friends of people, do you think? Shannon: I think so. But it’s quite a way away. OMNI: Could you imagine being friends with a robot? Shannon: Yes I could. I could imagine that happening very easily. I see no limit to the capability of machines. The microchips are getting smaller and smaller and faster and faster and I can see them getting better than we are. I can visualize sometime in the future we will be to robots as dogs are to humans.

When you’re pump­ing mes­sages around the In­ter­net be­tween het­ero­ge­neous code­bas­es built by peo­ple who don’t know each oth­er, shit is gonna hap­pen. That’s the whole ba­sis of the We­b: You can safe­ly ig­nore an HTTP head­er or HTML tag you don’t un­der­stand, and noth­ing break­s. It’s great be­cause it al­lows peo­ple to just try stuff out, and the use­ful stuff catch­es on while the bad ideas don’t break any­thing.

Inevitably we got round to talking about async. As much of an unneeded complication as it is for so many day-to-day use-cases, it’s important for Python because, if and when you do need the high throughput handling of these io-bound use-cases, you don’t want to have to switch language. The same for Django: most of what you’re doing has no need of async but you don’t want to have to change web framework just because you need a sprinkling of non-blocking IO.

Abashed the Devil stood, And felt how awful goodness is, and saw Virtue in her shape how lovely; saw, and pined His loss; but chiefly to find here observed His lustre visibly impaired; yet seemed Undaunted. "If I must contend", said he, "Best with the best, the sender, not the sent, Or all at once; more glory will be won, Or less be lost."

Providing validation, strength, and stability to people who feel gaslit and dismissed and forgotten can help them feel stronger and surer in their decisions. These pieces made me understand that journalism can be a caretaking profession, even if it is never really thought about in those terms. It is often framed in terms of antagonism. Speaking truth to power turns into being hard-nosed and removed from our subject matter, which so easily turns into be an asshole and do whatever you like. This is a viewpoint that I reject. My pillars are empathy, curiosity, and kindness. And much else flows from that. For people who feel lost and alone, we get to say through our work, you are not. For people who feel like society has abandoned them and their lives do not matter, we get to say, actually, they fucking do. We are one of the only professions that can do that through our work and that can do that at scale.

A parasol twirled Under midsummer’s rays Men’s eyes towards the clouds And yours toward the ground ...Embroideries and golds Insatiable desire Incenses at moonlight In Fuji’s twilight shadow Glances cloaked in silk A fan, a kimono Eyes and lips open Heart steeled Death Death to your son’s cousin Death Death to all opposition Smiling Half-pursed lips Stifling a giggle The glee of a young girl...

I think of that old woman in the song Who could not know herself without the skirt They cut off while she slept beside a stile. Her dog jumped at the unaccustomed legs And barked till she turned slowly from her gate And went—I never asked them where she went. ...I know now she went nowhere; went to wait In the bare night of the fields, to whisper: "I’ll sit and wish that it was never so." I see her sitting on the ground and wishing, The wind jumps like a dog against her legs, And she keeps thinking: "This is all a dream." ...The first night I looked into the mirror And saw the room empty, I could not believe That it was possible to keep existing In such pain: I have existed. Was the old woman dead? What does it matter? —Am I dead? A ghost, a real ghost Has no need to die: what is he except A being without access to the universe That he has not yet managed to forget?

Over the years, across multiple deployments, DynamoDB has learned that it’s not just the end state and the start state that matter; there could be times when the newly deployed software doesn’t work and needs a rollback. The rolled-back state might be different from the initial state of the software. The rollback procedure is often missed in testing and can lead to customer impact. DynamoDB runs a suite of upgrade and downgrade tests at a component level before every deployment. Then, the software is rolled back on purpose and tested by running functional tests. DynamoDB has found this process valuable for catching issues that otherwise would make it hard to rollback if needed.

The talk of believing by our volition seems, then, from one point of view, simply silly. From another point of view it is worse than silly, it is vile. When one turns to the magnificent edifice of the physical sciences, and sees how it was reared;...what submission to the icy laws of outer fact are wrought into its very stones and mortar; how absolutely impersonal it stands in its vast augustness—then how besotted and contemptible seems every little sentimentalist who comes blowing his voluntary smoke-wreaths, and pretending to decide things from out of his private dream! Can we wonder if those bred in the rugged and manly school of science should feel like spewing such subjectivism out of their mouths?...It is only natural that those who have caught the scientific fever should pass over to the opposite extreme.

Consumer products have had growth hackers for many years optimizing every part of the onboarding funnel. Dev tools should do the same. Getting started shouldn't be an afterthought after you built the product. Getting started is the product! And I mean this to the point where I think it's worth restructuring your entire product to enable fast onboarding. Get rid of mandatory config. Make it absurdly easy to set up API tokens. Remove all the friction. Make it possible for users to use your product on their laptop in a couple of minutes, tops.

When I said “Send a text message to Julian Chokkattu,” who’s a friend and fellow AI Pin reviewer over at Wired, I thought I’d be asked what I wanted to tell him. Instead, the device simply said OK and told me it sent the words “Hey Julian, just checking in. How's your day going?” to Chokkattu. I've never said anything like that to him in our years of friendship, but I guess technically the AI Pin did do what I asked.

An LLM knows every work of Shakespeare but can’t say which it read first. In this material sense a model hasn’t read at all. To read is to think. Only at inference is there space for serendipitous inspiration, which is why LLMs have so little of it to show for all they’ve seen.

Anything can be an instrument, Chigurh said. Small things. Things you wouldn’t even notice. They pass from hand to hand. People don’t pay attention. And then one day there’s an accounting. And after that nothing is the same. Well, you say. It’s just a coin. For instance. Nothing special there. What could that be an instrument of? You see the problem. To separate the act from the thing. As if the parts of some moment in history might be interchangeable with the parts of some other moment. How could that be? Well, it’s just a coin. Yes. That’s true. Is it? ...What’s the most you ever lost on a coin toss?

Almost in the same way as earlier physicists are said to have found suddenly that they had too little mathematical understanding to be able to master physics; we may say that young people today are suddenly in the position that ordinary common sense no longer suffices to meet the strange demands life makes. Everything has become so intricate that for its mastery an exceptional degree of understanding is required. For it is not enough any longer to be able to play the game well; but the question is again and again: what sort of game is to be played now anyway?

This spending of the best part of one’s life earning money in order to enjoy a questionable liberty during the least valuable part of it reminds me of the Englishman who went to India to make a fortune first, in order that he might return to England and live the life of a poet. He should have gone up the garret at once.

Inside the labs we have these capable models, and they're not that far ahead from what the public has access to for free. And that's a completely different trajectory for bringing technology into the world that what we've seen historically. It's a great opportunity because it brings people along. It gives them intuitive sense for the capabilities and risks and allows people to prepare for the advent of bringing advanced AI into the world.

It is a sound maxim, and one which all close thinkers have felt, but which no one before Bentham ever so consistently applied, that error lurks in generalities: that the human mind is not capable of embracing a complex whole, until it has surveyed and catalogued the parts of which that whole is made up; that abstractions are not realities per se, but an abridged mode of expressing facts, and that the only practical mode of dealing with them is to trace them back to the facts (whether of experience or of consciousness) of which they are the expression. Proceeding on this principle, Bentham makes short work with the ordinary modes of moral and political reasoning. These, it appeared to him, when hunted to their source, for the most part terminated in phrases. In politics, liberty, social order, constitution, law of nature, social compact, &c., were the catch-words: ethics had its analogous ones. Such were the arguments on which the gravest questions of morality and policy were made to turn; not reasons, but allusions to reasons; sacramental expressions, by which a summary appeal was made to some general sentiment of mankind, or to some maxim in familiar use, which might be true or not, but the limitations of which no one had ever critically examined.

As Borges himself showed us in so many stories—"The Aleph", "The Garden of Forking Paths", "The Gift", "Blue Tigers", "Shakespeare’s Memory"—a blessing is always a mixed blessing. As Borges noted sadly, he inherited a library, and blindness; we who study Borges inherit great sight, yet the rest of the library somehow fades.

Create a culture that favors begging forgiveness (and reversing decisions quickly) rather than asking permission. Invest in infrastructure such as progressive / cancellable rollouts. Use asynchronous written docs to get people aligned (“comment in this doc by Friday if you disagree with the plan”) rather than meetings (“we’ll get approval at the next weekly review meeting”).

“Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to”, said the Cat. “I don’t much care where—” said Alice. “Then it doesn’t matter which way you go”, said the Cat. “—so long as I get somewhere”, Alice added as an explanation. “Oh, you’re sure to do that”, said the Cat, “if you only walk long enough.”

For most software engineers, being well rounded is more important than pure technical mastery. This was already true, of course — see @patio11's famous advice "Don't call yourself a programmer" — but even more so due to foundation models. In most situations, skills like being able to use AI to rapidly prototype in order to communicate with clients to iterate on specifications create far more business value than technical wizardry alone.

It is easy enough to say that man is immortal simply because he will endure: that when the last ding-dong of doom has clanged and faded from the last worthless rock hanging tideless in the last red and dying evening, that even then there will still be one more sound: that of his puny inexhaustible voice, still talking. I refuse to accept this. I believe that man will not merely endure: he will prevail.

The rapid increase of COVID-19 cases among kids has shattered last year’s oft-repeated falsehood that kids don’t get COVID-19, and if they do, it’s not that bad. It was a convenient lie that was easy to believe in part because we kept most of our kids home. With remote learning not an option now, this year we’ll find out how dangerous this virus is for children in the worst way possible.

This [technological] pattern has happened over and over and over again. Someone built the system, they assumed certain user behaviors. The users came on and exhibited different behaviors. And the people running the system discovered to their horror that the technological and social issues could not in fact be decoupled.

Basically, a frontier model like OpenAI’s O1 is like a Ferrari SF-23. It’s an obvious triumph of engineering, designed to win races, and that’s why we talk about it. But it takes a special pit crew just to change the tires and you can’t buy one for yourself. In contrast, a BERT model is like a Honda Civic. It’s also an engineering triumph, but more subtly, since it is engineered to be affordable, fuel-efficient, reliable, and extremely useful. And that’s why they’re absolutely everywhere.

Normally, those people would never wake up from their fantasy worlds. They live meaningless lives. They waste their precious days over nothing. No matter how old they get, they'll continue to say, "My real life hasn't started yet. The real me is still asleep, so that's why my life is such garbage." They continue to tell themselves that. They continue. And they age. Then die. And on their deathbeds, they will finally realize: the life they lived was the real thing. People don't live provisional lives, nor do they die provisional deaths. That's a simple fact! The problem... is whether they realize that simple fact.

Education is a technology that tries to make up for what the human mind is innately bad at. Children don’t have to go to school to learn how to walk, talk, recognize objects, or remember the personalities of their friends, even though these tasks are much harder than reading, adding, or remembering dates in history. They do have to go to school to learn written language, arithmetic, and science, because those bodies of knowledge and skill were invented too recently for any species-wide knack for them to have evolved.

"I see by your outfit you may be a preacher." "Yes, I am,—of the non-theistic, non-sectarian sort." ..."A totally nondenominational prayer: Insofar as I may be heard by anything, which may or may not care what I say, I ask, if it matters, that you be forgiven for anything you may have done or failed to do which requires forgiveness. Conversely, if not forgiveness but something else may be required to insure any possible benefit for which you may be eligible after the destruction of your body, I ask that this, whatever it may be, be granted or withheld, as the case may be, in such a manner as to insure your receiving said benefit. I ask this in my capacity as your elected intermediary between yourself and that which may not be yourself, but which may have an interest in the matter of your receiving as much as is possible for you to receive of this thing, and which may in some way be influenced by this ceremony. Amen."

You might not know this old tree by its bark, Which once was striate, smooth, and glossy-dark, So deep now are the rifts that separate Its roughened surface into flake and plate.Fancy might less remind you of a birch Than of mosaic columns in a church Like Ara Coeli or the Lateran Or the trenched features of an agèd man.Still, do not be too much persuaded by These knotty furrows and these tesserae To think of patterns made from outside in Or finished wisdom in a shriveled skin.Old trees are doomed to annual rebirth, New wood, new life, new compass, greater girth, And this is all their wisdom and their art— To grow, stretch, crack, and not yet come apart.

There is a big difference between tech as augmentation versus automation. Augmentation (think Excel and accountants) benefits workers while automation (think traffic lights versus traffic wardens) benefits capital. LLMs are controversial because the tech is best at augmentation but is being sold by lots of vendors as automation.

A curious feature of the coming millennium is how little speculation it has prompted. I remember in the 1960s being rung up by journalists asking, "What will sex be like in the seventies?" They expected something strange and unimaginable, but looking back after 20 years it all seemed much the same. A general rule: if enough people predict something, it won’t happen.

It is a good rule, after reading a new book, never to allow yourself another new one till you have read an old one in between. If that is too much for you, you should at least read one old one to every three new ones. Every age has its own outlook. It is specially good at seeing certain truths and specially liable to make certain mistakes. We all, therefore, need the books that will correct the characteristic mistakes of our own period. And that means the old books...Nothing strikes me more when I read the controversies of past ages than the fact that both sides were usually assuming without question a good deal which we should now absolutely deny. They thought that they were as completely opposed as two sides could be, but in fact they were all the time secretly united—united with each other and against earlier and later ages—by a great mass of common assumptions...None of us can fully escape this blindness, but we shall certainly increase it, and weaken our guard against it, if we read only modern books.

One of the characteristics of successful scientists is having courage. Once you get your courage up and believe that you can do important problems, then you can. If you think you can't, almost surely you are not going to. Courage is one of the things that Shannon had supremely. You have only to think of his major theorem. He wants to create a method of coding, but he doesn't know what to do so he makes a random code. Then he is stuck. And then he asks the impossible question, "What would the average random code do?" He then proves that the average code is arbitrarily good, and that therefore there must be at least one good code. Who but a man of infinite courage could have dared to think those thoughts? That is the characteristic of great scientists; they have courage. They will go forward under incredible circumstances; they think and continue to think.

4chan's /b/ forum, which gets called things like the Mos Eisley spaceport of the web when people are being polite, and the asshole of the internet when they aren't, is energetic, anarchic, barely moderated, crude, irresponsible, vindictive if crossed, peculiarly creative, and full of hackers. It inspires loyalty in its core users, and makes everyone else nervous.

VII. ...The spray rainbows over the sloping lawns. With short jerks a robin runs up, stands motionless. The eucalyptus tree trunks glow in the light. The oaks perfect the shadow of May leaves. Only this. Only this is worth of praise: the day. ...VIII. And what if Pascal had not been saved and if those narrow hands in which we laid a cross are just he, entire, like a lifeless swallow in the dust, under the buzz of the poisonous-blue flies? And if they all, kneeling with poised palms, millions, billions of them, ended together with their illusion? I shall never agree. I will give them the crown. The human mind is splendid; lips, powerful, and the summons, so great, it must open Paradise. IX. They are so persistent, that give them a few stones and edible roots, and they will build the world.

Microsoft was slowing development of new versions of Internet Explorer in the hope that Web-based applications would not be able to compete with Windows applications, and Windows applications would keep people locked in to the Windows operating system. Thus XHTML2 was developed with no expectation that the leading Web browser would ever implement it.

15. Everything should be built top-down, except the first time.…\&30. In programming, everything we do is a special case of something more general—and often we know it too quickly.…\&31. Simplicity does not precede complexity, but follows it.…\&58. Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it.…\&65. Make no mistake about it: Computers process numbers—not symbols. We measure our understanding (and control) by the extent to which we can arithmetize an activity.…\&56. Software is under a constant tension. Being symbolic it is arbitrarily perfectible; but also it is arbitrarily changeable.

Getting agents using Beads requires much less prompting, because Beads now has 4 months of “Desire Paths” design, which I’ve talked about before. Beads has evolved a very complex command-line interface, with 100+ subcommands, each with many sub-subcommands, aliases, alternate syntaxes, and other affordances. The complicated Beads CLI isn’t for humans; it’s for agents. What I did was make their hallucinations real, over and over, by implementing whatever I saw the agents trying to do with Beads, until nearly every guess by an agent is now correct.

Much like an oral agreement, publishing microformats is an informal agreement between you and (hopefully) a developer community that sets up a relationship with plenty of vagueness, inertial resistance to change, and potential landmines to step on. Would you create a real developer API without a TOS, agreement, or at the very least, guidelines? [...] are you prepared to announce all frontend markup changes? Does publishing a microformat without a special agreement mean that you are implicitly allowing comprehensive scraping of your web data?

This is the difference between Data and a large language model, at least the ones operating right now. Data created art because he wanted to grow. He wanted to become something. He wanted to understand. Art is the means by which we become what we want to be. [...] The book, the painting, the film script is not the only art. It's important, but in a way it's a receipt. It's a diploma. The book you write, the painting you create, the music you compose is important and artistic, but it's also a mark of proof that you have done the work to learn, because in the end of it all, you are the art. The most important change made by an artistic endeavor is the change it makes in you. The most important emotions are the ones you feel when writing that story and holding the completed work. I don't care if the AI can create something that is better than what we can create, because it cannot be changed by that creation.

The problem of grues is, of course, their recursive nature. To wit: A) Grues are found wherever it is very dark. B) There are no light sources on the inside of a grue. Therefore, being eaten by a grue is a fate which entails being eaten by an infinite number of progressively smaller grues, presumably nested in a geometrically complicated and interesting way.

We can deploy new versions of our software, make database schema changes, or even rotate our primary database server, all without failing to respond to a single request. We can accomplish this because we gave ourselves the ability suspend our traffic, which gives us a window of a few seconds to make some changes before letting the requests through. To make this happen, we built a custom HTTP server and application dispatching infrastructure around Python’s Tornado and Redis.

A murmur of rain had started again. He lay there in the abyss of his thoughts as her breathing beside him steadied and deepened. Almost a voice stirred in him. It starts before Hanford, it almost said. It starts with Röntgen⁠, with the piece of barium glowing in the path of invisible rays, striking out the fire that God had put there. It starts with his wife’s hand on the photographic plate, its transparence there, the ashen bones visible within the milky flesh. Who could imagine that this radiance at the heart of matter could be malign? That with its light came fire? (Yet from the first the ashen bones were there to see within the flesh.) It starts with Becquerel carrying the radium in his pocket that burned his skin, and darkened the unexposed film. It starts with Marie Curie poisoning herself in that pale uncanny glow. With Rutherford guessing at this new alchemy, guessing that matter, giving up its glow, transformed itself one element into another. With the miners at Joachimsthal⁠, deep under the Erzgebirge⁠, inhaling the dust of uranium and dying of “mountain sickness”⁠. With women who by the thousands in watch factories tipped their brushes with that glow⁠⁠⁠, touched it to their tongues before painting the dial face, women who only much later, when the watches’ glow had faded, sickened and died from that radiance taken into their bones. It begins with Ernest Lawrence rushing across the Berkeley campus, the idea of a proton accelerator uncontainable in his mind, calling out, I’m going to be famous! With Oppenheimer at Jornada del Muerte that morning of Trinity. With the scientists who had prised open the gates to that blazing realm past heaven or hell. What were they now at the Lab in all their thousands, but the colonial bureaucrats of that realm, the followers and functionaries, the clerks and commissars? Mere gatekeepers of that power. Or in its keeping. It goes of its own momentum beyond Hanford, to Trinity, to Hiroshima, to the prisoners, the cancer patients, the retarded children, the pregnant women injected or fed this goblin matter to see would it bring health or sickness, the soldiers huddled in trenches against the flash, bones visible in their arms through closed eyes, staring up at the roiling cloudrise, the sheepherders, the farms, the homes, the gardens downwind. And in his sleep the voice long stilled spoke once more. It starts with Sforza; in case of need I will make bombards, mortars, and firethrowing engines of beautiful and practical design. It starts with Archimedes focusing the sun’s rays upon the fleet at Syracuse, it starts with the first rock hurled by the first grasping hand. It starts where we start. It is mind, it is hunger, it is greed, it is defense, it is mischief, it is the devil, it is the god; it is life.

"The people who wrote the medieval ballads", answered the priest, "knew more about fairies than you do. It isn’t only nice things that happen in fairyland." "Oh, bosh!" said Flambeau. "Only nice things could happen under such an innocent moon. I am for pushing on now and seeing what does really come. We may die and rot before we ever see again such a moon or such a mood." "All right", said Father Brown. "I never said it was always wrong to enter fairyland. I only said it was always dangerous."

Vigorous writing is concise. A sentence should contain no unnecessary words [and] a paragraph no unnecessary sentences, for the same reason that a drawing should have no unnecessary lines and a machine no unnecessary parts. This requires not that the writer make all his sentences short, or that he avoid all detail and treat his subjects only in outline, but that every word tell.

Every few weeks, someone on Twitter notices how demented the content on Facebook is. I’ve covered a lot of these stories. The quick TL;DR is that Facebook’s video section is essentially run by a network of magicians and Vegas stage performers who hack the platform’s algorithm with surreal low-value content designed to distract users long enough to trigger an in-video advertisement and anger them enough to leave a comment.

...we think so much reversal is based on "We think something should work, and so we’re going to adopt it before we know that it actually does work", and one of the reasons for this is because that’s how medical education is structured. We learn the biochemistry, the physiology, the pathophysiology as the very first things in medical school. And over the first two years we kind of get convinced that everything works mechanistically the way we think it does.

A King may move a man, a father may claim a son, but remember that even when those who move you be Kings, or men of power, your soul is in your keeping alone. When you stand before God, you cannot say, "But I was told by others to do thus." Or that, "Virtue was not convenient at the time." This will not suffice. Remember that.

If you work slowly, you will be more likely to stick with your slightly obsolete work. You know that professor who spent seven years preparing lecture notes twenty years ago? He is not going to throw them away and start again, as that would be a new seven-year project. So he will keep teaching using aging lecture notes until he retires and someone finally updates the course.

While we encourage people to use AI systems during their role to help them work faster and more effectively, please do not use AI assistants during the application process. We want to understand your personal interest in Anthropic without mediation through an AI system, and we also want to evaluate your non-AI-assisted communication skills. Please indicate 'Yes' if you have read and agree. Why do you want to work at Anthropic? (We value this response highly - great answers are often 200-400 words.)

I am increasingly worried about AI in the video game space in general. [...] I'm not sure that the CEOs and the people making the decisions at these sorts of companies understand the difference between actual content and slop. [...] It's exactly the same cryolab, it's exactly the same robot factory place on all of these different planets. It's like there's so much to explore and nothing to find. [...] And what was in this contraband chest was a bunch of harvested organs. And I'm like, oh, wow. If this was an actual game that people cared about the making of, this would be something interesting - an interesting bit of environmental storytelling. [...] But it's not, because it's just a cold, heartless, procedurally generated slop. [...] Like, the point of having a giant open world to explore isn't the size of the world or the amount of stuff in it. It's that all of that stuff, however much there is, was made by someone for a reason.

Most researchers don’t share their data. If you’ve ever read the words “data is available upon request" in an academic paper, and emailed the authors to request it, the chances that you'll actually receive the data are just 7 percent. The rest of the time, the authors have lost access to their data, changed emails, or are too busy or unwilling.

Because you’re allowed to do something doesn’t mean you can do it without repercussions. In this case, the consequences are very much on the mild side: if you use LLMs or diffusion models, a relatively small group of mostly mid- to low-income people who are largely underdogs in their respective fields will think you’re a dick.

Most people are theists not because they were 'reasoned into' believing in God, but because they applied Occam's razor at too early an age. Their simplest explanation for the reason that their parents, not to mention everyone else in the world, believed in God, was that God actually existed. The same could be said for, say, Australia.

After struggling for years trying to figure out why people think [Cloudflare] Durable Objects are complicated, I'm increasingly convinced that it's just that they sound complicated. Feels like we can solve 90% of it by renaming `DurableObject` to `StatefulWorker`? It's just a worker that has state. And because it has state, it also has to have a name, so that you can route to the specific worker that has the state you care about. There may be a sqlite database attached, there may be a container attached. Those are just part of the state.

The secret of Tokyu Hands is that everything on offer there inclines, ultimately, to the status, if not the perfection, of hikaru dorodango. The brogues, shined lovingly enough, for long enough, with those meticulously imported shoe-care products, must ultimately become a universe unto themselves, a conceptual sphere of lustrous and infinite depth. Just as a life, lived silently enough, in sufficient solitude, becomes a different sort of sphere, no less perfect.

“Nature” is a term that may lend itself to any ethic and any theology; it fits the science of Darwin and the unmorality of Nietzsche more snugly than the sweet reasonableness of Lao-Tze and Christ. If one follows nature and acts naturally he is much more likely to murder and eat his enemies than to practice philosophy..And yet, there is something medicinal in this philosophy; we suspect that we, too, when our fires begin to burn low, shall see wisdom in it, and shall want the healing peace of uncrowded mountains and spacious fields. Life oscillates between Voltaire and Rousseau, Confucius and Lao-Tze, Socrates and Christ. After every idea has had its day with us and we have fought for it not wisely or too well, we in our turn shall tire of the battle, and pass on to the young our thinning fascicle of ideals. Then we shall take to the woods with Jacques, Jean-Jacques, and Lao-Tze; we shall make friends of the animals, and discourse more contentedly than Machiavelli with simple peasant minds; we shall leave the world to stew in its own deviltry, and shall take no further thought of its reform. Perhaps we shall burn every book but one behind us, and find a summary of wisdom in the Tao-Te-Ching.

...For I will consider my Cat Jeoffry. ...For he is the cleanest in the use of his forepaws of any quadruped. For the dexterity of his defence is an instance of the love of God to him exceedingly. For he is the quickest to his mark of any creature. For he is tenacious of his point. For he is a mixture of gravity and waggery. For he knows that God is his Saviour. For there is nothing sweeter than his peace when at rest. For there is nothing brisker than his life when in motion. For he is of the Lord’s poor and so indeed is he called by benevolence perpetually—Poor Jeoffry! poor Jeoffry! the rat has bit thy throat. ...For he can jump from an eminence into his master’s bosom. For he can catch the cork and toss it again. ...For he is good to think on, if a man would express himself neatly. For he made a great figure in Egypt for his signal services. For he killed the Ichneumon-rat very pernicious by land. For his ears are so acute that they sting again. For from this proceeds the passing quickness of his attention. For by stroking of him I have found out electricity. For I perceived God’s light about him both wax and fire. For the Electrical fire is the spiritual substance, which God sends from heaven to sustain the bodies both of man and beast. For God has blessed him in the variety of his movements. For, tho he cannot fly, he is an excellent clamberer. For his motions upon the face of the earth are more than any other quadruped. For he can tread to all the measures upon the music. For he can swim for life. For he can creep.

Q: Why does the devil keep his deals? A: Because as an immortal he has an infinite time horizon of other deals he jeopardizes if he betrays any given deal. Therefore the opportunity cost of any betrayal is too high. Q: What does that make politicians then? A: Lower in ethical reliability than the devil.

One of the core constitutional principles that guides our AI model development is privacy. We do not train our generative models on user-submitted data unless a user gives us explicit permission to do so. To date we have not used any customer or user-submitted data to train our generative models.

What public man—what statesman projecting a coup—what king determined on an invasion of his neighbour—what satirist meditating an onslaught on society or an individual, can't give a pretext for his move? There was a French general the other day who proposed to march into this country and put it to sack and pillage, in revenge for humanity outraged by our conduct at Copenhagen: there is always some excuse for men of the aggressive turn. They are of their nature warlike, predatory, eager for fight, plunder, dominion.

This tutorial exists because of a particular quirk of mine: I love to write tutorials about things as I learn them. This is the backstory of TRPL, of which an ancient draft was "Rust for Rubyists." You only get to look at a problem as a beginner once, and so I think writing this stuff down is interesting. It also helps me clarify what I'm learning to myself.

I confess I love littleness almost in all things. A little convenient estate, a little cheerful house, a little company, and a very little feast; and, if I were to fall in love again (which is a great passion, and therefore, I hope, I have done with it), it would be, I think, with prettiness, rather than with majestical beauty.

There’s a funny irony in “tell your story” and “speak your truth”, in that those two things are fundamentally at odds with each other. Stories and narratives aren’t, and can’t be, the truth of our actual lived experiences. Real lives don’t follow the structures of narrative, they don’t move in linear tidy sequences of causes and actions and effects and consequences. Real lives are big jumbled messes that are almost impossible to make real sense of, and the act of imposing a narrative on them, sorting out our “life story”, is always an act of editing.

Goddammit. The Onion once again posted an article in which a portion of the artwork came from an AI-generated Shutterstock image. This article was over a month old and only a portion of the image. We took it down immediately. [...] To be clear, The Onion has a several-person art team and they work their asses off. Sometimes they work off of stock photo bases and go from there. That's what happened this time. This was not a problem until stock photo services became flooded with AI slop. We'll reinforce process and move on.

Here are some absurdly expensive things you can do on a trip to Tokyo: Buy a golden toilet. There is a toilet in Tokyo that is made of gold and costs around 10 million yen. If you are looking for a truly absurd experience, you can buy this toilet and use it for your next bowel movement. [...]

...in an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it...A relatively straightforward way of measuring how much scarce resources a message consumes is by noting how much time the recipients spends on it.

Look at Sony, or Microsoft, or Google, or anyone. They still don't get it. They're still out there talking about chips, or features, or whatever. Or now they're all hot for design. But they think design means making pretty objects. It doesn't. It means making a system of pieces that all work together seamlessly. It's not about calling attention to the technology. It's about making the technology invisible.