Quotes

The open secret Jennings filled me in on is that OpenStreetMap (OSM) is now at the center of an unholy alliance of the world’s largest and wealthiest technology companies. The most valuable companies in the world are treating OSM as critical infrastructure for some of the most-used software ever written. The four companies in the inner circle— Facebook, Apple, Amazon, and Microsoft— have a combined market capitalization of over six trillion dollars.

There’s been a lot of strange reporting recently about how ‘scaling is hitting a wall’ – in a very narrow sense this is true in that larger models were getting less score improvement on challenging benchmarks than their predecessors, but in a larger sense this is false – techniques like those which power O3 means scaling is continuing (and if anything the curve has steepened), you just now need to account for scaling both within the training of the model and in the compute you spend on it once trained.

Ananda: When will the perfume be digested? Vimalakirti: When the fields stop steaming, stop giving and needing, this perfume will be digested. When we have laid our hands on the body of a loved one and felt nothing but our own heat, this perfume will fade away, like fire sculpting itself into nothing.

...the large, highly evolved sensory and motor portions of the brain seem to be the hidden powerhouse behind human thought. By virtue of the great efficiency of these billion-year-old structures, they may embody one million times the effective computational power of the conscious part of our minds. While novice performance can be achieved using conscious thought alone, master-level expertise draws on the enormous hidden resources of these old and specialized areas. Sometimes some of that power can be harnessed by finding and developing a useful mapping between the problem and a sensory intuition.

Ah! let not Censure term our fate our choice, The stage but echoes back the public’s voice; The drama’s laws the drama’s patrons give, For we that live to please must please to live. ...prompt no more the follies you decry, As tyrants doom their tools of guilt to die; ‘Tis yours this night to bid the reign commence Of rescu’d Nature, and reviving Sense; ...Bid scenic Virtue form the rising age, And Truth diffuse her radiance from the stage.

I teach HS Science in the south. I can only speak for my district, but a few teacher work days in the wave of enthusiasm I'm seeing for AI tools is overwhelming. We're getting district approved ads for AI tools by email, Admin and ICs are pushing it on us, and at least half of the teaching staff seems all in at this point. I was just in a meeting with my team and one of the older teachers brought out a powerpoint for our first lesson and almost everyone agreed to use it after a quick scan - but it was missing important tested material, repetitive, and just totally airy and meaningless. Just slide after slide of the same handful of sentences rephrased with random loosely related stock photos. When I asked him if it was AI generated, he said 'of course', like it was a strange question. [...] We don't have a leg to stand on to teach them anything about originality, academic integrity/intellectual honesty, or the importance of doing things for themselves when they catch us indulging in it just to save time at work.

...It is no surprise that when this model fails, it is the likelihood rather than the prior that is causing the problem. In the binomial model under consideration here, the prior comes into the posterior distribution only once and the likelihood comes in n times. It is perhaps merely an accident of history that skeptics and subjectivists alike strain on the gnat of the prior distribution while swallowing the camel that is the likelihood.

Every few weeks, someone on Twitter notices how demented the content on Facebook is. I’ve covered a lot of these stories. The quick TL;DR is that Facebook’s video section is essentially run by a network of magicians and Vegas stage performers who hack the platform’s algorithm with surreal low-value content designed to distract users long enough to trigger an in-video advertisement and anger them enough to leave a comment.

At death, you break up: the bits that were you Start speeding away from each other for ever With no one to see. It’s only oblivion, true We had it before, but then it was going to end And was all the time merging with a unique endeavour To bring to bloom the million-petaled flower Of being here. Next time you can’t pretend There’ll be anything else.

The men of old sought to rectify all under Heaven. Seeking to rectify all under Heaven, they sought to rectify their nation. Seeking to rectify their nation, they sought to rectify their people. Seeking to rectify their people, they sought to rectify their family. Seeking to rectify their family, they sought to rectify themselves. And seeking to rectify themselves, they sought to rectify their hearts.

[On Python 3.12 subinterpreters] there's massive advantages for mixed C(++) and Python: I can now have multiple sub interpreters running concurrently and accessing the same shared state in a thread-safe C++ library. Previously this required rewriting the whole C++ library to support either pickling (multiplying the total memory consumption by the number of cores), or support allocating everything in shared memory (which means normal C++ types like `std::string` are unusable, need to switch e.g. to `boost::interprocess`). Now is sufficient to pickle a pointer to a C++ object as an integer, and it'll still be a valid pointer in the other subinterpreter.

... Facebook will be hosting the second User Experience Summit for OpenID on February 10th. The goal is to convene some of the best designers that leading internet companies can muster, and bring them together to develop a series of guidelines, best practices, iterations, and interfaces for making OpenID not just suck less, but become a great experience

For most software engineers, being well rounded is more important than pure technical mastery. This was already true, of course — see @patio11's famous advice "Don't call yourself a programmer" — but even more so due to foundation models. In most situations, skills like being able to use AI to rapidly prototype in order to communicate with clients to iterate on specifications create far more business value than technical wizardry alone.

The late F. W. H. Myers used to tell how he asked a man at a dinner table what he thought would happen to him when he died. The man tried to ignore the question, but on being pressed, replied: "Oh well, I suppose I shall inherit eternal bliss, but I wish you wouldn't talk about such unpleasant subjects."

A calculator has a well-defined, well-scoped set of use cases, a well-defined, well-scoped user interface, and a set of well-understood and expected behaviors that occur in response to manipulations of that interface. Large language models, when used to drive chatbots or similar interactive text-generation systems, have none of those qualities. They have an open-ended set of unspecified use cases.

In war, there is the free possibility that not only individual determinacies, but the sum total of these, will be destroyed as life, whether for the absolute itself or for the people. Thus, war preserves the ethical health of peoples in their indifference to determinate things; it prevents the latter from hardening, and the people from becoming habituated to them, just as the movement of the winds preserves the seas from that stagnation which a permanent calm would produce, and which a permanent (or indeed 'perpetual') peace would produce among peoples.

If you tell a friend they can now instantly create any app, they’ll probably say “Cool! Now I need to think of an idea.” Then they will forget about it, and never build a thing. The problem is not that your friend is horribly uncreative. It’s that most people’s problems are not software-shaped, and most won’t notice even when they are. [...] Programmers are trained to see everything as a software-shaped problem: if you do a task three times, you should probably automate it with a script. Rename every IMG_\.jpg file from the last week to hawaii2025_\.jpg, they tell their terminal, while the rest of us painfully click and copy-paste. We are blind to the solutions we were never taught to see, asking for faster horses and never dreaming of cars.

For we can always see and feel much that the people in old photos and newsreels could not: that their clothing and automobiles were old-fashioned, that their landscape lacked skyscrapers and other contemporary buildings, that their world was black and white and haunting and gone.

Reality is that LLMs are not AGI -- they're a big curve fit to a very large dataset. They work via memorization and interpolation. But that interpolative curve can be tremendously useful, if you want to automate a known task that's a match for its training data distribution. Memorization works, as long as you don't need to adapt to novelty. You don't need intelligence to achieve usefulness across a set of known, fixed scenarios.

My "why move away from SGML?" reason is the way that every time I have to explain to someone that their Mozilla bug in invalid because HTML is actually an SGML application [...] I finish up by saying "if you want to see the actual spec that I've been told says that, you can buy a copy for 230 Swiss francs."

Again, in the early days, I was limited in computing capacity and it was clear, in my area, that a "mathematician had no use for machines." But I needed more machine capacity. Every time I had to tell some scientist in some other area, "No I can’t; I haven’t the machine capacity", he complained. I said "Go tell your Vice President that Hamming needs more computing capacity." After a while I could see what was happening up there at the top; many people said to my Vice President, "Your man needs more computing capacity." I got it! I also did a second thing. When I loaned what little programming power we had to help in the early days of computing, I said, "We are not getting the recognition for our programmers that they deserve. When you publish a paper you will thank that programmer or you aren’t getting any more help from me. That programmer is going to be thanked by name; she’s worked hard." I waited a couple of years. I then went through a year of BSTJ articles and counted what fraction thanked some programmer. I took it into the boss and said, "That’s the central role computing is playing in Bell Labs; if the BSTJ is important, that’s how important computing is." He had to give in. You can educate your bosses.

I was at a leadership group and people were telling me "We think that with AI we can replace all of our junior people in our company." I was like, "That's the dumbest thing I've ever heard. They're probably the least expensive employees you have, they're the most leaned into your AI tools, and how's that going to work when you go 10 years in the future and you have no one that has built up or learned anything?

The whole scheme of things is turned wrong end to. Life should begin with age & its privileges and accumulations, & end with youth & its capacity to splendidly enjoy such advantages. As things are now, when in youth a dollar would bring a hundred pleasures, you can’t have it. When you are old, you get it & there is nothing worth buying with it then. It’s an epitome of life. The first half of it consists of the capacity to enjoy without the chance; the last half consists of the chance without the capacity.

Add tests in a commit before the fix. They should pass, showing the behavior before your change. Then, the commit with your change will update the tests. The diff between these commits represents the change in behavior. This helps the author test their tests (I've written tests thinking they covered the relevant case but didn't), the reviewer to more precisely see the change in behavior and comment on it, and the wider community to understand what the PR description is about.