Quotes

TL;DR: To serve users at the 75th percentile (P75) of devices and networks, we can now afford ~150KiB of HTML/CSS/fonts and ~300-350KiB of JavaScript (gzipped). This is a slight improvement on last year's budgets, thanks to device and network improvements. [... This is] what we should be aiming to send over the wire per page in 2023 to reach interactivity in less than 5 seconds on first load

We respect wildlife in the wilderness because we’re in their house. We don’t fully understand the complexity of most ecosystems, so we seek to minimize our impact on those ecosystems since we can’t always predict what outcomes our interactions with nature might have. In software, many disastrous mistakes stem from not understanding why a system was built the way it was, but changing it anyway. It’s super common for a new leader to come in, see something they see as “useless”, and get rid of it – without understanding the implications. Good leaders make sure they understand before they mess around.

My goal is to keep SQLite relevant and viable through the year 2050. That's a long time from now. If I knew that standard SQL was not going to change any between now and then, I'd go ahead and make non-standard extensions that allowed for FROM-clause-first queries, as that seems like a useful extension. The problem is that standard SQL will not remain static. Probably some future version of "standard SQL" will support some kind of FROM-clause-first query format. I need to ensure that whatever SQLite supports will be compatible with the standard, whenever it drops. And the only way to do that is to support nothing until after the standard appears. When will that happen? A month? A year? Ten years? Who knows. I'll probably take my cue from PostgreSQL. If PostgreSQL adds support for FROM-clause-first queries, then I'll do the same with SQLite, copying the PostgreSQL syntax. Until then, I'm afraid you are stuck with only traditional SELECT-first queries in SQLite.

GitHub, by default, writes five replicas of each repository across our three data centers to protect against failures at the server, rack, network, and data center levels. When we need to update Git references, we briefly take a lock across all of the replicas in all of our data centers, and release the lock when our three-phase-commit (3PC) protocol reports success.

For some time, I’ve argued that a common conception of AI is misguided. This is the idea that AI systems like large language and vision models are individual intelligent agents, analogous to human agents. Instead, I’ve argued that these models are “cultural technologies” like writing, print, pictures, libraries, internet search engines, and Wikipedia. Cultural technologies allow humans to access the information that other humans have created in an effective and wide-ranging way, and they play an important role in increasing human capacities.

My own favorites were Cuba voting "yes" to the fast-tracking of OOXML, even though Microsoft is prohibited by the US Government from selling any software on the island that might even be able to read and write the new format, and Azerbaijan's "yes" vote, even though OOXML as defined isn't able to express a Web URL address in Azeri, their official language.

Nature is commonly viewed as harmonious and human markets as full of strife, yet the above comparison suggests the opposite. The psychological prominence of unusual phenomena may explain the apparent inversion of the common view. Symbiosis stands out in biology: we have all heard of the unusual relationship between crocodiles and the birds that pluck their parasites, but one hears less about the more common kind of relationship between crocodiles and each of the many animals they eat... Similarly, fraud and criminality stand out in markets. Newspapers report major instances of fraud and embezzlement, but pay little attention to each day’s massive turnover of routinely satisfactory cereal, soap and gasoline in retail trade. Crime is unusual and interesting; trade is common and boring. ...Imagine that predation were as fundamental to markets as to biology. Instead of confronting occasional instances of theft in a background of trade, one would be surrounded by neighbors who had stolen their cars from dealers who had mounted an armed assault on factories in Detroit, which in turn had grabbed their parts and equipment by pillaging job-shops in the surrounding countryside.

"I will tell you what I am talking about", he said. "Most kinds of power require a substantial sacrifice by whoever wants the power. There is an apprenticeship, a discipline lasting many years. Whatever kind of power you want. President of the company. Black belt in karate. Spiritual guru. Whatever it is you seek, you have to put in the time, the practice, the effort. You must give up a lot to get it. It has to be very important to you. And once you have attained it, it’s your power. It can’t be given away: it resides in you. It is literally the result of your discipline. Now what is interesting about this process is that, by the time someone has acquired the ability to kill with his bare hands, he has also matured to the point where he won’t use it unwisely. So that kind of power has a built-in control. The discipline of getting the you so that you won’t abuse it. But scientific power is like inherited wealth: attained without discipline. You read what others have done, and you take the next step. You can do it very young. You can make progress very fast."

Here's the thing: if nearly all of the time the machine does the right thing, the human "supervisor" who oversees it becomes incapable of spotting its error. The job of "review every machine decision and press the green button if it's correct" inevitably becomes "just press the green button," assuming that the machine is usually right.

New authentication schemes such as OpenID, or Microsoft's CardSpace, may help as adoption increases. These systems make it possible to register for one site using credentials verified by another. Instead of having many sites with poor verification procedures, the internet could have a few sites with strong verification procedures, that are then used by others. The advantage for the user is that they no longer have to jump through multiple hoops for each new site they encounter.

I used to think Twitter would never catch on in the mainstream because it’s somewhat stupid. Now I realize I was exactly wrong. Twitter will catch on in the mainstream because it’s somewhat stupid. It’s blogging dumbed down for the masses, and if there’s one surefire way to build something popular, it’s to take something else that is already popular and simplify.

I like to make sure almost every line of code I write is under a commercially friendly OS license (usually Apache 2) for genuinely selfish reasons: I never want to have to solve that problem ever again, so OS licensing my code now ensures I can use it for the rest of my life no matter who I happen to be working for in the future

I would say ChatGPT (mostly the new GPT-4 model), with a lot of hand-holding and cajoling from me, wrote 60-70% of the code (PHP, Javascript, CSS, SQL) for this AMA site. And we easily did it in a third of the time it would have taken me by myself, without having to look something up on Stack Overflow every four minutes or endlessly consulting CSS and PHP reference guides or tediously writing tests, etc. etc. etc. In fact, I never would have even embarked on building this little site-let had ChatGPT not existed...I would have done something much simpler and more manual instead. And it was a blast. I had so much fun and learned so much along the way.

iPlayer usage, for streaming, peaks about 10pm - just a little later from TV. But interestingly, iPlayer on the iPhone peaks at about midnight. So people are clearly going to bed with their iPhone and watching in bed. And we also see on the weekends, there's a peak of Saturday and Sunday morning usage at about 8 to 10am in the morning on iPhone.

I did find one area where LLMs absolutely excel, and I’d never want to be without them: AIs can find your syntax error 100x faster than you can. They’ve been a useful tool in multiple areas, to my surprise. But this is the one space where they’ve been an honestly huge help: I know I’ve made a mistake somewhere and I just can’t track it down. I can spend ten minutes staring at my files and pulling my hair out, or get an answer back in thirty seconds. There are whole categories of coding problems that look like this, and LLMs are damn good at nearly all of them. [...]

in July 2023, we [Hugging Face] wanted to experiment with a custom license for this specific project [text-generation-inference] in order to protect our commercial solutions from companies with bigger means than we do, who would just host an exact copy of our cloud services. The experiment however wasn't successful. It did not lead to licensing-specific incremental business opportunities by itself, while it did hamper or at least complicate the community contributions, given the legal uncertainty that arises as soon as you deviate from the standard licenses.

On Monday, this Court entered an order requiring OpenAI to hand over to the New York Times and its co-plaintiffs 20 million ChatGPT user conversations [...] OpenAI is unaware of any court ordering wholesale production of personal information at this scale. This sets a dangerous precedent: it suggests that anyone who files a lawsuit against an AI company can demand production of tens of millions of conversations without first narrowing for relevance. This is not how discovery works in other cases: courts do not allow plaintiffs suing Google to dig through the private emails of tens of millions of Gmail users irrespective of their relevance. And it is not how discovery should work for generative AI tools either.

Rosencrantz: "Do you think Death could possibly be a boat?" Guildenstern: "No, no, no... death is not. Death isn’t. Take my meaning? Death is the ultimate negative. Not-being. You can’t not be on a boat." Rosencrantz: "I’ve frequently not been on boats." Guildenstern: "No, no... what you’ve been is not on boats."

I've actually been using the latest version of JAWS recently, as part of my work on HTML5. From a usability point of view it is possibly the worst software I have ever used. I'm still horrified at how bad the accessibility situation is. All this time I've been hearing people worried about whether or not Web pages have longdesc attributes specified or whatnot, when in fact the biggest problems facing blind users are so much more fundamental as to make image-related issues seem almost trivial in comparison.

I designed Dropbox's storage system and modeled its durability. Durability numbers (11 9's etc) are meaningless because competent providers don't lose data because of disk failures, they lose data because of bugs and operator error. [...] The best thing you can do for your own durability is to choose a competent provider and then ensure you don't accidentally delete or corrupt own data on it: 1. Ideally never mutate an object in S3, add a new version instead. 2. Never live-delete any data. Mark it for deletion and then use a lifecycle policy to clean it up after a week. This way you have time to react to a bug in your own stack.

The paradox of ChatGPT is that it is both a step forward beyond graphical user interfaces, because you can ask for anything, not just what’s been built as a feature with a button, but also a step back, because very quickly you have to memorise a bunch of obscure incantations, much like the command lines that GUIs replaced, and remember your ideas for what you wanted to do and how you did it last week

We simply don’t know to defend against these attacks. We have zero agentic AI systems that are secure against these attacks. Any AI that is working in an adversarial environment—and by this I mean that it may encounter untrusted training data or input—is vulnerable to prompt injection. It’s an existential problem that, near as I can tell, most people developing these technologies are just pretending isn’t there.

But unlike the phone system, we can’t separate an LLM’s data from its commands. One of the enormously powerful features of an LLM is that the data affects the code. We want the system to modify its operation when it gets new training data. We want it to change the way it works based on the commands we give it. The fact that LLMs self-modify based on their input data is a feature, not a bug. And it’s the very thing that enables prompt injection.

When attention is being appropriated, producers need to weigh the costs and benefits of the transaction. To assess whether the appropriation of attention is net-positive, it’s useful to distinguish between extractive and non-extractive contributions. Extractive contributions are those where the marginal cost of reviewing and merging that contribution is greater than the marginal benefit to the project’s producers. In the case of a code contribution, it might be a pull request that’s too complex or unwieldy to review, given the potential upside

Much like an oral agreement, publishing microformats is an informal agreement between you and (hopefully) a developer community that sets up a relationship with plenty of vagueness, inertial resistance to change, and potential landmines to step on. Would you create a real developer API without a TOS, agreement, or at the very least, guidelines? [...] are you prepared to announce all frontend markup changes? Does publishing a microformat without a special agreement mean that you are implicitly allowing comprehensive scraping of your web data?

"Man has Nature whacked", said someone to a friend of mine not long ago. In their context the words had a certain tragic beauty, for the speaker was dying of tuberculosis. "No matter", he said, "I know I’m one of the casualties. Of course there are casualties on the winning as well as on the losing side. But that doesn’t alter the fact that it is winning."

We used this model [periodically transmitting configuration to different hosts] to distribute translations, feature flags, configuration, search indexes, etc at Airbnb. But instead of SQLite we used Sparkey, a KV file format developed by Spotify. In early years there was a Cron job on every box that pulled that service’s thingies; then once we switched to Kubernetes we used a daemonset & host tagging (taints?) to pull a variety of thingies to each host and then ensure the services that use the thingies only ran on the hosts that had the thingies.