Quotes

[T]he work that people did had been broken down into jobs that were the same every day, in organizations where people were interchangeable parts. All of the story had been bled out of their lives. That was how it had to be, it was how you got a productive economy. But it would be easy to see a will at work behind this: not exactly an evil will, but a selfish will. The people who’d made the system thus were jealous, not of money and not of power, but of story. If their employees came home with interesting stories to tell, it meant that something had gone wrong: a blackout, a strike, a spree killing. The Powers That Be would not suffer others to be in stories of their own unless they were fake stories that had been made up to motivate them. People who couldn’t live without story...had to look somewhere outside of work for a feeling that they were part of a story, which I guessed was why Sæculars were so concerned with sports, and with religion. How else could you see yourself as part of an adventure?

When Noam and Daniel started Character.AI, our goal of personalized superintelligence required a full stack approach. We had to pre-train models, post-train them to power the experiences that make Character.AI special, and build a product platform with the ability to reach users globally. Over the past two years, however, the landscape has shifted – many more pre-trained models are now available. Given these changes, we see an advantage in making greater use of third-party LLMs alongside our own. This allows us to devote even more resources to post-training and creating new product experiences for our growing user base.

Why, for a decade of experience, can we not seem to see the IE 8 zombie coming? It’s not like it’s going to be some big surprise that unless we do something different, we’ll still be supporting it in 2015. That’s right: in 2015, you’ll still be thinking about a browser that doesn’t support canvas or video and doesn’t even have a JITing JS engine.

I’ve really come to appreciate that performance isn’t just some property of a tool independent from its functionality or its feature set. Performance — in particular, being notably fast — is a feature in and of its own right, which fundamentally alters how a tool is used and perceived.

To give a person one’s opinion and correct his faults is an important thing. It is compassionate and comes first in matters of service. But the way of doing this is extremely difficult. To discover the good and bad points of a person is an easy thing, and to give an opinion concerning them is easy, too. For the most part, people think that they are being kind by saying the things that others find distasteful or difficult to say. But if it is not received well, they think that there is nothing more to be done. This is completely worthless. It is the same as bringing shame to a person by slandering him. It is nothing more than getting it off one’s chest. To give a person an opinion one must first judge well whether that person is of the disposition to receive it or not. One must become close with him and make sure that he continually trusts one’s word. Approaching subjects that are dear to him, seek the best way to speak and to be well understood. Judge the occasion, and determine whether it is better by letter or at the time of leave-taking. Praise his good points and use every device to encourage him, perhaps by talking about one’s own faults without touching on his, but so that they will occur to him. Have him receive this in the way that a man would drink water when his throat is dry, and it will be an opinion that will correct faults. This is extremely difficult. If a person’s fault is a habit of some years prior, by and large it won’t be remedied. I have had this experience myself. To be intimate with all one’s comrades, correcting each other’s faults, and being of one mind to be of use to the master is the great compassion of a retainer. By bringing shame to a person, how could one expect to make him a better man?

Flickr users are marked as such in the Yahoo user database. What this means is that the account is permanently protected from deletion, even if you cancel your SBC-Yahoo DSL and even if you never check your Yahoo Mail (if you elect to have one). Both free and pro accounts are protected. And your Yahoo signon name will not be displayed anywhere on Flickr -- your existing Flickr username will stay the same.

The big thing I always get asked to find are dank dilapidated alleys, and New York City has, like, 5 alleys that look like that. Maybe four. You can’t film in three of them. So what it comes down to is there’s one alley left in New York, Cortlandt Alley, that everybody films in because it’s the last place. I try to stress to these directors in a polite way that New York is not a city of alleys. Boston is a city of alleys. Philadelphia has alleys. I don’t know anyone who uses the ‘old alleyway shortcut’ to go home. It doesn’t exist here. But that’s the movie you see.

Whether we believe the Greek poet, "it is sometimes even pleasant to be mad", or Plato, "he who is master of himself has knocked in vain at the doors of poetry"; or Aristotle, "no great genius was without a mixture of insanity"; the mind cannot express anything lofty and above the ordinary unless inspired. When it despises the common and the customary, and with sacred inspiration rises higher, then at length it sings something grander than that which can come from mortal lips. It cannot attain anything sublime and lofty so long as it is sane: it must depart from the customary, swing itself aloft, take the bit in its teeth, carry away its rider and bear him to a height whither he would have feared to ascend alone.

The [Apple Foundation Model] pre-training dataset consists of a diverse and high quality data mixture. This includes data we have licensed from publishers, curated publicly-available or open-sourced datasets, and publicly available information crawled by our web-crawler, Applebot. We respect the right of webpages to opt out of being crawled by Applebot, using standard robots.txt directives. Given our focus on protecting user privacy, we note that no private Apple user data is included in the data mixture. Additionally, extensive efforts have been made to exclude profanity, unsafe material, and personally identifiable information from publicly available data (see Section 7 for more details). Rigorous decontamination is also performed against many common evaluation benchmarks. We find that data quality, much more so than quantity, is the key determining factor of downstream model performance.

A traditional centralized database only needs to be written to once. A blockchain needs to be written to thousands of times. A traditional centralized database needs to only checks the data once. A blockchain needs to check the data thousands of times. A traditional centralized database needs to transmit the data for storage only once. A blockchain needs to transmit the data thousands of times. The costs of maintaining a blockchain are orders of magnitude higher and the cost needs to be justified by utility. Most applications looking for some of the properties stated earlier like consistency and reliability can get such things for a whole lot cheaper utilizing integrity checks, receipts and backups.

iPlayer usage, for streaming, peaks about 10pm - just a little later from TV. But interestingly, iPlayer on the iPhone peaks at about midnight. So people are clearly going to bed with their iPhone and watching in bed. And we also see on the weekends, there's a peak of Saturday and Sunday morning usage at about 8 to 10am in the morning on iPhone.

I think there's been a lot of decisions over time that proved pretty consequential, but we made them very quickly as we have to. [...] [On pricing] I had this kind of panic attack because we really needed to launch subscriptions because at the time we were taking the product down all the time. [...] So what I did do is ship a Google Form to Discord with the four questions you're supposed to ask on how to price something. But we got with the $20. We were debating something slightly higher at the time. I often wonder what would have happened because so many other companies ended up copying the $20 price point, so did we erase a bunch of market cap by pricing it this way?

I like the lies-to-children motif, because it underlies the way we run our society and resonates nicely with Discworld. Like the reason for Unseen being a storehouse of knowledge - you arrive knowing everything and leave realising that you know practically nothing, therefore all the knowledge you had must be stored in the university. But it's like that in "real Science", too. You arrive with your sparkling A-levels all agleam, and the first job of the tutors is to reveal that what you thought was true is only true for a given value of "truth". Most of us need just "enough" knowledge of the sciences, and it's delivered to us in metaphors and analogies that bite us in the bum if we think they're the same as the truth.

Beyond these specific legal arguments, Stability AI may find it has a “vibes” problem. The legal criteria for fair use are subjective and give judges some latitude in how to interpret them. And one factor that likely influences the thinking of judges is whether a defendant seems like a “good actor.” Google is a widely respected technology company that tends to win its copyright lawsuits. Edgier companies like Napster tend not to.

So something everybody I think pretty much agrees on, including Sam Altman, including Yann LeCun, is LLMs aren't going to make it. The current LLMs are not a path to ASI. They're getting more and more expensive, they're getting more and more slow, and the more we use them, the more we realize their limitations. We're also getting better at taking advantage of them, and they're super cool and helpful, but they appear to be behaving as extremely flexible, fuzzy, compressed search engines, which when you have enough data that's kind of compressed into the weights, turns out to be an amazingly powerful operation to have at your disposal. [...] And the thing you can really see missing here is this planning piece, right? So if you try to get an LLM to solve fairly simple graph coloring problems or fairly simple stacking problems, things that require backtracking and trying things and stuff, unless it's something pretty similar in its training, they just fail terribly. [...] So that's the theory about what something like Q* might be, or just in general, how do we get past this current constraint that we have?

I grew up in a college town, and one Halloween our doorbell rang and we opened the door expecting to see trickortreater - but what was in front of our open door - was another door! Like, a full-on wooden door, that had a sign that said "Please knock." So we did, and the door swung open to reveal a bunch of college dudes dressed as really old grandmothers, curlers in their hair, etc, who proceeded to coo over our "costumes" and tell us we were "such cute trick or treaters!" One even pinched my cheek. Then THEY gave US candy, closed their door, picked it up and walked to the next house.

Providing validation, strength, and stability to people who feel gaslit and dismissed and forgotten can help them feel stronger and surer in their decisions. These pieces made me understand that journalism can be a caretaking profession, even if it is never really thought about in those terms. It is often framed in terms of antagonism. Speaking truth to power turns into being hard-nosed and removed from our subject matter, which so easily turns into be an asshole and do whatever you like. This is a viewpoint that I reject. My pillars are empathy, curiosity, and kindness. And much else flows from that. For people who feel lost and alone, we get to say through our work, you are not. For people who feel like society has abandoned them and their lives do not matter, we get to say, actually, they fucking do. We are one of the only professions that can do that through our work and that can do that at scale.

Not funk but funk conquered is what is worthy of admiration & makes life worth having been lived. Courage, not cleverness; not even inspiration, is the grain of mustard that grows up to be a great tree. To the extent there is courage, there is connection with life & death. (I was thinking of Labor’s & Mendelssohn’s organ music.) But it is not by recognizing the want of courage in someone else, that you acquire courage yourself. One might say: “Genius is courage in one’s talent”.

Did you ever wonder why the 21st century feels like we're living in a bad cyberpunk novel from the 1980s? It's because these guys read those cyberpunk novels and mistook a dystopia for a road map. They're rich enough to bend reality to reflect their desires. But we're [sci-fi authors] not futurists, we're entertainers! We like to spin yarns about the Torment Nexus because it's a cool setting for a noir detective story, not because we think Mark Zuckerberg or Andreesen Horowitz should actually pump several billion dollars into creating it.

I toss these pages in the faces of timid, furtive, respectable people and say: 'There! that's me! You may like it or lump it, but it's true. And I challenge you to follow suit, to flash the searchlight of your self-consciousness into every remotest corner of your life and invite everybody's inspection. Be candid, be honest, break down the partitions of your cubicle, come out of your burrow, little worm.' As we are all such worms we should at least be honest worms.

Despite being rusty with coding (I don't code every day these days): since starting to use Windsurf / Cursor with the recent increasingly capable models: I am SO back to being as fast in coding as when I was coding every day "in the zone" [...] When you are driving with a firm grip on the steering wheel - because you know exactly where you are going, and when to steer hard or gently - it is just SUCH a big boost. I have a bunch of side projects and APIs that I operate - but usually don't like to touch it because it's (my) legacy code. Not any more. I'm making large changes, quickly. These tools really feel like a massive multiplier for experienced devs - those of us who have it in our head exactly what we want to do and now the LLM tooling can move nearly as fast as my thoughts!

This is an apology for the way we previously handled illegally sold copies of 1984 and other novels on Kindle. Our "solution" to the problem was stupid, thoughtless, and painfully out of line with our principles. It is wholly self-inflicted, and we deserve the criticism we've received. We will use the scar tissue from this painful mistake to help make better decisions going forward, ones that match our mission.

We argued that ChatGPT is not designed to produce true utterances; rather, it is designed to produce text which is indistinguishable from the text produced by humans. It is aimed at being convincing rather than accurate. The basic architecture of these models reveals this: they are designed to come up with a _likely continuation of a string of text_. It’s reasonable to assume that one way of being a likely continuation of a text is by being true; if humans are roughly more accurate than chance, true sentences will be more likely than false ones. This might make the chatbot more accurate than chance, but it does not give the chatbot any intention to convey truths. This is similar to standard cases of human bullshitters, who don’t care whether their utterances are true; good bullshit often contains some degree of truth, that’s part of what makes it convincing.

A smile, a laugh A grin, a whirl A suppression of truth and honor And the execution of young Hidetsugu So, stand on the ramparts Of Osaka Castle May your Hideyori Rule for 10,000 years Consumption of gold and pearls Spices and incenses Wines and sake And oh-so much attention Somewhere Nene is crying Hideaki is stewing Mitsunari is blundering Somewhere Hidetada is training with his blade Mototada is training his forces Ieyasu is setting his pieces Somewhere The East is rising The Christians are preparing And Hideyori is not But laugh, and joy, and whirl No one can see through you Certainly, the Toyomi Will live 10,000 years

So Fishing Times’s ad department is selling access to the prime Fishing Times readership. But the Data Lords can say, ‘we can show your ad just to Fishing Times readers when they’re on Facebook, or on some meme site, on the Times or TPM or really anywhere.’ Because the Data Lords have the data and they can track and target you. The publication’s role as the gatekeeper to an audience is totally undercut because the folks who control the data and the targeting can follow those readers anywhere and purchase the ads at the lowest price.

A further point, too, is that these people who never attain independence follow the views of their predecessors, first, in matters in which everyone else without exception has abandoned the older authority concerned, and secondly, in matters in which investigations are still not complete. But no new findings will ever be made if we rest content with the findings of the past. Besides, a man who follows someone else not only does not find anything, he is not even looking. "But surely you are going to walk in your predecessors’ footsteps?" Yes indeed, I shall use the old road, but if I find a shorter and easier one I shall open it up. The men who pioneered the old routes are leaders, not our masters. Truth lies open to everyone. There has yet to be a monopoly of truth. And there is plenty of it left for future generations too.

There was this clamour in the past to get companies to open source their products. This has stopped, because all the software that got open source sucked. It's just not very interesting to have a closed source program get open sourced. It doesn't help anyone, because the way closed source software is created in a very different way than open source software. The result is a software base that just does not engage people in a way to make it a valid piece of software for further development.