The Swyx Mixtape

Listen to a16z live: https://a16z-live.simplecast.com/episodes/one-on-one-with-a-and-z-11-how-much-rare-knowledge-is-there-4aF0UbP7

- rarely wrong just early
- pets.com, diapers.com
- hadoop -> databricks
- Nycira -> central control plane - 2b revenue company
- rare knowledge - what do you believe that nobody else believes
- 2 kinds - super secret, or in plain sight
- airbnb - history of hotels
- new ideas nobody else has
- take seriously things that nobody else believes
- history gets rewritten to be both obvious and inevitable
- bill gates the road ahead

Transcript
 

[00:00:00] Hey everyone today, I'm sharing a clip from the Andreessen Horowitz podcast from both Marc Andreessen and Ben Horowitz. Which is them talking about rare knowledge basically the secrets that you believe that no one else does as well as how to develop a view of the future which is important obviously for vcs but i think generally important as well 

[00:00:19] Marc Andreesen: How much rare knowledge is there in the world in your experience or concretely, how often does it happen that there are less than 10 people you can think of? No. I could do something and you'd be skeptical you can find anybody else that can, and then a huge, no asks. How can I develop a view for the future, which I think is actually a very related question.  

[00:00:38] Ben Horowitz: So way more than you would think. There's a lot of very knowledge and, mark and I like experienced this on our job. Every single day. There's just, the world is really dynamic. Now, in fact, it's probably never been more dynamic, I think, by any measure. And so what we see all the time is the old conventional wisdom just ceases to be true. 

[00:01:02] And we've seen this here. One of our favorite examples is just. All the dot bombs that everybody made, hysterical fun of during the early two thousands, all the idiotic ridiculous, stupid ideas that people had for the future, which were just so obviously. All eventually worked. And and it was a matter of the underpinnings of the internet and other things changing to the point where those really bad ideas, Pets.com or diapers.com or any of these kinds of things, they all work fantastically later, as world changed. And that all was super rare knowledge because the conventional knowledge was of course, those things are all the stupidest things ever, and you'd have to be some kind of moron to leave your high-paying consulting job to do that. 

[00:01:54] But we continually see this and in the firm, we even have kind of a rule. Which is, if you know too much about something you got to back off, because you know what, particularly, if historically what did not work that can be dangerous knowledge in our business because you can miss it the next time when it actually does work. 

[00:02:14] And w we just hit all the time on. Favorite investments that I've made was it was common knowledge in Silicon valley that Hadoop had one big data, like architecturally, like that was the thing. It had one open source that had hearts and minds, blah, blah, blah. It was going to be the. 

[00:02:30] And I think, probably the best, for sure one of the best investments I ever made was that wasn't true. And so it's just like a small piece of rare investing knowledge, but a big piece of rare knowledge for the entrepreneurs who invented spark. And then, later turned that into Databricks. 

[00:02:46] Marc Andreesen: Yeah. Ben, do you remember? So for we invested in a company called Nycira early on in the life of the firm, like nine, 2010. And then do you remember the we shouldn't name him, but we had a meeting with a a I believe the CTO of one of the really big networking companies at the time in our diligence literally said it was  

[00:03:01] Ben Horowitz: against the laws of physics. 

[00:03:02] It wasn't possible. They had already like steady. At length this very large, important networking company and you could not have a central control plane and the way that the Sierra proposed to do. And of course, now in the Sierra inside, VMware is like a $2 billion a year revenue. 

[00:03:21] Marc Andreesen: Yup. Yup. And then of course, a classic essence on the consumer side of course, is that everybody knew right up until 2004, that consumers would never put their real identities online. That was the one thing that would never happen. Never. And then of course Facebook like completely blew that open. 

[00:03:32] So the twist that I want, there's a couple of twists that I wanted to put on. This are circuit aspects of this very important question that I want it to get a little deeper into. For those of you who like, think about these things you will have, some of you will have at least read Peter Teal's kind of famous book zero to one, he talks a lot in that book about, what he calls the secret, which is this idea of the were knowledge that other people don't use. And then he has this famous question that he asks he had kind of dinner parties where he asked this question of what, what is the thing that, that nobody else knows. Or he asks a, there's a sort of a related verse and the question is just, what do you believe that nobody else believes? 

[00:04:03] And of course what's interesting is those are not necessarily the same thing. And then on top of that there's this question of okay, it's the rare knowledge, something that is actually not known. Like it's actually a piece of information that's like invisible. And so for example, rare knowledge, let's take a hypothetical case, rare knowledge of a chemical. 

[00:04:20] That has been invented in a lab and only the people who worked in that lab know that formula exists and only they know what the formula is, right. Or maybe the formula for KFC seasoning might be something that's, a formula it's literally locked  

[00:04:34] Ben Horowitz: 11 herbs and spices, but we do not know what they are. 

[00:04:38] Marc Andreesen: We do not know which one. So there's literally the rare knowledge is any knowledge, literally like you can't get to it. There's this other kind of rare knowledge which goes to the second version of Peter's question of what do you believe that nobody else believes her fear that people believe, which is like the rare knowledge of something that is actually like a fact that is actually completely like, basically in, in public view. 

[00:04:59] It's like a thing that anybody can walk up to or learn about or read about, or read on the internet and tout, or read an academic paper about or whatever. But it's just people just simply don't believe it. Like they just don't buy it. Like they're not having it. 

[00:05:11] They just, they, they found some reason to rule it out, by the way, they're often a ridiculing it. And so Ben, the question I always think about on this is. How many of the secrets, right? Or how much of the rare knowledge in the world, how much of it is literally the 11 herbs and spices that you can't go find out versus how much of it is the thing that's there in plain sight that everybody's just making fun of? 

[00:05:30] Yeah. The  

[00:05:30] Ben Horowitz: plane tech definitely seems much bigger. At least in, in artwork we see many more of the plain sight ones. One of my favorites was Airbnb, which had a kind of thing that was a little bit of a secret formula and that. Just discovered there was a lot more demand for running your mattress. 

[00:05:45] As I started out than one would have thought, particularly if there was a big design conference in San Francisco, but the other one that was really in plain sight was what was the history of hotels. And anybody could have known the history of hotels. Like when did hotels start? 

[00:06:00] They started a hundred years. But nobody really knew that who knows that nobody knows that. But they studied that after they found that out and I'm like why did hotels come about? Why weren't they around before? And it turned out that, there were before that there were Inns in bed and breakfast. 

[00:06:13] But quality would vary a lot and you couldn't really create a brand. There was no national media. There were, there the way there was at the turn of the previous century. And so that, that national media enabled you to create a national brand and and build this kind of common, known quantity, quality nationwide, but of course, Once you knew that you'd go, oh, now we have the internet. 

[00:06:37] So we don't need a brand. We can just rate every room, every light, bedroom, every house, every tree house, every everything. So everybody knows the quality of anything that they're staying. And then all of a sudden you've got this amazing idea for a new kind of  

[00:06:52] Marc Andreesen: Yeah. And in fact, many of you think about it, like the idea of Airbnb was in the Bible Joseph, and Mary, they stay in the manger, that worked out pretty well for them. 

[00:07:01] It's I was thinking back it's every, every little kid, at least in Western culture is here's that story. Cause it's, it's right there with Christmas.  

[00:07:06] Ben Horowitz: It's universally known  

[00:07:07] Marc Andreesen: in Western and yet it's yeah, no, th whenever we started this idea of people staying sort of houses, which is horrifying so it was like quite, quite literally in plain sight. 

[00:07:16] Ben Horowitz: Yeah. Yeah no, for sure. But I think that, that and, that's the, to me, that's the exciting thing, particularly if you're a young person getting into the workforce or what have you there are all kinds of secrets running around everywhere, and then there's this, it's actually one of the great things. 

[00:07:32] One of the great things, maybe the only great thing about this kind of mob mentality, cancel culture thing that we have going on. The conventional wisdom is so aggressive now that if you go against it you are likely the only one. And then you've got an amazing invention that can change the world. 

[00:07:49] It's, it is a, it's a great time to be an entrepreneur in this.  

[00:07:54] Marc Andreesen: Yeah. And then this goes to the second question. Why I paired the questions? Second question was, how can I develop a view for the future? And the way I think about this is there's basically two ways to develop a view for the future, that flow right out of what we just talked about. 

[00:08:05] So one is like trying to come up with ideas that nobody else has. So try to come up with a unique viewpoint of like literally new ideas. And of course that's hard. The other is just right. Takes really smart to do that. Yes. Yes. A small number of people can do that. Most of us can't do that. 

[00:08:20] The the, but the other way to do it is, quite literally believe things that other people aren't willing to believe, or equivalently take seriously things that other people aren't willing to take. I, And I think that's probably the most underrated approach for people who really want to have an open mind and a view of the future that actually gives them an advantage. 

[00:08:35] Ben Horowitz: Yeah. I totally agree with that. And that's true for like even small innovations, like how you organize your company or, how you run a certain kind of program. If you do the opposite of what everybody in the industry is doing, you might come across something that's.  

[00:08:48] Marc Andreesen: Yeah. And then, that's the other thing, right? 

[00:08:50] The other thing that happens as revisionist history, and we've got to, actually, the next question is about history, but all history is revisionist. All history is revisionists, right? Generally what happens when a new, when I knew it, when a new idea actually wins, right? When a new idea actually takes root and becomes conventional wisdom, sort of history gets rewritten so that everybody knew it was obvious the whole time. 

[00:09:09] Yeah. Actually interestingly, both obvious and inevitable, right? And but if you were there at the time where you actually get the true history of it, what you realize is actually that very much, that's not the case. The history just gets written by people who are basically covering from their own mistake. 

[00:09:22] Ben Horowitz: One of the most shocking pieces of technology history on that to me is always the internet because everybody like talks about it. Of course it was always going to be the internet. But you recall, like even after Netscape started. And was like pretty far along and got the venture funding and, you had invented the browser and all these things like the conventional wisdom was like, no, it's absolutely not going to be the internet. 

[00:09:45] It's going to be information super highway and the Internet's insecure and it's a bunch of, it's an academic network and there's just a bunch of weirdos on it and blah, blah, blah, blah, blah, blah, blah. And now nobody would admit to that. And my favorite part of that is bill gates. When he wrote the road ahead and the first edition, he didn't even mention the internet. 

[00:10:01] He wrote a book about the future in 1990. That did not mention the internet, not once. And then he revised it. Of course, then, swap the internet and everywhere where he had information super highway. So the ultimate revisionist, like the literal revisionist history on that one.  

[00:10:20] Hey, it's me again. So just to recap, some of the lessons learned Nicholas, I took notes. Um, basically there's a constant theme of in the injuries in the camp that most ideas, most startup ideas are rarely raw, wrong just early so that they raised a few examples. I think they raised four examples. This is pretty good.  

[00:10:38] pets.com diapers.com from the first internet rush, uh, basically became. Uh, Legitimate startups. And in the early 2010s and then, uh, Hadoop and Databricks and NYSERDA, which, uh, was Martin Casados startup. Which, uh, they dropped a really interesting stat where they said that it actually makes $2 billion in revenue internally within VMware, which is a huge, huge, huge,  

[00:11:05] Um, I, I did not expect that coming out and they, Sarah. They also broke down rare knowledge, which is what do you believe that nobody else believes? And into two kinds, either it's super secret like the KFC formula or it's hidden in plain sight. And that example was linked to Airbnb. If we've studied history of a hotels, you understand that hotels are actually a very recent phenomenon and.  

[00:11:27] The history of travel actually shows that Airbnb was probably a good idea. It's just that nobody else took it seriously. So the link that into having new ideas about the future. The two is to do that. One, which is have a genuinely new idea that nobody else has ever had. But to take seriously things that nobody else believes.  

[00:11:46] Or take seriously. And I think in order to do that, or in order to study history, And. Generate ideas or develop a process for that. You have to be aware of the narrative fallacy. And that history gets rewritten to be both obvious and inevitable. I particularly remember that I in my household had a copy of bill gates is the road ahead. And yeah, information super highway was plastered all over that thing and no mention of the internet. So I think a very interesting perspective on predicting the future and having novel ideas that are under appreciated.