In this week’s episode of Founded & Funded, Madrona Managing Director Matt McIlwain hosts NYSE Group President Lynn Martin ahead of our Intelligent Applications Summit on October 10th and 11th. Lynn will be speaking at the Summit, but we thought it would be great to have her on the show to talk more about her background, the NYSE, capital markets, IPO trends, and, of course, the role data, AI, and large language models are playing in the markets and in companies broadly! This conversation couldn’t come at a better time now that we’re seeing a number of tech companies teeing up IPO plans.

Lynn thinks we’ll be back to a more “normal” IPO environment in 2024 – but you have to listen to get her full take!

The 2023 Intelligent Applications Summit is happening on October 10th and 11th. If you’re interested, request an invite here.

This transcript was automatically generated and edited for clarity.

Matt: I’m Matt McIlwain, and I’m a managing director here at the Madrona Venture Group. I’m delighted to have Lynn Martin, who’s the president of the NYSE Group, which includes the New York Stock Exchange, and she’s also Chair of ICE Fixed Income and Data Services. Lynn, you started your career coding, you were a computer programmer back at IBM out of college, and now you’re leading the world’s largest global stock exchange and a bunch of related businesses. Before we dive into things like the capital markets and the businesses that you run, and of course, the topic of AI that’s on everybody’s mind, I’d love to jump into how you get from programming at IBM to the financial markets?

Lynn: So first, Matt, thanks so much for having me. It’s always good to spend time with you — great question as to how I wound up here. As you pointed out, I was a programmer. I was trained as a programmer, I have an undergraduate degree in computer science, and it was around the time of the dot-com revolution, and markets, in particular, were starting to have technology integrated into them.

I was at IBM, and I was really looking for something that spoke to my passions. My passions at the time were financial markets, financial market infrastructure, and the math that underpinned financial markets. It’s what drove me to get my master’s degree in statistics. I happened upon an opportunity in 2001 with a derivatives exchange, which was based in London called London Financial Futures Exchange. They had a tremendous amount of people in their organization that could speak about the products that they had listed, so really European short-term interest rate debt and equity index futures, but they didn’t have anyone who could talk to the new breed of trader, which was people who were writing to the API as markets started to go electronic.

I interviewed with them, and what really interested them about me was the fact that I was a programmer. I had conversational knowledge of some of the models and some of the products, but I could talk to a programmer who was writing to this thing called an API at the time.

Matt: Yeah, before APIs were cool.

Lynn: Exactly.

Matt: Yes.

Lynn: At the time, the whole management team of that exchange was like, “What’s an API?” So the right place and the right time, and the interesting skillset that led me to financial markets back in 2001.

Matt: No, that’s fantastic. To fast-forward a little bit, you ended up at NYSE Group and have had successive opportunities and broadening roles. I don’t think most people really have an appreciation for the breadth of the NYSE Group and then the related relationship with ICE. Can you paint that broader picture, and then we can talk specifically about the areas that you have responsibility for?

Lynn: Yeah, I’ll fast-forward through a little bit of history. In 2012, I was with NYSE. I was CEO of a futures exchange that NYSE was a 51% owner of, and 49% was owned by a variety of institutions on the street, and a company called ICE came along and acquired the New York Stock Exchange. ICE was an electronic group of exchanges, a company that was very much driven by the implementation of cutting-edge technology and the application of that technology to financial markets. That acquisition was completed in 2013, and I found myself in a very interesting position in that I was in a directly overlapping business unit.

I wound up taking a position being offered and then taking a position to run one of their clearing houses. I got to know the founder of ICE, a gentleman by the name of Jeff Sprecher, who is still our chair and CEO of ICE, who really got to know my background both from an academic standpoint and a business standpoint. He’s the type of guy who’s always thinking about two years, three years, five years down the road, and he’s been a great mentor to me in that regard.

At the time, he was thinking about, “Okay, a really important output of financial markets and equally an important input to the most liquid financial markets is data.” And he was already thinking at that time, “There’s something around data that I want this ICE group to be a part of.” It led us to form ICE Data Services in 2015, which I was named president of, and it rapidly grew into ICE’s largest single business unit through a variety of acquisitions. The way you should think about that group is pretty much what I just said. It’s all down to the premise that the most liquid markets have a tremendous amount of data as outputs. In order to make a market — and a market can be broadly defined — liquid and actionable, you need data to form what value is going to be.

That can apply to everything from the U.S. equity markets, it can apply to fixed income markets, it can apply to a lot less traditional markets such as where ICE Group is now entering the next generation of its evolution, the mortgage market, the U.S. mortgage market, and how we use data to make a more informed decision or allow for a more informed decision in the U.S. mortgage market.

Matt: You chair this whole group, the ICE Fixed Income and Data Services. Can you give us one tangible example of the more popular data services you provide today and how that works? We’ll come back later to talk about how AI is changing or giving you opportunities to enhance some of these areas.

Lynn: AI plays an incredibly important role in the Fixed Income and Data Services vertical that we have at ICE. You can take AI and large language models from an efficiency standpoint. How do you do more with less bodies? How do you cover a broader universe, as I like to think of in the fixed-income markets? In the fixed income markets, that manifests itself in us providing valuations for 2.7 million securities globally and also providing terms and conditions data, the underlying aspects of a bond where when it matures, what its coupon rate is, all those sorts of good things on 35 million securities globally.

We do that through the implementation of natural language processing, large language models, and all different forms of artificial intelligence. Still, importantly, we have a strong human overlay that knows if what those models are producing is good information or bad information. You don’t want the erroneous information to continue to perpetuate through a system because then that winds up polluting a system.

Matt: You go from a virtuous to a vicious cycle, and these capital markets are dynamic. Before we dive more into some ways you use ML and AI, what is the state of the capital markets today? I’m thinking about the public and the fixed-income markets generally, and how do you think about that market space, the context beyond all the services that you provide?

IPO Trends

Lynn: They’re operating as you would expect them to operate. So 2020, and 2021, if you think about the equity markets, you had a tremendous amount of IPOs coming out because the system was flush with cash, and people were taking advantage of high valuations. And then what you saw in 2022, early 2022, you saw volatility start to creep in, volatility instigated by multiple factors, the war in Ukraine, rising interest rates, monetary policy, how do we pay back some of that money that’s been injected into the system? That caused the IPO markets, the people who were tapping the U.S. equity markets, to grind to a halt, as we saw in 2022.

As we moved into 2023, you’re now starting to see the effects of the Fed implementation of rising interest rates start to work its way through the system. So effectively, you’re starting to see inflation come down, and you’re starting to see the desired effects that the Fed had with its monetary policy start to take hold. The way we’ve started to see that manifest itself in our markets is number one: the volatility has come way down. The key barometer that I always look at is the VIX. The VIX has been 13, 15, somewhere around there, definitely below 20 for a sustained period of time.

That’s really important for a company that’s thinking, “Okay, is it time for me to tap the public markets from a capital perspective?” What you’ve now seen is, at least in Q2, we saw the IPO markets start to reopen. So you saw Kenvue, which was a spin-out of Johnson and Johnson, raise a $3.8 billion price at the high end of the range upsize. You saw CAVA, you saw Savers, you saw Fidelis, you saw Kodiak more towards the end of Q2. What encouraged me about everything we saw in Q2 is you saw both U.S. domestic deals get done, you saw international deals get done, and you saw multiple sectors come to market, and that’s the first time that you’ve really seen a good healthy cross-section of activity tap the public markets since 2021, certainly.

Matt: Well, I’m sensing some cautious optimism in the return of the IPO market. It’s been a year and a half since Samsara listed on the NYSE back in December of 2021 — since you’ve really had a technology IPO and the sense of how we think about it in the tech venture business, how are you seeing the end of this back half of this year in terms of technology IPOs and then also the trade-off between the specific situation that some of these IPOs might be at a lower price in the public markets than the last private round?

Lynn: I think the investors have gotten their heads around the fact that the valuations are going to be different from 2021. 2021 was a bit of an anomaly in terms of the valuations. Very optimistic that we’re going to start to see deals come out in the second half, particularly in technology. Ultimately someone’s got to test the waters, though, so there’ll be a couple of deals that will probably come out in the second half of 2023, which are going to do that. And we’re working with a variety of companies in our backlog who are now saying, “Yeah, we’re going to go in probably early Q4 timeframe and there may actually be one or two that go prior to that.”

Very optimistic that those deals are going to come out, those deals are going to get done, and then that’s going to really set the stage for 2024, which is when I think we’re going to be back to a more normal IPO environment.

Matt: That’s encouraging. With that in mind, we work with private companies at all different stages. Some of them are getting close to being IPO-ready. What kind of advice do you and your teams give to these kinds of private, rapidly growing companies about what it means to actually be IPO-ready?

Lynn: Yeah, and I use the term IPO-ready a lot. The bit that 2021 showed was that 2021 was giving value to growth at all costs. It was for every dollar of growth, the market was rewarding you. The companies that are coming out now have balanced growth as well as profitability and how to not sacrifice growth but also come out with some discipline on the expense side. Ultimately, an investor’s going to ask you about expenses during your earnings call, they’re going to ask you about growth during an earnings call. I think being already of that mindset that you’re balancing growth with profitability is a good philosophy to have going into your IPO because if you’re a company that’s a year old, five years old, 10 years old, 100 years old, you’re going to get asked those questions on your earnings call.

Matt: It’s a tricky balance, and things got a little out of kilter in the 2020 back half of 2021. I agree that we’re coming back to more of a balanced outlook there. Maybe just turn just a little bit, you’ve got this whole area of fixed income you’re responsible for. Just give us a view of the fixed-income markets. Treasury prices have been volatile. I think they’ve stayed more volatile in general here and are trending downward. The rates on the ten years have popped up a good bit again. What’s your outlook on the fixed income side, and how does that influence the timing and this return to the IPO market?

Lynn: It’s all ultimately tied together. When the equity markets are a bit more volatile, there could be a flight to quality, and it could be more of the treasuries that people invest in. We’ve seen a lot of volatility not just in treasuries, we’ve seen it also in the muni markets, the U.S. municipal markets. Also, investment grade and high-yield corporate debt have been bouncing around a bit as well. The treasury market, though, has been incredibly volatile. Typically when the treasury market is volatile, the muni markets are volatile too because those two trade at a spread to each other. I think to the extent that there is continuing to be treasury market volatility, you’ll see it manifest itself across those two asset classes in particular.

Matt: There’s just so much input data and output data in all these different capital markets and related data services areas. Tell us about the groups that you are responsible for and how you guide and direct them around what more can we do with this data? What can we do with AI in general, generative AI in particular?

Lynn: One of the things that I’ve always been focused on the data side is observable data. Good data is gold, and bad data, when it gets into a system, is really challenging to expunge, so I tend to focus on how we harness the good data and build the right models off of that good data to extrapolate other pieces of information. One of the areas that we’ve been super excited about is how we can take data and how we can take these large language models and add efficiency to the trader’s workflow.

One of the big mantras I have is to always stay close to your customers and talk to your customers about what their pain points are, so one of the pieces of functionality that we’re incredibly proud of that we built almost a decade ago in our data services business is our ICE Chat platform. That chat platform connects the entirety of the energy markets ecosystem, everything from a drilling company to a producer, to an investor, and even to people who carry the cargo from different ports for delivery. Within that chat platform, we had developed a proprietary large language model, which knows if I’m talking to you about what day of the week today is, or it knows if I’m talking to you about the price of crude oil.

If it senses that we’re talking about the price of crude oil, it’ll pop in some analytics alongside it, including a fair value for what it detects we’re talking about. Then if it further detects information, it underlines the price and quantity that we’re exchanging information about potentially transacting and allows a seamless submission to our exchanges and our clearing houses. It’s adding a tremendous amount of efficiency to a trader’s workflow, and we’ve actually seen an uptick in volume on our benchmark energy markets on the ICE side of the business during the first half of this year by 60% using this mechanism.

I think that’s because people are becoming more and more conversant in large language models, how to use them, how to have them add efficiencies to their workflows. We just see it manifests itself in our energy markets and then, more recently, in our utility markets.

Matt: Now, it’s very interesting. It’s sort of a capital markets chatbot copilot, as it were. We’re having a conversation, we’re trying to understand things, and you are providing relevant, real-time information that facilitates the exchange of ideas, which ultimately can facilitate the exchange of financial transactions.

Lynn: Yep, that’s absolutely right. And it all comes back to our north star, which is data adds transparency to markets, and information adds transparency. You think about what all of these large language models are doing, they’re adding transparency, they’re giving you pieces of information. That’s why the underlying data that underpins them is so critically important to have correct.

Matt: Yes, I agree. It seems like much the whole world agrees now, whereas even nine months ago, I think the dots had not been connected for people, but of course, we had ChatGPT, and that was kind of a moment we’ll all look back on. We recently did some analysis here at Madrona, working with some AlphaSense data about just how many people are talking about generative AI and specifically looked at Q2 earnings, we’re not quite through the Q2 earnings season, but we’re getting close, and what was amazing was that we looked at both software as a service companies and then other or non-SaaS, and there was about a hundred of these software-as-a-service companies. They mentioned generative AI or AI on average seven times per company, with over 700 mentions on their earnings calls just this quarter.

And what was maybe an even more amazing statistic was that the non-GenAI companies, in this case, over 300 of them had mentioned on average five times in their earnings call generative AI or AI-related topics. And this is companies, Warby Parker, Lululemon, Warner Bros. I mean, you can go down the list. A whole bunch of amazing companies that you all, of course, have helped and trade on the New York Stock Exchange. What are you hearing from CEOs and CFOs from tech and non-tech companies in this GenAI area?

Lynn: Well, I agree with you. AI seems to be the letters of the year. I mean, I grew up in a generation where we had letter of the day, these are the letters of the year. From every company I talk to, they talk about the importance of data. And unsurprisingly, it’s a topic that I love to talk about. Now, there is so much data floating around in this world. It is really hard as a human to process all of the information that you have at your fingertips. I mean, petabytes of data at your fingertips on any one given day. You need things like AI, you need things like large language models to help parse through the data in an efficient fashion that’s going to give you additional insights, it’s going to give you good insights on some of the names you just mentioned on what your customers are doing, what your customers are saying about your products, and what your customers are doing in terms of purchase behavior.

That’s going to help drive your own investments, it’s going to help you drive where your next investment dollar is going to be. In the case of Warby Parker, it’s going to be on a pair of sunglasses. They recently rolled out sunglasses that flip from two different colors. What are people saying about that? Has that been a hit? If it’s been a hit, should we make more of those types of products? So it really helps the investment flywheel in a company accelerate because you’re getting that piece of information a lot quicker than you normally would.

Matt: I think that’s well said, and it makes me think back to the FinTech market specifically and 10, 12 years ago, just the rise of cloud computing and computing at massive scale. I think, as you’ve mentioned, part of what ICE took advantage of, I think architecturally as a business, how are you seeing within FinTech the potential for leveraging GenAI and LLMs? And of course, I know you’re already doing things, but how are you all strategically thinking about where this might go and evolve your business over time?

Lynn: We’re looking at ways within our own business across ICE to make processes much more efficient that are ripe for efficiency plays. One business, I mentioned it earlier, that ICE is keenly focused on is our growing mortgage technology business and how to make the process of homeownership in the U.S. much more streamlined and, therefore, more efficient for the consumer. And how do we put good data in the consumer’s hands so they’re making a more informed purchase or getting a more informed rate on an interest rate?

If you look at the tremendous amount of data that’s available within that vertical in particular, we think there’s a tremendous amount of insights that can be unleashed to make the process more efficient in terms of buying a home, securing a mortgage, refinancing a mortgage, even the securitization process. It’s an area that we’re keenly focused on at the moment.

Matt: Given the leading-edge work that you all at the NYSE Group are doing, as well as the fact that you’re really looked to from all these public companies or companies that are considering going public, are they also asking you for your perspective and advice in these areas? And how so?

Lynn: Yeah, and we like to be a good partner to our firms. I always describe companies the first time we meet, either early on the startup side or we’re pitching them to go public on us. We’re in it to be your business partner. And as part of being your business partner, we’re here to share our experiences. We’re here to share our learnings, we’re here to share our tools. I think it’s important to zoom out on issues because there might be a micro-type issue that someone is struggling with around data privacy, for example.

But if you zoom out just a little bit, and while you may think a consumer business doesn’t have anything to learn from a financial services business, they wind up having a lot to learn from each other because of customer information, for example, and the way people handle customer privacy information. There are a lot of learnings to be shared across our companies, and we’re sort of the clearing house, if you will, for that type of information. I sometimes feel like my job is to help issue spot and to spot macro trends for our companies and make connections within our community to allow companies to have those conversations on topics that they may think they have nothing in common on, but they actually do.

Matt: It brings to mind your own experience now having been president of the NYSE group, it’ll be approaching two years, what has been a bigger learning or two for you in terms of this leadership role and something that you’d want to share with others that are in similar leadership roles? I’m curious what you’ve been learning over the last couple of years.

Lynn: The thing that excites me the most about this job is just the platform that NYSE has and the role that we play in financial markets. Certainly, thought leadership and financial markets globally, but the role we can play is to be a convener of people to have really important in-depth conversations on a variety of topics. The topic of this year is clearly the two letters we’ve been talking about, AI, and that’s an area, but AI is one thing, and if you double-click on AI, you wind up with a variety of sub-issues, data privacy being one, what the application of AI to different industries is too? It’s such a meaty topic that I think, given the role that we have in financial markets, we have a very good opportunity to bring people together to discuss these topics.

Matt: It’s a topic that we’re going to be unpacking more, and we’re just delighted that you’re going to be coming out to join us for the Intelligent Applications Summit on October 11th in Seattle. It’s going to be a very special event again this year, and we’re looking forward to you sharing your perspective and learnings on what you’re doing in the AI area, as well as what’s happening in the capital markets.

Thinking back again to where we started in this conversation, the wonderful rise from being a computer science major and a programmer and working at IBM to being president of the NYSE Group and all these other responsibilities with ICE, tell us what advice you would give to college students today and college students that are trying to start off on the right foot with their career journey.

Lynn: I was fortunate in that I had the opportunity to start in a company, IBM, which gave me the ability to interact with multiple different companies. I started in a consulting role at IBM, and that gave me the opportunity to see a variety of different industries. The one thing that I think was incredibly important in my first few years was that I fell in love with a specific area, so the advice I always give to college students is you’ve got to love the field you’re in. You have to wake up every day wanting to learn more about it. I wake up every day and I come to 11 Wall Street, and I learn something new because that’s just what you should be doing. The world is constantly changing, constantly evolving, and if you have the opportunity to join an organization where you could love the subject matter and continuously learn, you’re in the right place.

Matt: It’s such good advice. Having that feeling that you wake up every day wanting to learn more, being at a place where you love the subject matter, you enjoy the people, you respect the people, completely inspiring to me as well, and I’ve been out of college for a few years.

Lynn: So have I.

Matt: I can’t wait to spend more time together here and it’s incredibly impressive what you’ve done in your rise through NYSE and then this broader role and what you and your teams are doing on their front foot to innovate in areas like AI and generative AI in particular. So Lynn, thank you very much for joining us on the podcast, and look forward to seeing you soon.

Lynn: Thanks for having me.

Coral: Thank you for listening to this week’s episode of Founded & Funded. We’d love you to rate and review us wherever you get your podcasts. If you’re interested in learning more about the New York Stock Exchange and what they can do for companies, visit their site at www.nyse.com. And if you’re interested in attending our IA Summit and hearing more about IPO trends from Lynn Martin, visit ia40.com/summit. Thank you again for listening, and tune in a couple of weeks for our next episode of Founded & Funded with Statsig Founder and CEO Vijaye Raji.

This week, Vivek Ramaswami talks with Chroma Co-founder and CEO Jeff Huber. Chroma is an open-source vector database, and vector databases have taken the AI world by storm recently, with Chroma quickly emerging as one of the leading offerings in the space. Jeff’s background is in machine learning, and for him and his co-founder, starting Chroma was a classic “there has to be a better way moment when it comes to going from demo to production”. Vivek and Jeff talk about why on earth we need another database, the new wave of software engineering happening right now, community-led growth, the competitive landscape, and so much more.

The 2023 Intelligent Applications Summit is happening on October 10th and 11th. If you’re interested, request an invite here.

This transcript was automatically generated and edited for clarity.

Vivek: I’m Vivek Ramaswami. I’m a partner at Madrona, and today, I am joined by Jeff Huber. Jeff is the co-founder and CEO of Chroma, an AI-native open-source embeddings database, which is also more commonly known as vector databases. Jeff, thanks so much for joining us today.

Jeff: Yeah, thanks for having me. Looking forward to the conversation.

Vivek: Jeff, you’ve got a history in the tech and startup world that predates your founding story at Chroma and I would love to hear about that. Maybe you can just share a little bit of background on yourself. How did you get into this world? How did you get to the founding of Chroma?

Jeff: Yeah, maybe the quick life story version of this is I actually was born in the Peninsula, grew up in North Carolina, and then moved back out to the Bay Area about 11 years ago to work in early-stage startups. It’s really fun to build things from scratch. A number of at-bats there. The most previous was a company called Standard Cyborg. We went through Y Combinator in the winter ’15 batch, which at the time felt like being late to YC, and now there’s winter ’23 or whatever, or summer ’23. So now winter ’15 feels like a long time ago. Standard Cyborg had many journeys, I think a few of the journeys that are maybe relevant to our conversation today. We had built out an open-source, online, dense reconstruction pipeline for the iPhone. The iPhone 10 came out in 2018, and we thought this was really interesting. There was a new sensor that was going to be in everybody’s pockets, and that was going to enable accurate 3D scanning to be in every person’s pocket for the first time.

Apple just gave you the raw depth data, so we built up this whole SLAM pipeline around that, open-sourced it, and then we were working with a bunch of companies to develop both conventional computer vision-based approaches as well as machine learning-based approaches to analyze that data at scale. We were doing things like virtual helmet fitting, powering a smart glasses company’s virtual try-on and fitting solution, working with a cycling company around shoe sizing and fitting, all kinds of interesting sizing and fitting applications.

We were doing stuff with ML. It was, like we said, the Wild West. It still is today, but at the time, TensorFlow 1.5 had just come out. PyTorch wasn’t really a big deal yet. We were doing point cloud segmentation, going way beyond object detection images. The thing that we really felt the pain of quite intimately was how hard it is to take anything that has machine learning in it, to take it from demo to production. That was the pain point that my co-founder and I, Anton, connected over. Was, wow, machine learning is really powerful and likely has the opportunity to change the way that we build software and change the world more broadly, but getting from demo to production is incredibly hard. That’s actually part of the reason we started Chroma — to solve that problem.

Vivek: Super interesting. Well, first of all, you were building in machine learning far before it became cool in this current hype cycle that we’re seeing and with all the interest. What were the challenges of building in that period relative to today?

Jeff: Yeah. Obviously, large language models are a bit of a paradigm shift. Before, if you wanted to build a machine learning model to do a thing, let’s say, for example, draw a bounding box around all the cars in a picture — a classic computer vision use case. You had to train a model to do exactly that from scratch in many cases or maybe you use a base architecture and you fine-tune on top of it. And cars is common enough. You could pick up a model probably fairly easily and get something decent, but let’s say that you wanted to draw bounding boxes around chairs, maybe that data doesn’t exist. It’s a huge effort to collect the data. It’s a huge effort to label the data. It’s a huge effort to train anything at all. And you can put in all of that effort before you even get your first taste of how good this thing could be.

Then you have to put in a month or more of work probably for each iteration cycle. You use the thing that you built, it works, we kind of have a sense for where it’s weak, and let’s do some things, label some more data to go try to make it better in that way, and then you wait two to four weeks to get your next answer. It’s a really long feedback loop. You basically have very little information as to why the thing is doing the thing that it’s doing, and really honestly, you have educated guesses for how to make it better, which makes for an incredibly infuriating developer cycle. Take somebody who has also built out React applications and you can change the style sheet and dynamically updates instantly. You can get that feedback loop down to milliseconds. Feedback loops of months or more is really painful in software development.

So yeah, those are a bunch of challenges with traditional deep learning. I think with language models and this news zeitgeist of people calling it generative AI. The interesting change that happened is that there are now general-purpose models, and you can do things with just base models that before you’d have to put a ton of effort into. Now application developers can pick up GPT-3 API and pick up some embeddings and in a weekend, have an interesting thing, and that is a very new thing. Before, building even the small demos was only the purview of machine learning engineers, data scientists at best and was sort of untouchable for your average developer, not average in terms of skillset, just average in terms of backend development, frontend development, full stack development, infra. And now those engineers can build things, and they are building things. And that’s a pretty incredible and exciting thing.

Vivek: Yeah, the resources and tools available to you as a builder in this space now are obviously significantly greater than they were not even just five years ago, even a year or two ago, and that’s terrific for the devs and the engineers in this space. I definitely want to get into how you think about generative AI, and even before getting into Chroma itself. Let’s just start at a higher level. We’re all hearing about AI, we’re hearing about vector DBs, we’re hearing all of these things. Just in terms of this market, why does AI memory matter at all? When we talk about things like AI memory retrieval, just paint a picture of why that matters, how that connects to this generative AI boom that we’re seeing today, and all the applications like ChatGPT and others that so many of us are familiar with?

Jeff: One way to think about what’s happening right now in AI is a new wave of software engineering is starting. Historically, the only way to get a computer to do a thing was to write explicit instructions in the form of code, and hopefully, the computer wrote those instructions, and hopefully, you didn’t write any bugs. If you followed those guidelines, you didn’t write any bugs, that program’s going to run 9999.999999 times out of 100. It’s always going to work. That is not the case with AI. At least today, we don’t yet have the tools necessary to get that level of reliability. That being said, traditional software is also bounded because there’s only a very narrow set of use cases where you can so tightly define all of the inputs such that traditional software development works. Imagine the space of all possible programs that can exist; code written by humans likely only covers 1% of that map of all possible programs that can exist.

It’s a classic example in xkcd. I believe it’s a manager goes to his employee and says, “Hey, can you add this new column to store a last name?” And the person says, “Yeah, it would take three minutes.” And then he says, “Hey, can you also add a thing that says if they upload this photo, whether it’s a cat or a bird,” and the person says, “Yeah, that would take three years.” That was an xkcd comic from a while back. I think that has changed and has gotten a lot easier, but it remains true that it is difficult to bring things in AI to production. So there’s, I think, a lot of exciting work going on. Obviously, retrieval is one method. So zoom out again. We want to be able to program language models, we want to be able to get them to do what we want them to do every time. And you basically have two options, and they’re not mutually exclusive.

One option is to change the data that the model has seen in its training, and you usually do that today through fine-tuning. The other method is to change the data the model has access to at inference time, and that is through the context window, and that is through, generally, retrieval. Instead of programming with code, we program with data. And that’s what this new zeitgeist in programming will be about — how do we guide the execution path of language models, this new generation of software, and how do we guide it using data? And again, retrieval is a really, really, really useful tool. I think that most production use cases today need to use retrieval, and I think most production use cases in the future will also need to use retrieval. I think fine-tuning today is probably a fairly low percentage of use cases, maybe like 10% need fine-tuning, though I expect that number to go up over time.

Vivek: That makes sense. And so retrieval is important, and I think there’s this new paradigm of companies that are being built. Especially if you are leveraging all of the AI tools that we have today, retrieval is more and more important. At the same time, one could argue that there’s a number of databases that already exist. And you just go on DB-Engines, and it lists 400 plus. Why do we need a new type of database? Why are we hearing so much about new databases and new vector DBs that are popping up, not just yours but others? What’s your sense of why we have this novel need?

Jeff: I think each generation of software, whether you look at web, mobile, web 1.0, they have different ergonomics and they have different needs that developers have, and that leads to new kinds of databases. What happened with web and mobile, developers both wanted to have faster development speed, so they wanted to get away from tightly rigid schemas and migrations, and then they also wanted to be able to support much larger scale. There’s the classic meme version of this, which is MongoDB is Web Scale, which is pretty funny to go back and watch, but there are some truths to that: different tools are better at different things. And you could put racing tires on a Honda Accord, but it does not make a race car.

In the current phase that we’re in, my recommendation to application developers is to not necessarily try to take an existing technology that’s added supposedly vector search on top of an existing database and try to make it work for your application because application developers need to squeeze all of the juice and get all of the alpha out of the latest and greatest technology. And you can’t be six months behind or a year behind because you’re too far behind. Furthermore, you really need to have things like really great recall. Again, production is very hard in this world, and so doing things like getting much worse recall but in an existing database is not a trade-off that I think developers should make.

So there’s a bunch more to it, which I can speak to. Everything from integrations, ergonomics, dimensions around scalability. There’s a lot to do with even just the low-level data structures that certain storage algorithms inside of traditional relational databases, for example, are good for and certain things that they’re not good for. I think, in some ways, the proof is in the pudding. Developers are reaching for, in many cases, these purpose-built solutions because they solve their problems. And ultimately, that’s what developers want. Developers want to build things, and developers do not want to spend lots of time taking a Honda Accord and retrofitting it to be an F1. They just want to go. They want to build something. And if both tools are open-source and both tools are free, there’s kind of an obvious answer there.

Vivek: I think you named a number of dimensions in which, to your point, you can’t retrofit a very different kind of car with different types of tires. It needs to be purpose-built for what the engineer and what the developer is trying to do.

Jeff: Adding to that, we’re just so early in all of this stuff. We’re just so early, and I think it’s comical how people are adding vector search to their existing databases. Oftentimes, they are pretty bad versions of vector search. There’s a ton of hair on them not advertised, but they add it and they throw up the marketing page, and the VP of marketing is happy, and so is the CEO. Maybe the public market is happy too. We’re so early.

It is not even obvious that cosine similarity search will be the predominant tool 18 months from now. It’s not obvious that points, which vectors are, and fancy rulers, which search is — points and rulers. I do not think that’s a sufficient tool set to build a production application. A lot more has to exist and has to exist at the database level. I think that’s the other point here is that just we’re so early to what memory for AI means and looks like, and it’s pretty likely that a learned representation ends up being the best anyways. You actually want to have neural nets running against your data and at this very low level to power retrieval. And that’s how you get the best memory. And if that’s how you get the best memory, that’s what you’ll want to use.

Vivek: Yeah, you made the point. It’s sort of like how every app is trying to figure out their generative AI strategy or their generative strategy, and they’re just creating something that sits on top of OpenAI or GPT-3 or 4. And now it’s, we can add this vector capability, but time will tell how much that really moves the needle or not.

Jeff: For the record, for the database companies listening to this, keep doing it, please, because you’re educating the market and doing me a huge favor, so we appreciate it.

Vivek: That’s great. Well, let’s get to Chroma then. So you kind of laid out really nicely what the market looks like and the needs for developers today in the kind of databases that they need and especially as they’re building these kinds of products. So what was the founding story to get to Chroma itself? Where did you see that there was a unique need for you and your co-founders that led you to this ah-ha moment?

Jeff: I’ll say upfront, in many ways, I think we’ve gotten really lucky, and I think the community ultimately is why the project has grown. We released the project on Valentine’s Day of this year. It was the day after when it was supposed to release, so we had a bug we had to fix. The growth we’ve seen from those early days to this point in terms of the number of users and people talking about it online has been entirely organic. For most of this year, we’ve been a very small team. We still are, to this day, a very small team. That will remain true for probably some time, honestly, into the future. With a small team, you can only do so many things. And the thing that we chose to focus on was trying to serve our users to developers. They use Chroma. Trying to serve them as best as possible, take their feedback seriously, give them help on Discord as fast as possible.

The community helped to make it better in terms of giving us feedback, opening up pull requests, and opening up issues. I think we’ve been lucky thus far or our path to this point. We started the company, again, because we felt like the gap between demo and production and building anything with machine learning or AI was too hard. We suspected that embeddings mattered, that there’s something about latent space and embeddings, which, pun intended, had some latent alpha. There’s something there that has not been fully explored yet. When you go on an exploration, you can have a hunch that there’s a promised land at the other end of the journey. You don’t know. You’re not certain. And so, the thing that you want to do is run really clean experiments. They’re tightly bounded for time. You’re very, hopefully, opinionated about whether something’s working or not. And don’t trick yourself. Don’t trick yourself into thinking, “oh, this is good enough, we’re getting good enough reaction to this, let’s spend another three years of our lives.”

I think that we as former founders had a high bar for what we wanted to see in terms of what existed versus having to push it for years and years. There’s a longer story there, but basically, we had built out a bunch of retrieval systems ourselves to serve some analytics we were doing for other use cases in embedding space. We found that we didn’t feel like any of the solutions that existed really addressed us as developers. We felt like they were really hard to use, really complicated, really strange deployment models, really strange pricing models, just didn’t fit. It just didn’t make sense. One of those classics, there had to be a better way, and started talking to some more people, some more users that were building stuff back in December when LangChain was a very early project. OpenAI embeddings had just gotten 100X cheaper or something like that with ada-002 embeddings.

And yeah, just started training a bunch of users and found that they shared the sentiment that something that was easy to use and powerful didn’t exist. And so we’re like, that’s interesting. Let’s go see where that goes. Let’s follow that trail and see where that goes. And that led us to, again, launching the project in open-source, and now cumulatively, the Python project has crossed 1.5 million downloads since launch and 600,000 plus of those in the last 30 days.

Vivek: Let’s talk about the community. It’s important and integral in the Chroma story and the momentum you’re seeing now and where you’re going. You talk about how you got some users, and people start to find out about you. There’s some organic growth. I’m sure a lot of the founders listening to this podcast, especially the ones who are building open-source products, are like, how do I even get the first few people in? How do I seed the initial community and start to attract more people? Is there something that you and your founders did to help build that up, or would you say it’s truly organic? I’m sure there’s some elements of both.

Jeff: The general sentiment is if you build it, they will not come. I have certainly always believed that because people underrate distribution, overrate technology day in and day out. I’ve done that before. It feels like that didn’t apply here for some reason. I’m not exactly sure why. It feels like we kind of did build it and they came. So I don’t know if there’s really much to learn.

I think one thing that we did do unintentionally is at the end of December, I had reached out to a bunch of people on Twitter who had tweeted about LangChain and embeddings and all this stuff just to get their feedback on what we were building and see if they shared our opinion. What I realized now, is that we not only were doing user testing and user feedback, but we were also talking to the loud users who had large presences, generally speaking, or were active on Twitter and thinking about the edges of technology. And so in a weird way, we ended up, I think, getting a lot of the early adopters and the early vocal people to be aware of what we were building and why we were building it and in many cases using what we were building. And again, that was very unintentional, but that didn’t hurt.

Vivek: It’s interesting. It’s funny when we say, “back in December” as if that’s years and years ago. It was eight months ago, but it feels like that because of how much the entire ecosystem has grown since. You talk about early days of LangChain and the number of users that they have now and what that community looks like and what Chroma is growing into. Do you find, and I think would be interesting to hear your take on, is building a community in this era and this cycle in AI very different from building a community a few years ago? One of the reasons I ask is I think we all see the number of GitHub Stars. To get to 50,000 stars is just sort of achieved much more frequently now than it was before, and you can kind of go down the list of metrics that matter and what you think about this open-source community. How do you think about building a community today and growing that and sustaining that community because a lot of this is going to be novel relative to maybe the open-source products of yesteryear?

Jeff: Yeah. I don’t know that any of the best practices have changed. I think you need to care about the people who are engaging with your thing. I think you need to give them good support. I think you need to try to get them on Zoom calls, and get to know their names, and get to know what they’re working on, and figure out how you can make what they’re working on better if you can. There’s just a lot of boilerplate community stuff, which isn’t particularly glamorous, but it is the way. There’s a difference, maybe with AI. In some ways, it’s a good thing. In some ways, it’s a bad thing. So in some ways, it’s a good thing that there’s just a lot of eyeballs on this stuff, and people are paying attention to it. Top of the funnel problem is a little more solved than it is in other areas where you’re trying to get people to pay attention.

I’ve heard stories of the early days of HashiCorp with the whole infrastructure as code movement. And really, it took years of going to conferences and tons of evangelism to get people to understand what this thing is and why they should like it. It’s not quite like that. It feels like people, at a minimum, want to like it, and their CEO is saying that they should have an AI strategy. There’s the zeitgeist. The downside of that is that you get a lot more tire-kicking type applications, or not every user who joins our Discord is going to stay engaged. The Discord is bigger than probably it would be if we were building infrastructure as code in 2017 or whenever. HashiCorp got started earlier than that. But the number of engaged users is also percentage-based, probably lower than theirs was at the time. I don’t know, and it’s kind of a mixed bag. It’s kind of weird and interesting to have that top-of-the-funnel problem halfway solved. That’s sort of a strange thing, I think.

Vivek: Yeah, I like the top-of-the-funnel analogy you draw because you’ve got a lot of people interested and willing to try something. Now the question is, how sticky can you make it? So you’ve got this big top-of-the-funnel, and then you’re going to have some percentage that comes down, and how do you keep them on? So any lessons that you have learned from that process? Which is saying, we’ve got all these people, a lot of initial tire kicking interest, how do you get them to continue using?

Jeff: Yeah, I think we’re in some ways not that focused on that as a metric. Clearly, if you get someone to use it and then they stop using it, it could be because what you’ve made sucks, in which case you should care. It could also mean because, I don’t know, they had a spare extra couple hours on a Saturday and their kids at soccer practice, and they hack something out for fun. And they’re going back to their job on Monday, and they churned. But is that a bad thing? I think, no. I think we’re very much in the age of experimentation with all this stuff. In some ways, what we care about is maximizing the number of things that are built. Even if a lot of those things don’t necessarily stick immediately, we take a very long-term view of this.

And so what matters right now is building the best thing possible that serves the community and making sure people don’t get stuck and trying to increase the chances that they get to production. They get us in reliable enough that they would want to use it. And then I think there’s some just returns to time. We’ve been going for five months. What will be true five years from now? What will be true 15 years from now? Just keep going. Also, if you become obsessed over things like churn and retention, you start doing a lot of bad behavior, which is anti-community. And so you start gathering people’s emails and sending out terrible drip content marketing just to try to keep them engaged. You do weird, bad stuff. And I never wanted to join a community that did that to me. And so I feel like there’s a degree to which you can overemphasize that.

Vivek: I think it’s a very good reminder of how early we are in all of this. Where my brain goes to funnel metrics, what does that look like over time? But really, this is experimentation. We’re just so early in this. To your point, ironically, I’m sure a lot of the classic marketing elements will do more to push people out than it will to bring people in and keep them engaged.

Jeff: Exactly. Broadly, people want authenticity, and the developers, first and foremost, just want authenticity.

Vivek: This is clearly one of the most exciting spaces when I talk about vector DBs in this broader category of AI we’re seeing right now, which also means that there’s a number of folks that are doing this. And there’s the number of competitors, and it seems more often than not, we hear of a new competitor coming up and getting funded and all of that. How do you think about operating in a space where a number of new companies have emerged and raised money? And I’ll ask this in two ways. I think one is there’s competing for users over time that’ll be customers, but the second part is competing for talent, which is not easy in any environment, but I think especially so in a highly competitive space like this. We’d love to get your thoughts on both.

Jeff: Yeah, in open-source, we don’t call them competition. We call them alternatives. I think, in some ways, the first rule of competition is don’t be distracted by the competition. If they were perfect, they would’ve already won. And you’d have no room to be good, and you’d have no room to win. I think we don’t spend a lot of time paying attention necessarily to what the alternatives are doing. We spend a lot of time focusing on our users and what they need. That’s sort of the reactive side. The proactive side is, again, going back to this drum I keep hitting, which is there’s a gap between demo and production. It is true about traditional deep learning. It is also true about language models and embeddings. It’s easy to build a sexy demo in a weekend, and people should. It is hard to build a robust, reliable production system today. Still very hard. And there’s a lot that needs to exist to get people there.

I think when you think about our obsession with developer experience, our rich and deep background in AI, our location in San Francisco, I think those will play out to be differentiating features and factors. I wish our competitors, our alternatives, well. I feel like we all had different theses about what’s happening here and what’s playing out. There’s maybe one point, which I will say, is for better or for worse, we are building a lot of this stuff in 2023. And in that sense, we’re “Late to the game” maybe compared to other people, but I think that we’ve made some different decisions specifically in the architecture of what we’ve built in our building. Those decisions, you probably couldn’t see a couple of years ago because nobody could see a couple of years ago. Those are a little bit hard. If you’ve ever made those decisions, they’re a little bit hard to unwind. Again, not necessarily important to get into detail. I think all that really matters for us is to serve the community we have well. We just got to do that.

Vivek: When you serve the community well, good things happen. And to your point, I think that 2023 we’re saying, “We can’t call it late because the next best time is now for all of this stuff.” It’s like things are happening right now, and this is when you want to jump in. We’d love to get your thoughts if we were to zoom out for a second; we’re kind of nearing what I would at least call the first year of this new generative AI boom that we’re in right now.

If I think of ChatGPT launching in November of 2022 as sort of the initial point, you can say we’re ending the first year, and we’re already starting to see from some applications there’s been a little bit of a decrease or at least a flattening of new users and a little bit of the muted version of that exponential growth we saw a year ago, especially on the application side. You have a really interesting vantage point as a founder building this space, more so on the infrastructure side than on the end-user application side. Give us your perspective. What’s happening here? Where are we in this game? Why might we be seeing some flattening in the growth, and what do you think is happening in all of this?

Jeff: I think it’s sort of natural. It’s like there’s a new technology. If you look at the curve of LK-99 reproductions, I was just thinking it’s all a very similar curve. It’s going to be this ramp-up really fast. Everyone’s like, what’s happening? Go, go, go. And then it’s going to taper off, and there’s going to be a bunch of people trying different variants of it to see if you can do something differently. I’m not an expert in this. I won’t pretend to be either. Eventually, maybe somebody finds a variant that works, and then it has this long, what is it called, the plateau of productivity and the adoption cycle, the hype cycle. It feels like Twitter and AI stuff tends to be these hype cycles are extremely compressed. It was pretty funny to watch.

Right before we raised vector databases, we’re like, “Just on the way up.” And then everybody on Twitter is like, vector databases are stupid. There was an immediate trough of disillusionment four weeks after we raised. And now it’s just a long road back up where people are like, “Oh, actually, long context windows, maybe you’re not a panacea with current architectures. Maybe it is useful to be able to control what goes into these context windows. We don’t want to just dump it all in there because these things could distract.” It’s just funny. These hype cycles are so silly. I think that the biggest risk here, period, to this entire space is if we can figure out a way as a community, as an industry, whatever you want to call it, to cross the chasm from a lot of demos that have been built to a lot of useful production applications, and that means many nines of reliability. Not one nine, 90%, which is even probably pretty good for a lot of LLM-based stuff today, but many of nines reliability.

If we can figure that out, then the amount of value that will be created, I think, through this kind of technology is easily in the trillions. It’s just really, really, really, really big. If we cannot, if it remains more, it’s great for some internal tools, it’s cool for some creative stuff, but we’re not going to put it in front of a user because it’s not good enough, or we can’t predict where it’s going to do well enough, so we’re just not going to put it in front of a user. We can’t trust it, ultimately, if we can’t get to that point. It’ll still be valuable, but it won’t be as valuable. It’ll be 1/10, maybe 1/100 as valuable. That’s something that we think a lot about, and so do the language model companies, and so do many of the other observability folks in the space. Everyone has the same goal, the same mission, which is how can we turn this thing from today, which is alchemy, and how can we turn it into engineering?

Vivek: Oh, that’s a great point, the alchemy to engineering point. I think just going back to what you said earlier is so true, which is the hype cycles were always fast in tech, but they have been increasingly compressed. I’m sitting here talking about nine months ago, there’s euphoria, and now the whole thing has been compressed so much. We’re in agreement with you, which is you’re going to have a natural tapering off of these things. I do think it encourages everyone in this space. There’s great experimentation happening, and then the next phase is going to be what’s useful? How do we get this in production and get someone to actually use it versus talking about all the cool marketing, hand-wavy use cases? I think when that happens, there’s no doubt that it’s going to be game-changing for every application, every piece of software, every workflow that we can think of.

Jeff: Exactly. I think more broadly, multimodal, large model-aided software development will more fully approach the space of all possible programs. That’s a good thing for humanity.

Vivek: Well, before we get to the lightning round, take us through where Chroma is going next. You’ve built this amazing community, and you’re getting an incredible amount of traction. In the last month, two months alone, you’re seeing the majority of your usage and growth come in, and so there’s this amazing compounding effect. What’s next for the company, and the community, and the business going forward?

Jeff: We want to do what’s best for the developers that use Chroma today and in the future. A few things need to happen to do that better than we do today. Number one, we need to have a good story around the distributed version of Chroma, which is mostly about scale. I think most data is not big data, but big data does exist and people want to have a path to scale, and I understand that. We’re working on that right now. It’s a distributed cloud native database version of Chroma, which will scale forever in the cloud. The next thing we’re working on is a hosted offering that many developers in their path of hacking out something on their local computer. Then they want to throw it up on a hosted offering to make it easy to share with their team or with their friends. There needs to be an answer to that, and it feels like none of the existing solutions are that answer. We’re really excited about that, and that’s coming down the pipe pretty soon.

And then the other big basket of work that I’d say is I want to help the community answer a lot of the questions that every developer faces in building applications. It’s very basic things. How should I chunk up my documents? Which embedding model is best for my data, both my data and my query pattern? How many nearest neighbors should I retrieve? 3? 5? 10? Are these nearest neighbors relevant or not? Did I get five of the same thing? Did I get five that are super far away? This is the most basic workflow that every single developer faces when building with these tools and technologies. And there are no good tools or strategies really that exist, at least that have been productized or popularized to help people cross the chasm.

So that’s sort of the first, I would say, task that our applied research team will focus on are those kinds of questions. Going deeper and going longer, again, our goal is to create program memory for AI. I alluded to earlier in the conversation that viewing cosine similarity search as the end of the road for advancements in programmable memory for AI would be extremely foolish. We’re not going to do that. We’re going continue to aggressively research and build more and more powerful approaches to do that well because it matters. We have to cross the chasm here.

Vivek: Really exciting roadmap and I’m excited to see where all this goes. Let’s dive into the lightning round here. Let’s start with the first one. Aside from your own company, what startup or company in general are you most excited about in the AI space, and why them?

Jeff: I’d say Meta. Not a startup, clearly, but I think they’re interestingly counter-positioned to a lot of the other labs. I think that the open-source motion has been extremely bold. And I think that open-source matters here for a few reasons. Number one, it really matters because a lot of the a dvanced retrieval and a lot of the advanced application that you’ll want to build need access to a level of the model where the weights could leak if you’re using a closed-source model. Again, in that goal of going to production, open-source models matter because you want to do that level of surgery, and in order to do that level of surgery, as we currently know and understand, it would be a leaking situation for a closed-source model.

Vivek: They are doing amazing things here. Outside of AI, what do you think is going to be the next greatest source of technological disruption in innovation over the next five years?

Jeff: Does it have to be a contrarian view?

Vivek: Any. Spicy takes or non-spicy takes. Whatever you want.

Jeff: I’ve written about this before. I feel like, in some ways, the two fundamental primitives of advancement are energy and intelligence. You need to have energy to take stuff out of the ground, energy to move it around, and energy to repackage it into other things. You need to have intelligence about what should we take out of the ground, where should we move it and how should we combine it together. And so broadly, technologies that lower the cost and hopefully increase the sustainability of both energy and intelligence are sort of in the direction of flourishing. So obviously, I think AI is marching along the intelligence sector, whether you believe that we’ll get to artificial super intelligence or not, which I’m not sure that I do. And then I think on the energy side, there’s a lot of interesting things happening already around net positive fusion, new geothermal techniques, the list goes on and on. You probably know more about this than I do, but I think that’s pretty exciting too.

Vivek: I was going to say that I think this might be the first episode we have that we talk about LK-99s because it’s probably the first one after this.

Jeff: This might be the first and only episode you talk about LK-99 — now that it’s been disproved, so I guess I’ll take that honor.

Vivek: Exactly. Yeah. We caught the moment in time really, really well. That’s awesome. Okay, last one. What is the most important lesson that you’ve learned over your startup journey but specifically related to talent? You’ve been a co-founder, you’ve been a founder before, you’ve been in this game for a long time. You know that talent is difficult to acquire and retain. Any lessons that you have that you can share with the audience about?

Jeff: I think I’ve really appreciated the idea of go slow to go fast. How that pertains to hiring is I think founders should be extremely picky about the technical bar for engineers. Obviously, or the broad talent bar for other roles, cultural fit, and stuff like EQ. How much of a grownup is this person versus not? And a lot of people are not. I think that that consistently gets underrated. I think what you want to do is build this very small, tight-knit team that shares the same vision and is working in the same direction with a ton of trust. That’s so rare. If you want to make magic, you need to do that. Magic might not still happen, but that’s the best chance that you have to make magic.

And to do that, you have to think in a bit of a weird way about what you’re doing, just dare I say, historical sense. It’s a weird mentality to put on. I think most people don’t want to allow themselves to be that choosy, and they believe I’ll just play the hand I’m dealt. It’s better to hire somebody now than hire somebody three months from now. But if you look at lots of success stories. I think Stripe — it took them over a year to hire their first engineer. I think that even after that, they were still under 10 for a couple of years after that. But that culture, that early DNA and culture of Stripe, even to this day, is paying huge dividends.

Another way to say that is if you believe that what you’re doing has compounding returns across time, you should optimize for the long term. And that’s for both personal health but also things like team composition. Do the thing that you want to be true 10 years from now. I think that’s hard because most founders, including myself, are not very patient. I always want to go faster than we’re going, but I think it’s right. We’ll see if it’s right for us, but I think it’s right.

Vivek: I think what you say is so true, which is coming from the fact that you’re a second-time founder, you’ve been through this before, allows you to take a much longer view. I would think that it’s probably easier for you to step back and take a look at the bigger picture versus eight years ago, your previous startup and that original founding journey. It gives you that perspective, which I think is really helpful for everyone to hear. Jeff, thank you so much for your time. I really appreciate everything, and best of luck with everything at Chroma.

Jeff: Thanks so much. This has been great. Appreciate it.

Coral: Thank you for listening to this week’s episode of Founded & Funded. Please rate and review us wherever you get your podcasts. If you’re interested in learning more about Chroma, visit trychroma.com. That’s T-R-Y-C-H-R-O-M-A.com. If you’re interested in these types of conversations, visit ia40.com/summit to learn more about our IA Summit and request an invite. Thanks again for listening, and tune in in a couple of weeks for our next episode of Founded & Funded with NYSE Group President Lynn Martin.

Today, partner Jon Turow and Cohere CTO and Co-founder Ivan Zhang dive into the world of foundation models for the enterprise. Cohere was founded in 2019, a time when even the most passionate believers did not realize how soon the world would be struck by the capabilities of foundation models. The company announced a $270 million raise in June, which put it in an over $2 billion valuation. And Ivan’s co-founder Aiden Gomez was a co-author of the seminal paper, Attention is All You Need.

Notwithstanding the big numbers, Cohere keeps itself scrappy and hungry, having still only raised a fraction of what others in the space have to date. And in this week’s episode, Ivan shares about his decision not to finish school and instead gain practical experience working in startups and publishing research to basically prove that he could without a degree. He dishes about applying that same renegade spirit to hiring, shares his thoughts about the importance of feedback loops with customers, as well as the differences between fine-tuning and teaching a model how to retrieve the right knowledge, and so much more.

The 2023 Intelligent Applications Summit is happening on October 10th and 11th. If you’re interested, request an invite here.

This transcript was automatically generated and edited for clarity.

Jon: Ivan Zhang, it’s so great to have you here today. Really excited to talk about you and your journey with Cohere.

Ivan: Thanks, Jon.

Jon: Now you folks got started in 2019, you and Aiden and Nick Frost. Ivan, so before we go back in time, let’s start today in 2023. What is Cohere?

Ivan: Cohere is an AI company based in Toronto building foundational models. We’re at about 200 employees now. We have offices in Toronto, SF, and London. Our mission is to accelerate the next leap in productivity for knowledge workers. That manifests in two ways. So one, we provide a platform where you can access foundational models to build features like summarization, search, and chatbots. And another way is to actually use these models to make employees more productive.

Jon: Let’s rewind all the way back to 2019 or a little bit further. Tell me about your journey into deep learning, and then what was the path from that in Cohere?

Ivan: I like to describe my journey into even tech itself as being a bit of an underdog. A bit more background about how I even got into that position in the first place. So at the time, I knew that I was a builder, and that’s how I learned best. And I wasn’t much of a sit-in-a-classroom-and-absorb-a-lot-of-information kind of guy. I needed to tinker. I needed to get my hands on the technology to learn. So when the opportunity came up to drop out of school to work at my friend’s startup, I obviously did it as a backend and infrastructure engineer. I wanted to expand my skills and learn a bit more. And that’s when I met Aiden Gomez, one of my co-founders now, who was interested in starting an indie research group. We wanted to be independent and do research basically for fun and, in a way, prove that we can do it. And yeah, I got pretty inspired by that, and I thought it would be pretty badass to publish papers as a dropout.

And so we just started working together. And after a few years, we felt like we were ready to start a company. We learned our working styles, and we both got more experience just working and learned how things were done properly. Myself, I was exposed to more and more founders just from the space and seeing how the sausage is made. I felt comfortable with the idea of starting a company.

And so in 2019, I pitched Aiden like, “Hey, why don’t we start something new together?” We tried a few ideas at the time, and they were all AI companies, and it was very difficult. Our first idea was a platform where you upload your neural net, and we’ll compress it and make it more efficient. But I mean, the thing is, nobody was using deep learning in 2019, so there wasn’t much of a market there. But what he had seen within Google was that he invented this thing called transformers, and internally it just proliferated. Every single product team was adopting this architecture for solving language problems, and the improvement gains they were seeing were crazy. Absolutely unbelievable.

Just this one tiny change outpaced 20 years of just heuristical engineering. And so we thought it was really cool, and we saw the potential of this technology like, hey, we can actually use this as a way to help computers interact with humans. So that was one thing. So we saw that, okay, computers being able to understand language was basically solved by birth style models at the time. And then we also saw, oh, GPT-2 came out, and that was sort of a hint at scaling these things up was important, increasing the capacity and also like, wow, these things are very efficient, so we can actually feed a ton of data into it.

And the architectural change of making it decoder only now these things can write. And so the two key pieces were there to actually build a system where it can read and write language. And we thought that was quite exciting, and we decided to quit our jobs and bring Nick along as well to build this company. And at the time, we had no idea what the product was going to be. We were just so excited about the idea of making computers understand language and talk to us. And that’s how I got into deep learning. And being an outsider doing deep learning almost gave me so much energy to invest all my after-work time every day, working till 3:00 AM just managing experiments and making the code base a little bit better. It was quite an exciting project.

Jon: Were a lot of people doing research outside universities and corporate labs at the time?

Ivan: So at the time, no. And I think a lot of listeners might know this group called EleutherAI who started a bit after us. We were For AI — and how we approached it was a bit more closed off. We would be open to applicants, but we would be very selective in who’s actually doing research and who we’re getting compute to. I think Eleuther had a different strategy where they were way more open about who was collaborating, and they were also doing open research. But at the time, basically, nobody was doing research outside of labs. And one shout-out I want to give was GCP’s Research Cloud program made this possible. They gave researchers like us free access to TPUs. In fact, we were one of the first users, first power users, of TPUs outside of Google. And that’s how we ran basically all our experiments.

Jon: How do you think the renegade spirit that you had in those early days with Aiden reflects in the way Cohere operates today?

Ivan: We hire differently. We look for people like us who are from very different backgrounds, interested in the field and want to make a big impact. We’ve brought For AI into Cohere as a way to keep the effort going and provide people with unconventional backgrounds a path into research. And what we learned with For AI is that, hey, you don’t need this perfect background of doing all this research at Fair or DeepMind or Google to make impactful work happen. Some of the papers we published were with people doing research for the first time, and they brought some interesting ideas from whatever field they were coming from, and that made a big impact. So we took that philosophy in how we built the early team at Cohere. We didn’t look for the marquee brand name 10-years-at-Google brain, sort of talent. We found folks who are very clearly builders, and they’re interested in the field. We gave them a chance, and a lot of them have paid off.

So I think even though Cohere is our first official startup, Aiden and I learned a lot from that For AI experience. We’re also way more risk-tolerant, I think. The culture is also very playful in that sense. So we like to do a lot of exploring on the technical front. We do word it that way. We’re playing with the technology to find breakthroughs, and we’re very practical. When you’re running a research lab with basically no money, you have to be very practical about what you can and can’t do. And so we’ve spent some time in our engineering to make sure our tools and platforms are working well for our researchers.

Jon: So when you started Cohere, was it clear you were going to provide foundation models as a service?

Ivan: No, at the time, we weren’t sure what to do with a thing that can talk back to you. It’s such a bizarre experience. I think anyone who tried GPT-2 at the time has shared the same experience. It’s cool, but we’re not sure what we could do with it. So, we try to build a product as our first project at Cohere, which was an autocomplete that sat in every text box, and we thought we were just going to throw ads on it and start making money. But I think we were very naive at the time on how difficult the front end would be for something like that. Because even at the time, the text boxes were coded in all these weird ways. They weren’t just native HTML text boxes anymore. There were weird react components, and there were spell components.

So there’s all these edge cases. It just wasn’t our competitive advantage to focus on. So we decided to just rip out the front end and provide an API service. And we knew at some point, well, we’d find some use cases for the generative model. But we thought the embeddings and fine-tuning were also important features to have for developers trying to build something for production. At the time, 99% of NLP use cases required word embeddings, and we knew to take any model to production required fine-tuning some way to customize the product for your problem. So after a few months, we pivoted the Chrome extension into this API platform with generations, embeddings, and fine-tuning.

Jon: If I kind of zoom forward today, what we have as Cohere today is a collection of really high-quality foundation models as a service behind APIs and a particular traction with enterprises, a particular focus on that. Can you talk about what are the most urgent kinds of things that enterprises are doing with foundation models today?

Ivan: Yeah, so at Cohere, we apply an enterprise lens in all we do. Whether that’s the product roadmap, resource allocation or go-to-market. And so they’re very particular product problems for the enterprise, and that’s not endpoint specific. There’s fine tune control of how scalable you want your models to be. Making the precise trade-off between throw put and cost, endpoints that actually make it easy for you to adopt this use case. So stuff like better search summarization, building assistance for your customer support workforce. Stuff like platform management that’s like a non-starter for enterprises if you don’t have that. Being cloud agnostic, with a comprehensive set of deployment options. Whether that’s using the API deploying into your VPC being where the cloud AI products are like SageMaker or serving people on-prem. Beyond just the raw endpoints themselves, we think about the whole enterprise experience and the precise problems they have.

Jon: So, what are some of the misconceptions that CIOs have when they start engaging in these products?

Ivan: I think the biggest one that’s made its new cycles is hallucination. It’s totally reasonable to have an issue with these models that are basically making things up from your prompt. I think it’s actually a feature, not a bug. The model is doing its best to complete your request. The solution to hallucination isn’t to, oh, well, we’ll just fine-tune the model with all our company documents. Well, no, no, no. That’s not what you do with your employees either. You don’t make your employees memorize all your internal processes and documents. You actually just give them a tool, your internal document store, they do a search against that tool, find the relevant information, and then they produce an email, an answer, a report, whatever it is. We have to think about models in a very similar way. We don’t need these things to memorize. Instead, we need to teach them how to use our tools to retrieve the right information. So I think retrieval augmented generation is a big deal, and that’s something we’re serving enterprises with.

Jon: When we touch on retrieval augmented generation, RAG, and we talk about things like in-context learning, that’s often juxtaposed against things like fine-tuning, which you also mentioned. How do you guide your customers about when each of those would be more appropriate?

Ivan: I think the nice thing about this version of AI that we know with transformers and LLMs is that it’s actually quite intuitive to how humans learn and what the limits are applying our intuitions about how humans operate and learn. I would say you need to fine-tune if you need to teach the model a new capability. Like a new process, a new function. Whereas if you’re finding that, oh, it’s not accurate in its knowledge, if you just want to give it more knowledge, that’s a RAG problem. We need to teach the model how to retrieve the right knowledge. So basically, fine-tuning for new functions and give it a database for everything else, like knowledge.

Jon: Let’s talk about the limits of feeding data into a model. In the context of a bunch of announcements that have been made recently from large data platforms, there’s kind of a debate in our industry over whether it’s appropriate to feed lots of raw data into an ever-expanding context window or whether we should instead be building up a semantic model inside an LLM and feeding that back to run a deterministic code in some traditional environment.

Ivan: I’d like to clarify that’s not one or the other. I think, obviously, there is a balance based on the product that you’re trying to solve. As for the limits of these models today, I definitely don’t think these models are at the capacity of solving RAG perfectly — or even learning how to use tools. We have more scaling to do from what we’re seeing internally, 52 billion parameters or 10 billion parameters. Those models solve a specific set of problems, but the customers are demanding more and more complex and sophisticated use cases that we actually do need to scale up these models. So I think there’s more scaling work to do to raise the limits, yeah.

Jon: What do you see as the role of proprietary or enterprise, home rolled models in an ensemble together with hosted models from something like Cohere?

Ivan: I think there will always be model builders within enterprise because it’s such an interesting problem to work on. However, I think most users will just go with an out-of-the-box solution. It’s very hard to hire MLEs and keep them motivated. So if enterprises have the option to just pull an API or pull a SageMaker deployment to solve the problem that they actually want to solve, they’re going to do that. So I think it’s not unlike open source software where people pay for services on top, people pay for a managed offering because they don’t want to be experts in managing Postgres. They just want a database to store information. And I think we’re going to see a similar thing with foundational models. People don’t want to build up the internal expertise to pre-train a 200 billion parameter model. They just want to solve the language problem in their product.

Jon: When I think about this from an implementation standpoint, what are the most important kinds of data sources that enterprises want to prioritize?

Ivan: Yeah, so I think it’s less important about what the source is. I think the most important thing is the data feedback discipline. How well does their feedback loop work within their product? Are they able to get feedback from their users relatively quickly and easily and then feed that back to their ML team? I think that gives me more of a signal on whether that company would be successful with LLMs or not.

Jon: Does that have implications for who should be doing the work within the organization, whether it’s a data engineering team or a product service team? Who ends up doing the integration work within the enterprises?

Ivan: Typically, it’s a product owner who’s entrepreneurial enough to try this new technology, and they see the opportunity because they realize, hey, I have this feedback loop that gives me great RLHF data to further enhance my models. The great thing about these models is that they’re so flexible in whatever language format you give it’s quite easy to fine-tune these models.

Jon: What we see in a lot of waves of disruptive new technology is individual developers and teams excited to adopt the technology bottom-up without a lot of top-down controls. But senior executives are rightly concerned about the sensitivity of the data that’s flowing through these models. So what should enterprises know about the security of using foundation models in the enterprise?

Ivan: Yeah, I think with bottoms-up adoption, it’s about education. I mean, they should know that they’re potentially sending a ton of sensitive information to a third party like Microsoft or OpenAI. But the solution isn’t to ban this technology. There’s so much productivity to be gained that we should instead find an alternative. And that’s Cohere’s bread and butter is that we provide a private LLM that can deploy into your VPC on-prem, so that you don’t have these risks of sending us any data. We don’t actually see any of our customer’s data if they’re deploying with those options. So if you can’t ban it, well yeah, you have to provide a better alternative. Right.

Jon: So I’ll do a couple of lightning round questions, Ivan, that we can use to wrap this up. So one, what has been the biggest surprise to you about the developments of the past six months in our field?

Ivan: I’m surprised that the iPhone moment came so early. My timeline for this tech was three to four more years out, but unexpected things happen.

Jon: What would you expect to see happen in the next six months? What are some of your predictions of developments in our field?

Ivan: I think most work in front of a computer will be fundamentally different. I think the barrier of leveraging computers to do your work will be way lower. You don’t have to learn a special language to make computers do what you want. You don’t have to learn a special graphical interface to navigate programs. I think in a couple of years, if you know how to describe what you want to do, the computer will do exactly that, and that’s beyond just generating emails, but also taking a series of actions and doing most of your work.

Jon: Well, Ivan Zhang of Cohere. This has been just so much fun. Thank you so much for spending time on the discussion.

Ivan: Yeah, thanks for having me, Jon. This was very, very fun.

Coral: Thank you for listening to this week’s episode of Founded and Funded. If you’re interested in learning more about Cohere, visit cohere.com. If you’re interested in attending our IA summit, visit ia40.com/summit to request an invite. Thank you again for listening, rate and review the show wherever you get your podcasts, and tune in in a couple of weeks for our next episode of Founded and Funded with Chroma Co-founder and CEO Jeff Huber.

This week we have the pleasure of having former Snowflake CEO Bob Muglia on the show again. Bob is an active investor and sits on the boards of many next-generation data platform companies, and more recently, he launched his first book — “The Datapreneurs.” Given the long history between Bob and Madrona Managing Partner Soma, we had to have Bob joins us again to talk about his new book and dive into the world of data and AI. These two old friends discuss what exactly a datapreneur is and the Arc of Data Innovation concept Bob wrote about in his book. They also talk through how companies can add value with AI through copilots and agents and what white spaces and opportunities there are for entrepreneurs right now — especially when it comes to semantic models.

The 2023 Intelligent Applications Summit is happening on October 10th and 11th. If you’re interested, request an invite here.

This transcript was automatically generated and edited for clarity.

Soma: Hello, everyone. I’m Soma, a managing director here at Madrona. Today I’m really, really excited to have Bob Muglia, a datapreneur himself with a large body of data platform work to his credit across Microsoft, where he was one of the topmost senior executives and then, most recently, he was the former CEO of Snowflake. Bob is also an active investor and sits on the boards of many next-generation data platform and tools companies. Before we launch into our conversation today, Bob, I do want to take this opportunity to congratulate you on publishing your first book, Datapreneurs.

Bob: I appreciate it. It was more work than I anticipated it was going to be.

Soma: But Bob, given your experience and accomplishments in the world of data over the last 35 plus years, I’m not at all surprised to see the focus of your book, which explores the people and critical pivots and technology history that catapulted us into the modern age of computing and AI. Bob, I thought we’ll just jump quickly into a set of questions that I have for you today to start off the conversation. First, how do you define datapreneurs, Bob, and more importantly, how do you see their contribution in building the data economy of today and in the future?

Bob: Well, a datapreneur is simply the concatenation of a data entrepreneur together, and that’s where the term comes from. This realization, I worked on the book with my co-author, Steve Hamm, and when we first started writing the book in 2021, we really didn’t have a specific objective in mind. We didn’t start by saying we’re going to wind up writing a book. I knew I’d had some things to say and communicate, and we were trying to decide the best vehicle to do this, and after Steve and I talked for a little bit, it was obvious there was a narrative that could be had and turned into a book, and so we started outlining the chapters and things.

In that I realized, in that process of those conversations, I realized that throughout my career, even though I was working at a really large company like Microsoft for a good part of it, I was actually working with very entrepreneurial people and teams building largely new products for Microsoft and for the industry and certainly doing things that were very much revolutionary for the industry. I realized that I’d been working with entrepreneurs all along, even though they were at this big company. So that’s where the idea came from and the recognition that the technology that we experience and live with every day is really the culmination of the work of thousands and thousands of people at many, many companies.

It certainly includes the work of some of these great datapreneurs that I highlight in the book, and their accomplishments have really led us to where we are today, so I wanted to highlight that and describe why some of these things are so important — and give us some people and some background on the technology so they have some foundation because obviously, it’s impacting all of us as we see AI being really the topic du jour of 2023 and clearly going to have a huge impact on all of us.

Soma: That’s great, Bob. Having read the book, I can tell you that you referenced a number of datapreneurs that you worked with in the past and ones that you’re working with currently in a variety of, sort of different companies. But first and foremost, Bob, when I think about you, I think of you as a datapreneur, given what you’ve accomplished with data at Microsoft and then more recently at Snowflake and the impact you’re having today with a variety of startups. I see you first and foremost as a datapreneur. But you also worked with many other datapreneurs over the span of your career. How would you categorize the role the datapreneurs have played in your career?

Bob:Well, they’re the source of all inspiration in some senses. I’m not the deep technologist that builds the product and the code. Mostly I was highlighting the people who did that who were actually in there building core parts of the technology and creating some of the revolutionary ideas that have led us to where we are today. My role, as you know so well, since we worked together for many years, my role was really what Microsoft would call a program manager, the modern term is more of a product manager. But in the Microsoft definition of it, it was really more about the product and building the product and defining the product for the customer.

So I’m used to specifying things, talking to customers, understanding their requirements, and then passing those requirements on to the technologists and the architects inside the engineering teams that build things. That’s sort of always been a core part of my role. As a manager and as an executive, a lot of that comes down to leadership principles, running organizations, instilling values into teams, things like that. I see my role as being very different than a lot of these brilliant people that are actually coming up with these incredible ideas. I couldn’t do that, but I have the ability, hopefully, to help provide them with some guidance to help as people are building the products.

Soma: Absolutely, Bob, absolutely. I want to go back to a little bit the Microsoft timeframe, Bob now. If I remember right, you came pretty early into Microsoft. At that time, Microsoft was working on Windows and OS/2 and LAN Manager and starting to work on Windows NT and maybe even the early days of-

Bob: Actually, it was slightly after that. I mean, it really got started in the latter part, right?

Soma: Yeah, yeah.

Bob: But yes. That’s the very beginning, so.

Soma: From your perspective, if you look back in time, when did you feel like, hey, data is going to be a key, key part of the future of the world? When was that moment in time where you felt, aha, there is something here that is going to fundamentally change how the world is going to be operating?

Bob: Well, no matter how much I tried to get away from it, I kept coming back to data or data kept coming back to me in a way. My first job, the technical job while I was still in college, was working for a company in Ann Arbor called Condor Computer, which had a relational and honest to god relational database. It had a joint command. It was not SQL, it was the very earliest days. SQL was just emerging at that point. This was in the late 1970s, early 1980s. It ran on a tiny microcomputer with these massive 8 inch floppies that stored almost nothing and literally 16K of memory, literally 16K of memory. That was my first experience, and that was building applications for companies.

In college, I was focused on communications, and my first role, my job out of college was at ROM Corporation, which built a telecommunication system for business, an internal switch called a PBX. Our term was a CBX. That was ROM’s product. I worked on the team that configured those products. So again, I had a data-focused job, and I was building data-oriented solutions when I was at ROM. It was really that experience at ROM that led to my moving to Microsoft, largely driven by my wife’s desire, our collective desire to move up to the Seattle area, we were in the Bay Area at the time, and her really finding Microsoft as being an incredible company that was emerging back then.

I wound up joining Microsoft as the first technical person on SQL Server. In a way, that really cemented the focus on data throughout the rest of my career. Because SQL Server, the PC version of SQL Server that Microsoft offer, really did revolutionize business for smaller companies. A large part of what we did in the 1990s was bring out both the server products as well as the database products and tools products. Visual Basic was a big part of that. So you had Windows Server and SQL Server and Visual Basic is the front end for a lot of those early applications. That’s what automated the dentist offices of the world is that software. Some of it’s still running probably, which is probably not a good thing, but that really changed the way people worked with information.

Soma: That is cool. That’s cool. I’m going back and forth in time here, but I want to sort of revisit what happened in the 2017 timeframe. That is when we connected up upon, hey, Snowflake is coming along, is starting to make some revenue, has got some customers, and starting to realize the power of what Snowflake could be kind of thing. First of all, thank you for giving us the opportunity to invest in Snowflake at that time and be on the journey with you.

But one thing that I remember from one of our earlier conversations, Bob, is I was thinking about, Hey, how much should we invest? Because remember, we are an early-stage investment firm, but I remember you telling me, “Hey, Soma, you’re going to make money in this deal. I can’t tell you how much, but you’re going to make some money.” I was thinking, “Hey, how much money am I going to make?” Because the valuation was a little higher than what we were normally seeing at the time.

Bob: Seemed rich at the time.

Soma: Yeah, it seemed rich at the time. In hindsight, I would say, “Hey, you just foreshadowed what the world was going to move toward kind of thing.” But I did not imagine, and I’ll be the first to tell you, I did not imagine what kind of a trajectory Snowflake could have at that time. In fact, so much so that when Snowflake became a public company or went through the IPO process, ended up being the biggest software IPO in history. How did you see the trajectory, and how did you feel about Snowflake becoming the largest IPO kind of thing? But more importantly, what does that signal in terms of the importance of the data cloud platform to the future of the computing world?

Bob: It was really great working with you in the early days. That was really when we were looking at opening an office in Bellevue and bringing in talent in the Seattle area. That was driven by our desire to move to Azure as a platform, as a second platform. We were on AWS already, and we wanted to add Azure as another choice for customers. The realization that there was nobody in the Snowflake team that could work with Azure down in San Mateo, California, we needed some people up north to help that. Obviously, this has worked out very well. I knew the company was going to do very well. It certainly exceeded my expectations in that regard, and the IPO was extremely successful.

I would say it was really successful from this perspective, but for public investors, I have a little bit of concern because the stock went way high, and it’s now still trading below that amount, which is definitely hard for public investors. But it’s been incredibly successful, and I knew it would do well. How well, I didn’t know. The reality of it was the reason it’s been successful is because it solved a problem that was not solved before, and that was how do you scale your databases for analytics to be able to consume and work with all the data you wanted to at the same time and make that information available to all your end users.

Up to that point, the technology before Snowflake really didn’t enable that, and Snowflake was a breakthrough product in the sense that it really was the first to enable a general-purpose SQL database that allowed for unlimited scale. I think that has been amazingly valuable because people are working with data and information. It’s only getting more valuable over time as we begin to find more and more uses for data, and, certainly, the emergence of these large language models and artificial intelligence that is obviously of great interest right now, that raises the importance of data even further. So I think all of these things are important.

Now, Snowflake, as I’ve said a number of times, I mean, I see Snowflake as being in three somewhat different businesses. They’re in the sort of more traditional data analytics, data warehouse business. They’re in the data sharing business and have now established a very strong position in helping companies access data that is outside of their organization and make sure the data can be appropriately shared within the organization. Then more recently, they’ve been really focusing on an application platform, a coherent application platform, the so-called data cloud that provides a set of coherent services for people that are building these next-generation intelligent applications, AI applications, data applications, call them what you want, and then enabling those to run anywhere that Snowflake runs.

They’ve emerged as an important player in the application market, and that’s a fairly new space for them, but it’s going to become increasingly important as we begin to build application services, these intelligent application services, that take action on our behalf. The fundamental distinction is, is that a typical business application acts on behalf of a direct request from an end user, from a person. These class of applications take actions based on information that’s coming into them, and they can do things on their own. So it’s a whole different way of working than creating different business process, but I think it’s going to become a very big part of what people are doing in the next five years.

Soma: The thing that was interesting to me, Bob, as I thought about Snowflake and the Snowflake’s journey kind of thing, you literally have five what I call data cloud platform vendors of some scale and consequence in the world, right? You got the three hyperscalers, the cloud infrastructure guys in Microsoft, Amazon, and Google, and then you got Snowflake and Databricks. I don’t know that I would’ve predicted even, say eight years ago, that there’s going to be five cloud data platform at scale vendors kind of thing, right?

So it’s just fantastic to see how the hyperscalers obviously, that’s not a surprise that they are sort of the meaningful players here, but the fact that Snowflake and Databricks have been able to come to where they are today, it’s just fantastic to see the innovation and how broad the space is and the opportunities ahead in terms of how this can fundamentally change what I call another world of computing and by extension all applications and everything else that people do on that.

I want to come to your book now for a second. There was a great thing that you talked about in your book called the arc of data innovation. To me, now, I was excited when I sort of read through that because it sort of helped me think about, “Hey, how do we visualize that technology progress while making predictions, knowledge predictions, or what could happen in the future kind of thing?” Can you talk a little bit about how you came up with that and how that concept came to life in the book?

Bob: It was always, in a sense, the central concept of the book. It had been present from the earliest versions of it in some form. The idea is that there has been an acceleration of progress over time as technology has continued to improve. That’s why I drew it as an arc because the line represents the speed of progress, and that’s increased as the decades have gone by. The idea of it was to identify both the key data types, the key types of data that were introduced. People don’t always think about it. There’s structured data, there’s text data, there’s semi-structured data, and then there’s what people often call unstructured, which I would call complex data, and that’s video and audio. It really has structure associated with it. It’s just that the structure is so complicated that we tend to think of it as unstructured.

Those sources are now real sources of data that they never were before in the past. That idea of that progressive evolution was always present. The interesting thing is that the finish of it changed while I was writing the book because when I started writing the book, the apex of it was the data economy and the idea that data is a central part of what we do in our business. What I recognized was that as I was writing the book and I saw the explosion of what’s happening in the AI space, I realized that the horizons that I had thought in my head for when some of this intelligent technology was going to become available and useful and really achieve some major milestones like artificial general intelligence, and potentially super intelligence, those were going to happen in a much shorter horizon than I expected.

If you asked me in 2021, when would we have AGI, I would’ve said 2100, and now I will say 2030, thereabouts. That’s a pretty dramatic change. Frankly, it was a breathtaking realization for me because I had always believed that we would develop this intelligent software, that that would come, but I just didn’t think it was going to happen in my lifetime, and now I realize it is. The implications to all of us are just really profound. So the arc of data innovation really now goes to super intelligence and potentially a technological singularity, which is just really a continued acceleration of progress largely faster than human speed. I think it’s going to happen. It’s certainly something that, in general, I think it’ll be very positive, but I know it’s also a bit scary at the same time. I think it is not 50 years away, I think it’s probably a lot less than that.

Soma: Yeah, I think this notion of predicting when you’re going to see major inflection points in technology, I think it’s anybody’s guess kind of thing, but it’s one of those things where sometimes we overestimate, and sometimes we underestimate. I actually remember when I was still at Microsoft back in the 2012, 2013 timeframe, if you talk to any of the traditional automobile companies, “Hey, when are you going to have a truly self-driving car?” They were all talking about 2030, 2035, 2040. Then for a while there, now with the self-driving progress that companies like Tesla made, everybody thought, “Hey, in the next three years, it’s going to happen.” The reality is it’s somewhere in between, right? But the rate of innovation is the one that I think I’m more strict there about because that sort of signals what is possible and whether it is actually 2021 or 2025 or 2030. We cannot debate that.

Bob: Well, I mean, having played with Teslas and things. From what I could see, the technology was still a few years away. I had thought, and this is just getting reinforced in the past set of months, that the 2030s would be the era of robotics and that we’re going to see another explosion of innovation in autonomous robotic devices that are part of our lives in a variety of ways, ultimately resulting in humanoid robots that have intelligence and can do things on our behalf. The interesting thing about a car was the autonomous car, which was I think about an Uber ride, and every Uber ride that I’ve ever been in ends with a brief conversation with a driver about where to drop you off.

I was like, “Well, how are you going to do that with a machine?” Now I realize you can just tell it. Now you’ll be able to tell the robotic car where to drop you off. So I mean, some of the problems that seemed very intractable, now that English is a language that computers can understand, I mean, a whole bunch of problems that were there before disappear and become at least potential to solve.

Soma: That is very true. That’s a part of NLP or natural language processing. I guess you’re right. Hey, you can talk to a machine. Now pretty soon, I think I know you can already touch a device with touchscreens. You can now speak to a device. The interaction — the more you can interact with the device or with a machine or a system, similar to how you interact with each other, the more natural it’s going to be, and the more people can seamlessly think about computing in an ambient environment as opposed to, hey, I need to walk up to a machine, or I need to walk up to a device, as opposed to, I’m going through my life, I’m doing my job, I’m sort of having fun, and I can leverage technology in my natural workflow, so to speak.

I think that’s an exciting world that I’m looking forward to. Like you said, for a while there, it wasn’t clear whether it’s going to happen in our lifetime or not, but I think the rate of innovation, there is a fond hope that could happen well within our lifetime. So we’ll see how it goes.

Bob: Even for old people like us, even for old people like us, but for the youngsters in there, I think they’re going to see a whole lot. For people who are in the earlier stages of their career, they’re going to see a massive amount of change. I mean, think about the fact that these devices have intelligence in them. We’ve always been able to put rules, and we’ve always been able to program rules in a computer, but now you can actually take and have intelligence associated with decision-making. It’s really extraordinary. It’s an extraordinary innovation.

Soma: That’s true. Bob, you can’t talk to a company today without them telling you why they’re an AI company. Every company says that.

Bob: You can’t, you can’t. That’s for sure.

Soma: But here is the thing that, and we sort of have this taxonomy, at least a mental model that says, “Hey, there are going to be AI-native companies, and then there are going to be AI-enhanced companies.” Which means most companies that exist today have to think about how to incorporate AI in fundamentally core parts of what they are doing as a business or as a company kind of thing. The challenge, though, is all of these companies have an existing what I call data architecture, how they think about data, how they deal with data, how they process data, how they take advantage of data kind of thing. As AI becomes more and more central and core to every business and every company, how should an existing company, where should they even start about, hey, I want to be a AI-first company, but I do have a way of doing things today with data. What should I do? Any sort of thoughts on what you tell companies?

Bob: One of the really interesting things about this technology is that it really can be additive to existing applications and add value. I think Microsoft demonstrated that with this co-pilot approach, which I think will be the predominant, at least initial, way in which AI will be incorporated into existing applications, which is these agents that sort of support you in your efforts to do things. I think most companies who have an existing product can take and modify that product to put in one of these co-pilots that can build off of the knowledge basis that it has, and fundamentally, essentially just be a much more effective help system to help you work and navigate through the product you have.

If you don’t have an existing product but you have a business that you’re running, one of the interesting things that now there are opportunities to build applications that couldn’t be built before. Here I always just say, “What is the domain expertise that you bring?” Because it’s now possible to effectively take that domain expertise and leverage the intelligence in these large language models to bottle effectively the expertise you have and put it inside the application so that the knowledge and behavior and processes that you created can now be run and executed by the machine versus by people that previously performed these actions.

So there’s opportunities across the board to do that if you’re in an existing business, whether you have an existing application where you can obviously take the knowledge base and everything you did and directly incorporate it or whether you’re building something new. Then I’d say, just to continue along the lines of the different types of apps, you say there’s just new kinds of applications that you couldn’t build before that now, for the first time, can be done.

Soma: That is true. I fundamentally believe that AI is going to permeate every industry, every company, every walk of human life kind of thing, okay? But having said that, if you think about the different verticals or the different industries, do you think some are more naturally attuned to having the most innovation and positive impact by AI? Or do you think it’s just going to be across the board?

Bob: I think it’s pretty horizontal. I mean, like anything, there’ll be uneven distribution of it. Certainly, different industries will have different speeds of adoption somewhat based on the regulatory challenges they face. Frankly, in some industries, take the medical industry for example, we have to make sure that the technology is matured to the point where it’s working correctly. I mean, if AI hallucinates and writes a poem incorrectly or something, or it puts something in a poem, it’s one thing, but if it misreads a doctor’s notes and that results in a patient getting the wrong drug, that would be very bad.

So you want to make sure you have very high degrees of accuracy in certain scenarios. So that will take a little bit more time, but every domain will be impacted. It will certainly impact a lot of people in their careers and jobs because the more mundane sort of repetitive tasks, the tasks that tend to be repeated, those are the ones that are historically been the providence of people. Those are the sorts of tasks that will wind up being automated first.

Soma: One of the things that I really enjoy about your book is in addition to talking about data, data technology, the modern data stack, what’s happening in the world of data kind of thing, you go through what I call case studies or specific examples of companies and the kinds of technologies that they’re building. You talk about many, many different companies developing solutions for what I broadly refer to as the modern data stack. But having said that, do you think there are still white spaces or big problems that need to be solved? Particularly if you’re talking to the next generation of startup founders, what would you tell them about, Hey, what opportunities exist in the modern data stack? Or do you think that’s all being solved, and we should just move on?

Bob: Well, it’s not all being solved. Let me just start by saying that. The modern data stack has made great progress. As you mentioned earlier, there are really five different vendors providing solutions. It’s good to see those solutions maturing over time. They all come from different places. They all start from a different place, but they’re all kind of building the same product now. They’re all heading towards similar products that they’re creating. Although there’s strengths and weaknesses of the vendors, like I say, largely associated with their history.

There are definitely open spots in the modern data stack that still need to be filled, largely around compliance and management. Those are areas where I think there’s a lot of challenges still. Certainly, access control and managing coherent data access control continues to be a significant problem that I don’t feel is well solved. I think one of the biggest areas of future potential is building and creating what people talk about is semantic models for different things. In particular, I think it’s interesting to think about the semantic model for a business. If you look at any company’s business product, any organization’s business, and their process associated with executing their business, it’s the knowledge of how that work is scattered in different spots.

It exists somewhat in the applications they run. It exists somewhat in the analysis things that people do using tools like Tableau and Power BI. It exists in the heads of a bunch of people. It exists in documentation and probably maybe more than anything in Slack messages. It’s all over the place, and it’s not very well defined. In order for these language models, these agents, to behave in a way that is consistent with what we want, they need to understand the semantic model of the business. So I think we have to be much more explicit about that because if we want these machines to do things for us, we’re going to have to explain what we want them to do. Today, where would they look? I mean, it’s all over the place.

So I think centralizing that is going to be a fairly major opportunity. I’ve made the statement, I think this is an accurate statement, which is one profession, which has emerged as a really interesting profession in the last few years, which is data engineering. I think that profession’s going to change over the next few years to become business engineering. We’ll begin thinking not just of the way the data is modeled, but of the overall business model and ultimately will derive the data models from it. That’s where this concept of knowledge graphs come in. I believe these things can be very coherent with the large language models.

Soma: Bob, one of the things that you’ve done in the last couple years, at least now, is be an active investor, as I mentioned before, in the next generation of data companies in one way, shape, or form. If you put on your investor hat for a minute, how do you evaluate which company you want to invest in and be a part of kind of thing? What trades do you look for as you make that decision?

Bob: The traditional thing is the best return on your investment, right? That’s probably the traditional focus of most venture capitalists. That’s really not high on my list. I mean, I want something to be successful, obviously, but actually focusing on the actual return is not what I’m caring about as much. I care more about the technology and the impact that the technology is going to have on the industry and the world as a whole. I try and focus my time on companies that I think are doing things that have the potential to have a material change in the way the industry works.

One of the reasons working with infrastructure technologies, it provides the opportunity to do that because they tend to be very horizontal versus just applications which are focused on a given vertical or a given industry. So most of my focus is on horizontals, largely because I’ve always had kind of an infrastructure focus. So in a way, my criteria is about the importance of the technology, the relationship with the entrepreneur and the CEO, and the potential impact for this in the world. That would be where I really put my key focus. To me, the return is a more secondary thing. In fact, one of the questions people always ask is what’s your check size. I’m like, “Really, I follow what you need and what the other investors do.” To me, the important thing is supporting the organization.

Soma: I actually agree with what you said, Bob, about what you look for kind of thing. Because as much as the venture capital industry is about making sure that we have the right returns on our investment, not just for us but also for our LPs kind of thing, to me, focusing on the team and the impact automatically will lead you to great reports of goal along the way kind of thing, right? So I-

Bob: I think they’re very related. I mean, they’re very related together, but it’s just a question of is it a spreadsheet-driven decision or is it more of an impact-driven decision. It’s the latter.

Soma: Got it. Bob, before we wrap up, I want to ask you one final question. If you were to start a company today, what kind of an AI company would you start? I’m assuming it’ll be something to do with data and AI, so I’m presuming that but what would you want to do in today’s landscape?

Bob: I will do this serendipitously through the entrepreneurs that I work with, and I am sort of doing that in a way, but it’s really what I described about that business semantic model, that to me is building high-level semantic models of everything is of interest. I think about this one sometimes, but I haven’t braved myself to go into it yet, which is the semantic models associated with legal contracts and laws because I think there’s a discipline called computational law where you can apply computational approaches to legal problems and really a contract is a program. It’s a program that is interpreted by lawyers and executed by people.

I don’t think that’s the way things are going to work for very long, because, in a fairly short time period, machines will also interpret these contracts, and much of the execution of the tasks will be done by the machines. So I think we’re seeing a transition of high-level semantic concepts into executable form that can be operationalized within the business. So it would be along those lines that I would put my focus right now. Whether the domain is management of the IT infrastructure, which is an interesting domain, or whether it’s a different domain like law, all of those things are potential applications.

Soma: Bob, as always, it was a fun conversation. Thank you for taking the time to be here with us today and really enjoyed the conversation. Thank you.

Bob: Thanks so much. It’s always great talking to you.

Coral: Thank you for listening to this week’s episode of Founded and Funded. If you’re interested in reading Datapreneurs, visit, www.thedataepreneurs.com. If you’re interested in attending our IA summit, visit ia40.com/summit. Thank you again for listening, and tune in in a couple of weeks for our next episode with Cohere’s Ivan Zhang.

This week, investor Aseem Datar is possibly making app dev and product development history with his guests. We’ve got former Tableau CEO Mark Nelson and James Phillips, the former president of Microsoft who drove the creation of many of the company’s most successful products, including of course, Tableau’s number one competitor — Power BI. James was also just named the VP of Google Cloud, post recording this podcast! Mark and James are both part of the Madrona family, as venture partner and strategic director respectively. And today, we have the rare treat of them together sharing their learnings and experiences about modern app development, data and analytics, product-led growth, and scaling in the face of stiff competition. You won’t want to miss this.

This transcript was automatically generated and edited for clarity.

Aseem: Hey, everybody, I’m excited to be here. Actually, I’m super excited today. I have the unique pleasure of welcoming James Phillips and Mark Nelson as our esteemed guests. And you both led and built products that are truly unique and market-defining, so this is going to be an exciting conversation. And before we dive in maybe let me just hand over to each of you for a quick background and intro, and then we can go from there. James, you want to go first?

James: Sure. Most recently, I spent 10 years at Microsoft and developed something we’ll talk about today, a product called Power BI, along with a number of other things. It’s super cool to be on this with Mark, who was running my primary competitor at the time. Prior to Microsoft, I founded a number of companies, Couchbase, Akimbi, a company called Fifth Generation Systems. Spent some time at large organizations as well, Intel, Synopsys, and did a little bit of a detour, spent two years as an investment banker helping technology companies go public and execute mergers and acquisitions.

Aseem: That’s awesome. I’m glad you hit the compete question head on. Over to you, Mark.

Mark: So I’m Mark Nelson. I’m currently spending my days as a venture partner at Madrona. Prior to that, as you mentioned, I was CEO at Tableau, ran product and engineering before that. And I’m coming off of a long career, which I think just means that I’m old, with a lot of experience in the data space. Started off working for a database company called Informix, which is now a tiny little division of IBM, and then spent 17 years at Oracle, which is where I really kind of grew up, before moving over to be CTO at Concur prior to the acquisition, and then hung around for a while afterward. And then went to Tableau, again, to run product and engineering, and then became CEO sometime after the acquisition.

Also lived through the two biggest software acquisitions in the history of Seattle and so that’s a small claim to fame.

Aseem: So awesome. I’m just amazed at the fact that there’s so much wealth and so much knowledge in this virtual room today, and can’t wait to dive in. So let’s just jump into it.

One of the questions that founders often wonder about is, how should you think of building products from the ground up? What are key items that they should keep in mind, especially when they’re thinking about a new category or value creation in something that’s never been solved before? James, do you have a perspective on that?

James: I do. I think probably the most important thing that one can do is to recognize that putting a product out in the marketplace is your first, I think, real opportunity to get real actual feedback. It’s sort of the ante for learning. And so the principles behind that for me are, get in market as fast as you possibly can, because once you do that, you’ve got the lines of communication open, you can begin learning, you can begin incrementally improving. And I think that pairing that with a system that allows you to incrementally improve and to ship very frequently while listening is the key to taking that feedback that you just effectively paid for, by building a product and learning and turning and burning, if you will, and improving the product as you go.

So fast-to-market, and perhaps as important, fast iterative cycles to continually improve the product because you’re learning once you’re out there.

Aseem: Couldn’t agree more in terms of the faster iteration. Mark, is that something that you guys also had as front and center as you thought about category creation or building products? Or do you have a different perspective on that?

Mark: No, I don’t have a different perspective. I’ll add onto it, and it’s this fine balance of being convicted about the problem that you’re going to solve and the who you’re solving it for, and then the ability to listen as you find out where you were right and where you were wrong. And it’s this fine balance between having this super passionate conviction that you know what you want to build while listening and getting that real feedback. You don’t want to overbalance on either one of those, right? As soon as you stop listening, you’re doomed, and yet, especially for category creation, no one’s done this before, so you have to be convicted that this is a problem that the world wants solved and that you can solve it, just to get across that initial barrier.

And so having that delicate balance and delicate judgment, and then 100% on getting out there fast, iterating, and learning, there’s nothing like having it in the hands of customers to get the good news, bad news, and feedback that you need.

Aseem: James, I know you have a perspective on the five minutes principle. I’d love for the world to hear it. I’ve heard it so many times, but nothing like coming from you. So can you share a little bit of that in terms of how you think about customers and how do you think about value creation for them?

James: Yeah, we had this saying, “Five seconds to sign up and five minutes to wow,” which was shortened to five by five, and it was really sort of a guiding principle for how we built Power BI if we want to talk specifically about that for a moment. The goal was that any user should be able to show up, sign up to use the product within five seconds, and at the end of five minutes if they were asked, “What do you think?” We wanted to hear them say, “Wow, this is amazing.”

And for the first probably year and a half of Power BI, we literally, every single week, ran a 5-by-5 user study where we would invite people in, we would sit them in front of the computer, we would ask them to sign up, and we would ask them to use the product. We started a stopwatch. We stopped it at five minutes, and we asked for their feedback after that period of time, in order to ensure that the ability to discover, ingress, and begin using the product, or at least have an emotional experience with it, was positive. And if we got that right, then we could start working our way down the funnel and driving usage, et cetera. But it really started with that, “Let’s go find users, let’s get them excited, and let’s hook them.”

Aseem: It’s so funny, it reminds me of the early days of Visual Studio and DevDiv. One of the things that’s similar in this vein was the number of clicks it takes for people to sign up back in the day, and if you’re an engineering manager, if you have to pay in terms of headcount, you don’t get a certain amount of headcount in your team if the clicks are more than 10. And so such a powerful principle to abide by, especially as you’re building early on.

Mark, did you have anything in terms of dos and don’ts or principles in the days of building Tableau? And what were some of the goals you set for the team?

Mark: Yeah, for sure. This predates my time at Tableau, so I’ll talk about it, but I’m very familiar with how this story went, and I’d like to think that a lot of what James instituted was a reaction to what Tableau had done in the market, right?

Aseem: Well played, well played.

Mark: Because it was Tableau’s mantra and go-to-market motion, from the beginning, was… Again, we predated what is now called product-led growth, but it was product-led growth. There was no sales team. It was amaze the analyst, right? That as soon as they get it in their hands, they realize this is something they can’t live without. You had to be able to download it, use it immediately and immediately, again to the five seconds to wow, you had to immediately get value from that. You had to immediately see this as something that was beautiful, and you could not live without.

And so there was a maniacal focus. Again, it wasn’t sign up because we started off as a desktop tool. So it was a maniacal focus on download time, a maniacal focus on installation experience, a maniacal focus on what was that first thing that got you going and how beautiful it was and how beautiful that experience was. Because that was the lifeblood of Tableau early on, it was the first X number of years was inside sales. It was all led by the passion that individual analysts had about the product.

And then the other part of the equation was the community that then grew up around that product. That was the go-to-market motion for Tableau for a long time until it grew up.

Aseem: So true that it’s worthy to highlight that both of you built these massive businesses when PLG was not even a thing, or I don’t even think the term was around back then. And so the question that I think most founders have is, what makes PLG the right strategy for founders that are early? I mean, a lot of it is, as a first-time founder or an early founder, you are selling it. You are doing this sale, whether it’s to the enterprise or the SMB you are sitting in on, it’s high touch to a certain extent. At what point does that PLG equation, in today’s terms, come into being? Is it business-specific? Is it industry-specific? How should one think of PLG, and what makes it the right strategy?

Mark: I’m happy to dive in here because Tableau was an early poster child, whatever you want to call it, and then I had the fortune, being part of the Salesforce universe, of seeing Slack come in. And I think the modern definition of PLG really came from Slack and from their motion, and of their freemium into your really paid motion that they pioneered.

And so I’ll say a couple of things. One is, it is not the perfect motion for every product. I’ll go to Concur. Concur is about expense management and governance. That is not product-led, it is not led by individuals. ‘Cause in order to get the product-led growth, it really has to be something where an individual picks up the product, gets enamored of it, and then it grows. There’s viral growth out from that, from that land and expand motion. And there are great products, and it’s a great motion when you can get it, and it’s a great motion especially for small companies, again, ’cause you don’t need a sales team. Well, you don’t need a heavy sales process to go do that. But it is not applicable for every product and every market. It would be lovely if the world worked that way, but it’s just not. Again, Concur was not product-led growth, will not be product-led growth. It is not Concur’s way of selling. You’re selling into a large organization from the top down.

But again, back to the notion of getting your product out there and iterating, when your product is for an individual — when it is that product-led growth — that flywheel spins so fast. Of the advantages, not only that you don’t have to start with the heavy sales motion, it is also that the feedback is immediate. ‘Cause when they abandon it after 30 seconds or after five minutes or after 10 minutes, you’re done, and that feedback is immediate, as opposed to something that takes a long time to set up and get in there. You still get that feedback, but it’s a longer feedback cycle.

And I just want to touch on one more thing you said. No matter product-led growth or not, as a founder, you are the first seller. Always, always, always, always. You are the person who believes, you’re the one that is out there. Whether it’s a six-month heavy enterprise cycle or the five minutes whatever it is, you are still that first person. You’re the first evangelist, you’re the first salesperson, you’re the first everything regardless of what in motion looks like.

James: That’s right. One thing I’d add to that, I think no question that not every product category is a good fit, if you will, for PLG, but I really, really encourage being thoughtful and finding a way to work your way into an organization through some sort of PLG motion if you can. Salesforce…

By the way, we copied Dropbox. That was sort of the model for us with Power BI. Dropbox was a perfect example where it took you no time to start using it and storing your files. It was easy to share links, other people discovered it. You started to grow an audience, and at some point in time, you tripped over a paywall where you’d stored so much that you had to start paying, and by then, you were sort of hooked. And this notion that you’re delivering value and delivering value and delivering value until you’ve reached a point where you can really ask for revenue, I think, is important.

Salesforce was a company that most of us would believe today as sort of an enterprise sale, but if you look at the way it was originally adopted, if you go back to the early 2000s, it was adopted by individual salespeople as a contact manager initially. It was very, very easy for people to sign up and start using it and storing their contacts. They thought it was cool, and it started to go viral in some ways until it became the solution, if you will, for the organization all up. And so even in this case where you’ve got a business application that ostensibly is sort of an org adoption, finding a way to get individuals at least to raise their hand so that you can then follow up with a land-and-expand motion, I think, is worthy of consideration.

Aseem: Yeah, I think there’s definitely one thing I want to highlight there. And as I meet a lot of companies and founders, a lot of them are focused on the friction-free experience to sign up, but there’s an important element of a friction-free experience to also expand or drive or increase usage.

One of our partners, Karan, who was early and who’s sort of a resident expert in PLG, often talks about this thing of, take the floodgates off the product and let people use and fall in love with the product. And going back to James, what you were saying, which is once you experience the product in its fullness and its glamour and its glitz, and you start using all the features and bells and whistles, then I think it’s a little bit of like… You, as a founder, can then start to be in a position where revenue just becomes second nature. So it’s a super interesting point that you guys both made, and I want the founders and the entrepreneurs, and the early builders to take that away for sure.

They say hindsight is always 20/20, so I did want to ask you that question of what is something you wish you would’ve known if you were still running these companies or divisions? Is there something that we don’t know that would be worthwhile discussing on that piece?

James: I can go first. I’m not that far out, so I’m not sure I’ve learned my lessons yet. But I will say just taking a step back has been really valuable. You always try when you’re in the role to make sure that you’re taking a step back and looking at what the whole world actually looks like. It’s so easy, especially in these big businesses, to get lost in the minutiae. There’s always a crisis, there’s always something on fire, and it’s easy to get lost in that. Consciously trying to take… I tried when I was in the role, but now I’ve been out of the role for a few months, wow. Should have spent more time really looking over the whole landscape.

So the Power BI that the world knows, powerbi.com, is really Power BI V2. There was a Power BI prior to powerbi.com, and it was a set of plugins, actually, for Microsoft Excel. So there was something called Power View, Power Pivot, Power Query. It was sort of the Power family of capabilities that were add-ins to Excel, and you would share these artifacts just like you would share an Excel document. There was no SaaS service. And the learning from that was that, notwithstanding that Microsoft Excel is universally used, the ability to get people to discover add-ins and to have the right version of Excel, and to download and install that into Excel and then to share these artifacts, was really, really awkward. And it wasn’t a success, ultimately.

And it wasn’t until we stepped back and realized that this sort of SaaS model where you can really provide this complete and total five seconds to sign up, five minutes to wow experience, where you don’t put on the user this burden of collecting all the pieces and bringing them together in order to create an experience, was a learning for the team and one that I wish perhaps that the team had learned earlier. But you learn, and you move on.

Aseem: Yeah I mean, goes back to the signup or the usage tax, take it down as much as possible to drive more adoption. We talked about this a little bit as we introduced each one of you, but I’d like to ask you both this question of, what did you think of your biggest competitor at that time? I’ll let you pick who the competition is, but hint, hint, they just might be on this call. What did you think they did well, and what was sort of the thing that kept you up at night? And James, let’s start with you first — if that’s okay.

James: Sure. So look, Tableau was the competitor, full stop. Tableau and Qlik, but really Tableau. Microsoft, for a very long time, had been a leader in enterprise BI. It had some wonderful products, very large customers, and had built a very successful business, but missed this whole self-service BI opportunity that Tableau, and Qlik to perhaps a slightly lesser extent, had driven in the marketplace.

So we were in some ways a little bit fortunate in that we were fast following into a market that clearly had legs. Tableau proved that the world desperately wanted and needed the ability to engage in business intelligence, the ability to analyze data, without being completely dependent on an IT organization to make it possible. This ability to give analysts the power to go connect to data and get value from data was clearly a latent need in the marketplace that had been filled, and that we had missed.

So we thought that they did two things very well, and Mark hit on both of these earlier. Number one, they built an experience that allowed you to very, very quickly, easily, with minimum friction go get the product and start using it and getting value from it without a bunch of handholding, with minimized friction. And two, they built an enormous ecosystem of fans. They built an ecosystem of product experts, they built a community, they built love around the product. And those two things we certainly wanted to mimic, and then we wanted to bring our own unique view on what that market could be and should be.

I always say that we took Tableau and Tableau-ed Tableau on the server side. Tableau had this desktop offering where you could download and begin doing the analytics and create these artifacts, but if you wanted to share those insights, you needed to have a Tableau server set up, and now you’re back to have an IT involved, and having servers and maintaining infrastructure. And so the ability to allow anyone to sign up very quickly and begin sharing those insights, to have an entire organization literally in a matter of minutes have access to the insights through this organized SaaS capability, was where we innovated to try and out-Tableau Tableau, if you will. But we certainly took our cues from all the things that they did right.

Aseem: That’s wonderful. Mark, flipping to you.

Mark: Well, it’s the flip side of the coin. I think James hit on a lot of the exact right points on how we saw the world as well, and how we saw Power BI, which, no surprise, Power BI was the competitor, right? Yes, there was Qlik, and there was MicroStrategy, and there were a few others hanging around, but it is a duopoly of Tableau and Power BI out there in the market.

And I agree with James on what Power BI did really well. We throw all these things in, and we say, “This is business intelligence,” or, “This is analytics,” right? There are variants on this in where the products come together, where there’s overlap. Because Tableau really started as a tool for the analyst to explore data. It was not a dashboard creation. Dashboards weren’t even possible until 10 years into the existence of the product.

And what Power BI did very well was pick up on that and pick it up from that point going forward, it was a very good way to create dashboards and disseminate dashboards, where that was the center of the product, where that was… 10 years into Tableau’s journey the first dashboard was created. It was an ancillary effect, but also where it became viral, where that became something more than was just in the hair and the hands of the analysts.

And I think part of Tableau’s magic is we expanded the definition of analysts hugely, but it still was a minority in the organization. It was dashboards that could be consumed by anyone in the organization that really helped that spread, and that was the sweet spot where Power BI came in. And it became a very, very good dashboard creation tool and dashboard dissemination. And getting the advantage of coming second is you didn’t have to worry about the desktop, didn’t have to worry about being on-prem, it was at that point in time where you could focus on a SaaS service that provided an easier experience for sure, because no one actually likes installing software, shockingly. If you can avoid installing software, it’s just better.

And then the other huge thing that kept us up at night always was Microsoft as a whole, right? It wasn’t just us against Power BI, it was the mammoth thing that was and is Microsoft, right? A go-to-market motion and power that is not just the Microsoft sales team but the biggest reseller network on the planet. The way that it gets in with CIOs and with IT departments, right? I mean, there are Microsoft shops. And then our biggest worry were the deals not where we lost head-to-head, ’cause we did very well there, it was where we weren’t even in the deal ’cause they were just like, “Well we bought an E5 license for Office 365, we have analytics, we’re done.” And there was never a Tableau discussion. That was always a worry when competing with not just Power BI but with Microsoft, because of the reach that Microsoft has.

Aseem: And one of the things that our founders often start to think about, and sometimes pretty early, is this whole notion of GTM and channel. They look at the Microsofts, the behemoths of the world, and they’re like, “Oh, I should just go work with a channel partner,” pretty early. But I think therein sometimes lies the fine balance of — get to true product market fit. Get your first big wins and then get to channel. Versus — starting to think of channel pretty early. Because otherwise you just get spread too thin. And that’s one of the things that we always try to be super cognizant about when we work with teams we invest in and be like, “Hey, what’s the right time for that channel mix to really start lighting up?”

We can go on and on on this topic, but I wanted to shift gears. So much great stuff to talk about. We can’t leave this without talking about gen AI, all that’s happening in the generative AI space with LLMs. How do you folks think that the data analytics category is evolving with these new advances? What should entrepreneurs keep in mind in terms of creating value or building on top of the goodness that gen AI has to offer?

Mark: I’m happy to dive in. I think it’s a really exciting time. Gen AI is the latest great user interface to come on top of this, right? Now finally, there’s a model that really looks like and feels like human language, and can give you a user interface that feels like that because it is… As you work with data, data’s only useful if you can put it into a model that helps you understand the world. That’s it. Data by itself is not useful. One of my favorite sayings is, “Every model’s wrong, some models are useful.” And it is finding those models of the world that are good that help you understand.

And I think what has really been eye-opening with gen AI and the LLMs that are coming out is how good and how powerful that model of the world is, based on language and what… understands. And I think it’s going to have a big effect, and what I’m really excited about is especially the messy part of data, right? Because the messy part of data is not like the output that you see coming out of Tableau. That’s the beautiful, amazing, magical part, that’s the tip of the iceberg. The ugly underbelly is everything that it took to get that data to that point, where that beautiful understanding could come out of.

And I think LLMs have a real chance of helping that process hugely because this has always been something that — well, not always, but now that we’re in this world where we have so much data, machines can do that better than humans. And we need that help ’cause the human intellect, we’re just swimming in these seas of data, and there’s no way that humans are going to sort through all of that. I firmly believe the human intellect will always be the last mile of that, back to that understanding of which models are right and which models are wrong. Where the model is telling you your airplane’s flying 10 feet below the ground, that model’s wrong, and that’s where the human comes in. That’s not going to go away, but the power that you can put at a human intellect’s fingertips is going to increase hugely with these things, because of what’s there.

There is always this trick, though, especially as you get with data and analytics, why it’s fraught with a little bit of danger is that the hallucinations and everything else on what genAI still gets wrong. When you ask for profit, you expect to get profit, and you expect the right answer. This is not a Google search where it’s “Give me four answers, and I’ll pick the right one.” This is data analytics. You’re looking for ground truth, you’re trying to get to it. So it’s going to be fascinating to see how these things, like amazingly powerful, amazing ability to collate so much data and really put sense to it that’s hard for a human mind at that size and scale to do, but then also get to a point where it’s correct enough, how you get the human in the loop, how you really get to answers that you know and believe to be true is going to be really interesting in the next couple of years.

James: I completely agree. I think the thing that we’re going to have to get right, and I think where all the value is going to come from, is getting that human-machine interface right. Where you can arm a human to take advantage of the technology while building confidence in the technology in the human. And the user experience, I think, is where the most work needs to be done.

I think in some ways, as you were talking, Mark, I thought about Trifacta, Paxata, and some of the work that was being done just a couple of years ago where we were trying to classify and do entity extraction and understand the data that you were trying to cleanse, and sort of move into a place where you could get value from it. I think that’s an area where we’ll see huge advances as a result of these large language models, and I’m excited about it because I do think that one of the biggest barriers to getting value from data is cleaning the data and understanding the data and getting it staged to have value extracted from it.

I also think one of the biggest challenges that we’ve had, and one of the holy grails, if you will, is enterprise search. For years and decades, we’ve wanted to unlock the data in the enterprise so that you could ask about things that matter, but the problem’s always been very difficult because the data scale, certainly relative to, say, the internet, is de minimis. And it’s really hard to train models when you’ve got small amounts of data on a relative basis. And I think what we’re seeing now is the ability to take incredibly large volumes of data and make it applicable even to smaller volumes of data, and I think that’s going to potentially unlock the ability to truly, finally, perhaps for the first time get your arms around the small, quote-unquote, data that is your unique data in the enterprise. And I think that’s incredibly exciting.

Aseem: One thing that I keep wondering, James, you talked about plugins, and you talked about integrations into Excel. And I was at Microsoft Build recently, and one of the things I noticed was the plugins are back. And a little bit of these co-pilots are nothing but things sitting on your shoulder advising you what to do, whether it’s changing your settings in the OS or looking at data differently, or enterprise search. And I can’t just wait to see where this goes in terms of, are these still point solutions, are these features, are these platforms? And that’s a space that’s very exciting, just personally, from a productivity standpoint.

You guys both ran massive teams, you built large organizations, but there’s a critical element in every founder’s mind today as they build these companies around hiring talent, hiring great talent, hiring for today, hiring for scale. One question that often hits founders is, when does that shift happen? Should I hire ahead, should I hire for today? What do you folks think is the right way to think about it?

Mark: Delicate judgment. So neither of those answers is completely true. I do love the phrase from Amazon that is, one-way doors, right? Don’t go through any doors that you can’t go back through the other way. Beyond that, of course, you have to build for today to some degree, right? If you’re building for scale before you have scale, that’s a good recipe for never getting to scale, right? You will have so many things to do, you need to worry about. And this is not just organization, this is also for product building, right? Are you going to make non-scalable choices in your product early on? Yes, of course, you are. And you should, because if you don’t, you’re not going to get the feedback, and you’re going to build a really scalable product that nobody wants to use. And so you’re just going to have to feel your way through that on, where am I really at, where am I going, where am I going to be? But focusing on the problems that you have today, don’t worry about, “Well, but three years from now, if it all goes well, I’m going to be here, and I need to build for that.” Again, just don’t paint yourself into any corners, but beyond that, live for today and make choices.

And I always tell people, the biggest ability you have to have in your organization, and it is culture building, which I would say is more important than are you building the exact thing for today or tomorrow? It is building this culture and building a culture, most importantly, that can learn and be empirical about itself. Because, of course, you’re going to make wrong decisions, of course, you’re going to make decisions that don’t scale. That’s okay. You have to make those decisions in order to live today and live to see the next day.

The important part is being able to be empirical about, “Yeah, I sweated blood over that decision a year ago, and now we’ve grown three times, and it’s wrong, and I’ve got to move on.” As long as you can do that and continue to do that and continue to evolve. Because if you’re healthy, again, the decisions you’re making today you hope are wrong three years from now because you hope to be seeing a whole nother size of scale and problem when you get there, as long as you didn’t paint yourself in a corner and as long as you’re very rational and empirical about being able to assess where you’re at and what your problems are and the strengths, and not get married to, “But we grew so fast through here, that must be the right answer.” It was the right answer then, and that’s awesome. The right answer for today is probably different if your business is healthy and growing and changing.

James: Yeah, the one thing I would add very specifically is to be careful about bringing on a big go-to-market machine too early. It’s amazing how many organizations that I’ve seen, and I talk to a lot of founders and even just in the last two weeks, had a conversation with two companies that I think got out in front of their skis by hiring big, expensive enterprise sales leaders before the product was really ready for it. And that can get you upside down.

Salespeople are going to go out, and they’re going to sell. They may sell you into an environment where the product’s not appropriate, where now you start getting all these requirements that are specific to that customer. You can get very distracted, you can start to love to see those big revenue numbers coming in, and you can move from this world where you’re building a scalable machine, where you’re building a product that can drive usage that you can then harvest. You’re sort of going upside down and selling something that you then need to drive usage behind. And I think that’s one of the bigger mistakes that I see repeatedly.

And so in that regard, I would be very careful about, quote-unquote, building for scale too early because it could kill your products and ultimately require a reset later.

Aseem: I think very wise advice. One thing I’ve also observed in meeting these companies is you tend to focus more on top-line versus usage and stickiness, and I think this goes back to our land-and-expand conversation around what is it that’s driving expansion, what is it that’s driving fan creation and fandom of the product? Something for folks to definitely keep in mind. Any particular spaces and companies you folks are excited about?

Mark: I’m excited about what genAI and, more specifically, what LLMs can really do for data, and especially that data, the mussy machinations. Numbers Station, a Madrona-funded company — super excited about what they’re doing, trying to bring that technology here. And I think we’re going to see a whole bunch more innovation in this space on all the messy pipes. What does ETL look like? What does data prep look like? ‘Cause this is the biggest problem. Every customer I talked to at Tableau, like yes, there were things about the active analytics that were still to be done, but that wasn’t their biggest problem by far. It was, how do I find data? How do I understand what that data is? How do I clean it and get it into a shape where it can actually be useful? And this is the problem that companies like Numbers Station are starting to tackle, and I’m really excited about what that can do to really accelerate the usage of data.

James: Yes, and I think that gen AI in general, or LLMs, and the applicability to the business process layer, what applications are going to be enabled, both horizontally and industry-specific operational applications?

We’ve talked a lot today about product-led growth, knowing your customer, understanding your user, learning from them, turning, burning, improving the product. Companies like Interpret and Viable, I think, are examples of interesting applications that are very specifically about gathering as much unstructured information as you can from your users, from your customers, from the market and interpreting it, understanding what you can learn in a way that previously was pretty high touch. One of the things that we did, and one of the things that I found frankly, back to Power BI, we were maniacal about collecting user feedback, and it was both structured and unstructured. We’d do NPS prompts, but we always had a place where people could type in their thoughts, and the ability to analyze that and to understand the trends was always time-consuming. And LLMs are offering an opportunity, I think, in a way that we couldn’t before, automate some of that so that you really can get the learnings very, very quickly and perhaps across far more channels than were previously available.

So, excited about that and all the other potential use cases that we see in business applications.

Aseem: This has been so amazing. Any parting advice for those building companies, founding companies? If there’s one thing you want to leave them with, what would that be?

Mark: I always say, and told my groups all the time, we only get to come to work at the pleasure of our customers. And it’s all about what problem you’re solving for your customer. And if you make happy customers, you will do just fine on the rest of it. Like James said, don’t focus on your go-to-market motion, don’t worry about what your customer… The first thing you got to do is build a product that’s solving a problem for a customer. If you’re solving problems for customers, the rest of it’s still going to be hard work, but it will take care of itself.

James: I would just hammer that home. We had this thing that we coined called Loose LUSRP, logos, usage, satisfaction, revenue, profitability in that order, and sort of walking down that stack. And by logos, I mean the ability to have a brand new customer who’s not a customer today or a user find the product and begin using it. And then you sort of work your way down to, “Okay, now let’s get more usage out of that. Let’s drive satisfaction.” And eventually, you can worry about revenue and perhaps gross margins at the end. Focusing on the user, driving usage, minimizing friction, increasing the velocity of improvements, listening, learning, and improving ultimately is, I think, the key to building a great company and certainly a great product.

Aseem: I want to bring back an example that I heard while I was at Microsoft, was this whole notion of, “We’re in the business of delivering happy meals. Happy meals make happy customers, and happy customers give you more happy customers.” So that was sort of what came to my mind as you guys were both talking about this.

But this has been amazing. Nothing short of fun-filled and a lot of insights that comes from years of you guys’ experience that our listeners and founders will enjoy. Thank you so much, this has been pleasure for me to host this and have this conversation. So on behalf of Madrona and the entire crew, thanks, and more to come.

James: Thanks for having us, Aseem.

Mark: Yeah, thank you, Aseem.

Coral: Thank you for tuning into this episode of Founded & Funded. Be sure to follow us wherever you get your podcasts, and tune in in a couple of weeks for our next episode of Founded and Funded, which featurs Bob Muglia, who just released his first book, The Datapreneurs.

Welcome to Founded & Funded, my name is Coral Garnick Ducken, I’m the digital editor here at Madrona. This week, Madrona Partner Jon Turow brings us a story about a great partnership forming between two people — who had never even met — when they each find themselves on a mission to focus on what it is they do best. You’ll hear from Hannes Mühleisen, creator of the DuckDB open-source project, and Jordan Tigani, the database leader who saw an opportunity to commercialize it by creating MotherDuck. They share the lightning-bolt moment that led to one of them flying half way around the world to meet – how does this happen, and how do they set their partnership up to be the foundation of a really big business while still supporting the open-source community? Jon gets into all of this and so much more.

MotherDuck and DuckDB have become integral for students of the modern data stack, but this story of inspiration, partnership, and execution is something that builders everywhere can learn from. So, with that, I’ll hand it over to Jon to take it away.

This transcript was automatically generated and edited for clarity.

Jon: Here’s Jon Turow. I’m a partner at Madrona. And I’m just really excited to be here together with my good friends, Jordan Tigani and Hannes Mühleisen. Thanks so much for joining, guys.

Jordan: Great to have the chat with you, Jon.

Hannes: Yeah. great to be here. Thank you.

Jon: So, I want to get into the genesis of DuckDB and MotherDuck. Jordan, you’re the founder and CEO of MotherDuck. Can you tell us what MotherDuck is?

Jordan: Sure. MotherDuck a serverless data analytics system based on DuckDB. You know, we’re a small startup company. We first got our start or even starting to think about it in April of 2022. And were funded by Madrona, among others, a few months afterwars.

Jon: Hannes, can you talk about what is DuckDB? What was the genesis of it, and sort of your part of that story?

Hannes: Sure, I’m happy to. So what is DuckDB? DuckDB is a database management system — a SQL engine. It is special because it is an in-process database engine, which means it’s running inside some other process. It is an open-source project. We have been working on this for the last five years or so. And it’s the creation of myself together with Mark Raasveldt, who was my Ph.D. student at the time. From this word Ph.D. student, you can already deduce that this was in some sort of academic environment. And at the time, I was a senior scientist at the Dutch National Research Lab for mathematics and computer science, the CWI in Amsterdam, which is famous for being the place where Python was invented, among other things. And there, I was in a group called Database Architectures, which has been working for many years on analytical data management engines. For example, they kind of pioneered columnar data representation for database architectures. They pioneered vectorized query execution. It’s been quite influential, let’s say. That’s nothing to do with me. I joined after all these things happened. But I did notice that while there were all these great ideas and great concepts flying around there was really that much in terms of real-world impact. And as a result, people were kind of using, let’s say, not the state of the art, right? I found that a bit sad. So we started talking to practitioners, figuring out where the problems were. And it turned out that this management of setting up data management systems, of transferring data back and forth, was really a concern. And it really didn’t matter how fast the joint algorithm was if your client protocol is horrible. That is, I think, one of the basic insights. People have written hundreds of research papers on joint algorithm, but nobody has ever thought about the end-to-end here. And so, we decided that we’re going to change that, and we’re going to actually look at the end to end and we’re going to bring the state of the art in research to a broad audience. And so we started implementing DuckDB back in 2018, I believe. It’s a bit insanity to say, okay, we are two people going to write a database management system. Like, these are things that usually hundreds of people work on for 10 years. And we have been warned. But I think one of my character traits is to kind of leap without looking sometimes. And that’s definitely an instance of this where I was leaping without looking. You could also say the companies case of leaping without looking, but we can talk about that later.

Jon: So Hannes, one of the things that you’ve shared with me over the time that we’ve known each other is even in those early days, you started to get customer feedback about using DuckDB and how to get it to work. And without leading the witness too much. There was an example of getting this thing to run in an academic compute environment with just all the things that are locked down by IT and the implications of that. Can you share that and how it impacted DuckDB?

Hannes: Yeah, absolutely. So part of the job of a database researcher is to try other people’s stuff. Like somebody writes a paper, maybe they ship some code. In an ideal circumstance, you get to try it. It’s very exciting. But it’s very difficult. Usually. That’s a code that is not meant to be run on like anything else. And then if you are, as you said, confronted with the sort of, absolutely lockdown environment of an ancient Fedora version where you don’t have root and the admin has like a 3-day turnaround, and you just want to try something and see if it’s like not worthless. It’s just completely impossible. And over the years, it built two things. One is an uncanny ability to run things without Docker. So – – prefix is my friend. And the other is, of course, a deep hatred for dependencies. And I think we underestimate the real-world cost of dependencies. It’s something that’s one of my, how should I say this, vendettas, especially given the recent rise of containerization that it seems to be just fine, or you have rust with its cargo thing, which is just fine to add dependencies. It’s not fine. It’s actually a, I like to say, an invitation for somebody else to break your code. But that was another one of these deep convictions that we got in designing DuckDB that can’t have dependencies. That was totally born out of the environment and the design, as you mentioned, of DuckDB as well, like we talked to people in the data science community and essentially listened to them. It’s a very uncommon sort of thing to do for a database researcher, oddly enough. They told us what they didn’t like, and they were also super happy to iterate on our half-baked ideas and give us feedback. So that was really, really valuable in shaking down the early design parameters, if you want, of this thing.

Jon: If I move to the next part of the story, Hannes, here you have this thing that you’ve built that’s really useful, and yet you’re doing a job that you love, you’re a researcher. Did you think about turning DuckDB into a company yourself? How did you think about that? What was the exploration, and how did you land on DuckDB Labs?

Hannes: Yeah, that’s, interesting because it’s a kind of a push-pull thing. So, first of all, in our research group, there has been a precedent in spinning off companies. There was, for example, VectorWise, which is obscure, but it’s the first vectorized database engine that came out of our group that was a spinoff. The CWI is also a place that is generally supportive of spinning out companies. But there was also a lot of pull, right? We had DuckDB we open sourced it in 2019, and then people started using it, and then people started essentially asking us questions like, when can we give you money? it’s an interesting situation to be in. You are in a research institute, somebody asked you, can we give you money? And you have to say no because there is just no process in this research institute to take money. It’s really weird. And I think it was about the same time when the VC started badgering us, for lack of a better word. There was this, endless stream of, “Hey, have you thought about starting a company?” So, we were a bit reluctant at first. I think it took us a couple of months of people asking us whether they can give us money. VC’s kind of asking us, “Can we give you money?” And us sort of think like, “Uh, not so sure. I don’t know.” There’s many stories about what exactly tripped us to do to start leaping. A story I like to tell is that kindergarten in Holland is just so expensive that I had no other choice but to start a company. Another one is that it’s absolutely clear that we needed to spin out in order to actually give DuckDB the room to grow because there’s only so many things you can do as an employee of a research institute. So that started this whole process of then, then we didn’t know anything about starting a company. Right? Like how do you do that? It’s not something they teach you at computer science school. Obviously, lots of discussions followed. Lots of soul searching, figuring out what’s going to be the business model. What’s going to be the process? Who are we going to trust? I think that’s, that was an important first question. Who are we going to trust?

Jon: And you decided, that you want to focus on the technology itself.

And that’s kind of where this landed.

Hannes: Right. But that was the long process. This process was very interesting because we talked to VCs, and they were like, “Okay, so you’re going to make a product, right?” Like, we have piece of software, isn’t that a product? And like, no, no, no, no, no. You have to be a Snowflake. It’s like, okay, but we don’t want to be a Snowflake. Yeah. Well, hmm. Difficult, right? And so there’s, there was a lot of discussions that went exactly like that.

But this idea that you can just be a technology provider of sorts, that didn’t resonate well, I think. And we were really like also wavering a bit on like, okay, they all wanted us to do this, should we really do this? But in the end, we talked to some people that have made database as a service companies. Very successful ones. They told us about their experience with this. They said, okay, this is what you are looking at if you do this. And that was clear that we didn’t want to do that. We wanted to be more open, we wanted to be more flexible. We wanted to not target one particular application area. Because, in our mind, DuckDB has so many different possibilities that going just for one, would be a bit restrictive. And because there were people already that were commercial users that were willing to give us money, we also had a different approach, which we could just say, “Hey, okay, we’ll take their money and, uh, we’ll run the company from that like, in the olden days. And that is still what we are doing. And it’s been, I would say — I’m quite happy with how this worked.

There are some people that we are thankful to that helped us, in the beginning, there was a Dutch entrepreneur that basically turned up with his lawyer, like on day three of this adventure, and said, “Here this is my lawyer. You need to talk to this guy.” And he’s still our lawyer. Right? There’s been, one of your former colleagues, Anu Sharma, who was extremely helpful and supported us in the beginning without any agenda if you want. There, there were a couple of people that have been extremely supportive, and I’m probably forgetting some, but it’s been, a great experience to do this non-standard thing because there were people out there were super willing to help.

Jon: That’s a fun introduction to the first thread. Jordan, can you maybe take us back to when you learned about this thing DuckDB? And what was the light bulb that went off in your head, and what you did about it?

Jordan: Yeah, so I was chief product officer at SingleStore, and we were really focused on database performance, building the fastest database in the world. We were looking at some benchmarking reports that somebody had done, and it had a number of different databases and I saw one that said, DuckDB, and was like, what is that? And why is it so fast? And where did it come from? And so, I did a little bit of poking, and I encountered some of the papers that Hannes and Mark had written. And they just really resonated with kind of the experience I had had over the last 12 years of working on these big data systems, one of which is that most people don’t actually have big data. And that scale-up is actually quite a reasonable way to build things. In SingleStore we were working on these sort of distributed transaction features that were taking a long time to build. And in BigQuery we worked on shuffle in order to do joins or in order to do kind of high cardinality aggregations that were very, very complex, basically relied on specialized hardware, and had big teams of people to work on them. And then in DuckDB, in order to do like these joins, you just build a hash table, and then you share a pointer to the hash table and it’s like, wow, that’s just so much easier. So there was the complexity side of things. There was the scale upside of things that it’s like, what you can do on a single machine is so much more than you used to be able to do. Then there was also the part that Hannes was talking about, which I think people haven’t actually grokked, yet, as the special sauce for what makes DuckDB so awesome. Which is that everybody focuses in databases on what happens once the query starts and until the query finishes. But there’s a bunch of stuff that happens both before and after. So before: How do you get the query there? How do you set things up? And then after there’s, okay, how do you get the data out? And so often, they go through these incredibly antiquated ODBC/JDBC interfaces or a rest interface or there’s the Postgres wire protocol and the MySQL wire protocol. And they’re just not really great. I think DuckDB was one of the first things that I had seen that really focused on the overall end-to-end.

To give an anecdote in BigQuery, we outsourced our JDBC driver and ODBC drivers to a company called Simba. And there was a bug in the outsource driver that added like a second and a half to every query. If your queries are taking minutes and you add a second and a half, that’s not a big deal. But if you want to do some sort of BI dashboards, etc., adding an extra second and a half is terrible. And in fact, there were some cases where it would add tens of seconds or even minutes because if the data sizes were large, they would basically pull through this very narrow aperture of the whole table back. And so, it was unusable for some BI tools.

And the thing is, we had no idea that this was even a problem because all we focused on was, okay, we get the query, we run the query as fast as possible, and then we give you the results. The fact that DuckDB is actually focusing on these kinds of problems, I think, is why it’s doing so well. Somebody Tweeted, “Why is DuckDB so fast?” The reason that DuckDB is fast is because all the stuff that everybody else isn’t paying attention to, they’re actually paying attention to, and so it feels fast.

Jon: There’s two things that really strike me. One is that you, Jordan, immediately imagined single-box execution. Just like Hannes, with a much bigger box. You realized that hosts in the cloud also count as single boxes, and yet so much RAM, so much compute. And, and I guess you’re going to say that comes from the sort of family of origin where you’d been raised at SingleStore and at Google. But the second thing is you, Jordan, I think are excited about doing complementary activities with your day, around team and company and business building with this advanced technology. And so maybe you could just kind of comment about that part.

Jordan: Sure. Yes, I did immediately think of cloud. I mean, I’d been on a, team in Google that was supposed to build a data marketplace, and then we said, well, you don’t want to just download the data. You want to actually compute over the data where it sits because it’s large data. So we built BigQuery, which is essentially taking a service that already existed in Google called Dremel, and we’ve built a product around it. And then at SingleStore, they were in the process of a cloud transition, and I spent the last 18 months taking an on-prem database and sort of building a cloud service out of it. And so, I know the pain of it, I know how it works and that’s just, that’s how my, that’s how my brain works. The other thing is in my career, starting as a software engineer for 15 years, as you move up the corporate ladder, you handle larger problems that have more complexity and more ambiguity. And then, as a manager, it’s another big step beyond that, which is people are more complex and more ambiguous, and getting them to do something is harder. So you have to figure out what makes them tick. How to get things to work, how to get the right output, and then as a manager with larger scope, you end up actually designing. It’s similar to a design problem in software as you’re designing with your organization. Okay, we need these pieces and these pieces, and this is how communication works. And it’s almost like it’s a distributed system. And then sort of moving to product, because then I switched from engineering to a product manager, is like you’re designing in the product space. And the product space is just this even broader palette of things that you can do. Because it turns out what actually matters is how are customers going to interact with something? If you build a beautiful piece of technology that nobody wants, it’s going to be really disappointing because nobody’s going to end up using it.

And so, you’re painting with this broader and more complex and more ambiguous space, and to me, that’s been sort of exciting. Nowadays, even though I love technology and I love to sort of geek out about databases I’m also realizing the thing that gets me excited is building products. That involves not just the tech, not just the architecture, but also all the market, the pricing, the packaging, the customers, all those other pieces that go along with it.

Jon: So, here we are in this moment where you spotted DuckDB. And a light bulb goes off in your head, and there’s a moment where you are so motivated that you get on a plane. Can you tell that story, Jordan?

Jordan: Well, I think I need to back up a little bit because I was really excited about this, and I’m like, Serverless is something that I think is the right way to, build cloud systems. And I felt like a serverless DuckDB should exist. There were so many nice things about it and so many things that other systems couldn’t do, like being able to scale down to zero and pay for what you use, and being able to sort of rebalance and move things around. It reminded me actually, sort of, of BigQuery, but like rotated 90 degrees. Big query was sort of very wide and thin. And then, with this, we could be very thin and deep, But also do the same sorts of things and perhaps be, be even more flexible. So I’m like, all right, it’s been a long time since I’ve coded. So maybe I’ll just sort of hack on this for a little while. And I got about two days into it. And then I asked, a friend and mentor of mine for an intro to Hannes and Mark because I knew that he had been working with DuckDB. The morning I talked to Hannes and Mark, it was like, huh, this seems like it could actually work. They’re not doing exactly what I’m talking about, but they kind of are looking for somebody to come in and build something like this. So that could, that could really work.

And then, in the afternoon, I talked to Tomasz, who was then at Redpoint. I got about 15 minutes in, and he is like, “I like this idea. I want to fund it. Come to my partner meeting.” And I’m like, “What?” I was not thinking of starting a company. I was thinking of learning Rust. That was actually my goal was to learn Rust. The next day I talked to another VC about something totally different, and I ran the idea by them and they said, ” I like this idea. You just had the partner meeting. How much do you want?” And I had no idea what to even say.

The next day I talked to a neighbor met who worked at Madrona. who I’d been meaning to have coffee with, and I let slip that, “Hey, I’ve been thinking about this idea.” And so she brings, Jon along with her to the coffee meeting. And that’s how Jon and I met. And that’s also sort of like within 48 hours kind of realizing, hey, there’s an interesting idea here. The next day I one of my first vacation, post the start of COVID. And I was in Portugal for a few days, and we’re trying to do all this sightseeing, and I’m taking all these calls from other founders, from VCs. I felt a little bit bad for my wife because it wasn’t as much fun of a vacation as it otherwise could be. And then I realized, okay, if we’re going to make this work out the most important thing is I need to have a great relationship with Hannes and Mark, and I need to really kind of see them in person and kind of look them in the eye. So I rerouted my trip, and I came back through Amsterdam. Hannes books four hours and I’m like, four hours — there’s no way we’re going to talk for four hours, and then like five hours later, we’ve been geeking out about databases — it was just sort of a really fun conversation. We’re like, oh, we’ve got to get to dinner, and we had dinner with our spouses. And that was the start of MotherDuck.

Jon: Was Jordan the first person who came to you with an idea to commercialize DuckDB?

Hannes: No, he was not the first person, I’m sorry to say. But how should I say this? He was credible. I think what really made, made you Jordan stand out from anything I’ve heard to this point. And I think also anything I’ve heard since, to be quite honest, is that you came from this background at SingleStore and BigQuery, and in a way, it was a big shock to me that somebody who had this kind of background would consider our scrappy single-node system for, for something serious. And I thought, okay, that was really crazy because we were being ridiculed for not taking distributed systems seriously with DuckDB. People were like, no, this is pointless. But we always thought like, okay, if we are some odd balls in Holland, and no one cares. But then to see somebody like Jordan come and say, no, no, you’re, you’re totally right. This is what we’re going to do. That was shocking. And I think, it really was clear that if we were going to work with somebody that does this, it’s going to be Jordan. That was pretty clear from the beginning. Certainly, after he changed his travels at the last second.

It’s quite funny to hear the other side of the story from you, Jordan, because while I’m aware of sort of the points where you were in contact with me, I had, of course, no idea of the background chatter with everyone else that already was going so far. But when, when we first talked, I was like, yeah, no, we can totally do this. You came over. I think we had indeed we had a good feeling about this. Then it went super quickly, of course. Like this was all in the matter of days. From us saying, yeah, we’ll be on board. It felt like it was minutes after that that things started being like, set up or something.

Jordan: And I was worried I was going to freak you guys out because things had moved so fast. It was like, oh, this intense American is like just cause, just cause we said yeah, it sounds like a good idea. All of a sudden, you’re like, okay, now boom, boom, boom. Here’s the money on the table. And I was kind of terrified that because it was, things were moving way faster than I had expected and was just sort of like riding the wave. I was very cognizant of trying not to freak you and Mark too much.

Hannes: I think at that point, we had spoken to enough Americans. And I have to say my wife is American, so I have a daily exercise in this. But it wasn’t scary, I thought. I don’t know. It seemed so logical and obvious that I wasn’t scared at all, and I don’t think Mark was either.

Jordan: That’s good to hear.

Jon: There’s a certain sort of trust in friendship that’s evidence here, but I want to also pull out this connection that you mentioned to me once or twice, Hannes. The fact that Jordan has not just built a lot of cool stuff, but has built a lot of cool distributed systems that scale out versus scale up. Coming to you and saying, “Hey, scale up is actually pretty cool.” That kind of narrative violation, I think…

Hannes: There was a shock to me. Yes.

Jon: It seems like it gave credibility and also has been an animating theme for MotherDuck. Isn’t that right, Jordan?

Jordan: Yeah, I mean, the recognition that most people don’t have huge amounts of data. Even working on BigQuery, people were not doing big queries. For the most part, they were doing little queries and focusing on getting data in and getting data out, and the user experience of the query and using the system is actually more important than size. And then also, yeah, you can scale up. And then the last piece is — you can always make it distributed. I’m sure somebody, even if we don’t do it, somebody’s going to come up with a distributed DuckDB. We have a bet internally about whether we’re going to end up doing sort of a distributed version of, MotherDuck. My bet is, no, we won’t need it. Other people think that we will, but we’ll see. Like in BigQuery, we had BigQuery BI Engine, which was a scale-up single node system that sits on top of BigQuery storage. And because it had to run in these constrained machines that have to run Google search, there were no big machines, and so we ended up having to build out a scale-out version of it. It took a year and like three or four engineers. It can be done, but, ideally, you wait as long as possible, and so you get as much innovation into the core engine until you have to do that.

Hannes: I think what this biblical transformation from scale out to scale up. The reason why this was so surprising. I think what, what is so, so transformative for us, I think was because our idea of why we wanted to scale up was based on a feeling, I want to say. And the feeling was based on actually using things like Spark and having this feeling that something’s wrong with the world, if this is the best we can do. But this was based on a feeling that scale-up was the way to go. We didn’t have data on this. It was only recently, I think in maybe 2020 or so, that Google published the TF data paper. That actually said this explicitly, like 95 percentile of all our machine learning in job size is like a hundred gigabytes or something like that. But then, of course, Jordan, you also had seen this from the inside. I feel like if you haven’t seen the big data, then probably no one has. So, it made the difference for us from a feeling to something that, was actually a thing. It was a great moment. I would have to say.

Jon: If we go back to coffee, Jordan. The first time you and I met, there, there’s a thought that went through my mind and a question I asked. The thought that went to my mind was that this is almost witty to say, let’s go for, whatever’s the opposite of big data. That’s almost funny, considering the way that so much of our data technology has been designed to scale. And the first question that I asked you was, well, if we change that constraint, and we looked at the world as it is and the workloads as they are instead of how it could be if we waved to magic wand, what can we do that’s possible that wasn’t possible before? Maybe you can share either what you answered then or what you would answer now about that if it’s different.

Jordan: I wish I remembered what I answered then, but it was a rough couple of days. What I think I probably said is, when we started BigQuery, the mantra we used — and it came from the Turing award winner and database researcher Jim Gray — what he said was, “With big data, you want to move the compute to the data, not the data to the compute.” When I kind of described building this system where you didn’t want to just download the data. is because moving the data is so expensive and it’s so hard. And so BigQuery was sort of built around that premise. But then, once you kind of recognize that data may not be that large after all, then how would you design the system differently, and can you move the data to the end user and leverage the compute power that, the end-user has? George Fraser, the Fivetran CEO, just did a benchmarking report. I think it’s just crazy and amazing that the CEO of a multi-unicorn company is running, database benchmarks and doing a good job of it. But anyway, he found that his 2-year-old Mac laptop, not even state of the art, was faster than a $35,000 a year data warehouse. It used to be the laptop was synonymous with underpowered, and nowadays it’s, it’s a huge amount of power. And so, why when you run a query against one of these cloud data warehouses and you wait three seconds for it to run? And meanwhile, everybody else is waiting for that same hardware, and your incredibly powerful computer on your desk is sitting there idle. Why not actually let that computer on your desk participate in that query? A — it’s less expensive because you’ve already paid for that laptop on your desk. B — it’s a better user experience because you can get results back in milliseconds, not seconds. I think there is a new paradigm and a new architecture that can be used. And then there’s further things. There’s edge, there’s mobile, there’s leaving data where it is when it gets created, instead of having to worry about consolidating the data. Like people complain about, okay, why do I have to consolidate all this data in the same place? It’s so expensive to move up. If you can have the compute be anywhere, then there’s so many kind of interesting things that you can do.

Jon: It’s amazing to see that two things can be simultaneously true in this area. The marginal cost of a cycle of compute may be the lowest in a cloud because of the scale and the optimization of that. And, yet, when we dial up the capacity of some distributed system analytics in the cloud, it’s a lot like adding lanes to a highway which produces more gridlock in about 24 months.

Jordan: The reason that adding lanes to highway doesn’t make things faster is because what’s slow is getting things on and off the highway. Getting back to sort of DuckDB is like getting your query in and getting your data out are very often the most important things. And what happens in the middle all converges to the same place over time.

Hannes: Yeah, it is really shocking to see what is possible on laptops. It’s something that we have kind of forgotten about. And, of course, the marginal cost of a, CPU cycle in the cloud isn’t what your cloud database is billing you for it. Right? I think there’s also a big difference between what the cycle costs and what you are paying for the cycle. And I think it is maybe also part of the reason why it is so much nicer to run things locally than it is to go through that.

Jon: Guys, I want to leave this a little bit where we started. T o produce DuckDB and MotherDuck and this really exciting opportunity for your customers, it took each of you doing what it is that you love that is complementary. I think our audience would be interested to hear however many months in it is since that all happened, what it is that you’ve loved or learned from the other person that you’re working with that has helped you become stronger since then?

Hannes: Well, my life has turned around pretty radically since we first spok, Jordan. My life has changed completely from a mostly academic researcher to somebody who is running a, I mean, it’s not a giant team, but it’s a team of competent people that are building DuckDB here at DuckDB Labs in Amsterdam. And my position has changed into something that is much more what Jordan described. And I’m not sure I’m entirely there yet that — this is going to be the thing that I really love doing. But it has changed a lot and it’s been really interesting for me to see how Jordan goes about doing things. Because he’s been, of course, also building his company the last 10 months at a much greater speed than we do, of course. But since this is all so new to me, it’s been extremely interesting and valuable for me to just watch that a bit.

Jordan: On my side, at the end of 2022, I sent a letter to kind of our team and to investors, and I said, if there was one word, to sum up 2022, it’s lucky. I feel just incredibly fortunate that we kind of hitched our star to DuckDB and to Hannes and Mark and their team because there’s such incredible tailwinds that come behind this really groundbreaking technology. And we were in the right place at the right time, and, hopefully, we’re going to have this great partnership going into the future. And one of the things that worries me the most going into that is that we’re going to do something to screw up that relationship. And from other founders and people who have commercialized open-source technology, including the Databricks founder, have shared with me is that it’s going to get hard because your incentives are going to diverge, the things that you care about are going to be at odds. And so, it’s just something to actively maintain. As fortunate as we are, we want to acknowledge that and also acknowledge that in order for this partnership to be successful in the future, it’s going to take, active work and deliberate trust in being willing to say, “okay, Awell, maybe we wanted to do this, but for the sake of the relationship we will take a slightly different approach.”

Jon: Hannes Mühleisen, Jordan Tigani, thanks so much for your time. This has been a lot of fun.

Hannes: Thanks for having us.

Jordan: Thanks, Jon. Thanks, Hannes.

Coral: Thank you for listening to this week’s episode of Founded & Funded. If you’re interested in learning more about MotherDuck, please visit MotherDuck.com. If you’re interested in learning more about DuckDB, visit duckdblabs.com. Thank you again for listening, and tune in in a couple of weeks for another episode of Founded & Funded with Panther Labs Founder Jack Naglieri.

Today we have the pleasure of hosting GitHub CEO Thomas Dohmke. He and Madrona Partner Aseem Datar talk about how Thomas got into working with computers and coding and the work he’s been doing since becoming GitHub CEO in November 2021, including the recent launch of Copilot X. But these two discuss so much more, including the rise of generative AI, talking about everything from how it is a new way for developers – everyone really – to express their creativity to how it democratizes many skills and access to those skills to the generative AI-powered developer experiences and how the constantly evolving world developers have always worked in has set them up with the perfect safety network to leverage generative AI to its fullest potential. Thomas also offers up advice for people just launching a startup. But you’ll have to listen to hear it all.

This transcript was automatically generated and edited for clarity.

Aseem: Hey, everybody. My name is Aseem Datar. I’m a partner at Madrona Ventures. Today I have my close friend and GitHub CEO Thomas Dohmke. I’m excited to chat with him on this wonderful topic of generative AI.

Thomas, welcome.

Thomas: Yeah. Hello, and thank you so much for having me, Aseem.

Aseem: We are excited more than you are, Thomas. It’s always fun to talk to somebody leading the charge on innovation in this industry. Maybe start by giving us a little bit of your story and introducing yourself.

Thomas: I’d like to say I’m Thomas, and I’m a developer. I’ve been identifying as a developer ever since the late ’80s early ’90s when I was about 12-13 years old, and I got access to computers first in the geography lab in school and then later when buying a Commodore 64. I’ve been fascinated by building software. And, obviously, as a kid also gaming and playing with all kinds of aspects of computers. And I have been working with code and being passionate about code ever since building my own applications, studying computer engineering first in Berlin. And then, doing my Ph.D. in Glasgow, I worked at Mercedes, building driver assistance systems. And then, in 2008, Steve Jobs announced the App Store. And it pulled me into the app business. I had a startup called Hockey App that was ultimately acquired by Microsoft in 2014, and that moved me from Germany all the way here to the West Coast into Microsoft and that path. Then led me first into GitHub through the acquisition, running special projects at GitHub, and since November 2021, I’ve been the CEO.

Aseem: What a fun journey. Thomas, I can’t stop myself from saying developers, developers, developers all the way from the Steve Ballmer world. And it’s so much fun to be talking to you. Clearly, a lot has changed in the world. There’s this rapid pace of innovation that we are seeing with this new capability set called generative ai. And we are all excited about talking and hearing more about generative ai. What’s your worldview? I would love to just understand that.

Thomas: If I look back over the last six months or so, we had multiple moments that you could compare to the App Store moment I described earlier. That happened in 2008. I think the biggest of those moments clearly was ChatGPT late last year. And you know, I have heard people describing that moment of ChatGPT launching and seeing fast adoption as the Mosaic moment of the 2020s. If you’re old enough, you might remember the first browser Mosaic and then quickly followed by Netscape and, actually, last night over dinner, I argued with folks — is it the Netscape moment or the Mosaic moment? I think it doesn’t really matter, but what matters is that within a rapid amount of time, people adopted ChatGPT and had seen the way they work shifting. And before that, before ChatGPT, we had already seen a shift through Midjourney and Stable Diffusion — those image models. And I think, you know, those models are great to describe what generative ai does, and part of it is really creating a new way of people expressing their creativity. And we have heard stories of folks spending, you know, their evenings rendering images instead of watching Netflix. I think that’s exciting. My example always is, you know, depending on what city I’m in, what customers I’m speaking to is like, you know, ask Stable Diffusion to render the skyline of Tel Aviv as if it were painted by the French impressionist Monet. And obviously, Monet hasn’t seen the skyline of Tel Aviv as it looks today. And yet, those models generate a picture that resembles a Monet rendering the skyline of Tel Aviv, Sydney, or San Francisco. And I think that is really the power of this new world of generative ai.

And the other thing that it brings is it democratizes a lot of skills and access to those skills. And especially if you think about students and kids that sit in class and where the teacher in front of a class of 30 kids just doesn’t have the time to be the tutor for each every single individual kid, but have it giving them an AI assistant where they can ask all the questions that they might not dare to ask in class, or where they, you know, didn’t have the time or the teacher didn’t have the time or the parents don’t have the time because they’re working three jobs. I think that is where really the power of this AI moment comes from and where we see tremendous excitement in the industry and really in, in everybody you’re talking to.

Aseem: Yeah, I mean, no question. Right? I think productivity is such a massive space where generative AI is having an impact today. It’s awesome to see these scenarios in real life, come to light, whether it’s for students, whether it’s for business workers, whether it’s for information workers, but behind it, all is the ethos of creativity in some senses in the software world are developers, right? And I think you can’t run away from the fact that there are developers creating these intelligent applications and embedding AI into it. So what does this moment really mean for developers? How do you think the generative AI-powered developer experiences will change?

Thomas: The role of developers has always changed, right? If we look back over the last 40 years, we went from punch cards and machine language and mainframes and cobalt and whatnot to modern programming languages. We went from building everything ourselves before the internet to leveraging thousands of open-source components ever since, you know, the early 2000s, I’d say.

Aseem: By the way, I thought Visual Basic was a big moment. just You know, going back to those days but, but carry on.

GitHub CEO Thomas Dohmke: And you can probably make that argument for many programming languages in their own right. I think Ruby was a great moment as well. And a lot of startups in the last decade or so were founded on Ruby on Rails because it’s just so easy to iterate with Rails. And Python, you know, unlocked a lot of the machine learning that we are now seeing. And the nice thing you know about software development is that it has been always part of the practice of software development to solve issues, right? No developer is perfect, whether we made mistakes on punch cards, we made mistakes in assembler and now we are making mistake in code. It has always been around solving issues, fixing your own bugs, or fixing your team’s bugs. And the word bug even comes from the bug on the punch card. And so, we built all this tooling, compilers, and debuggers to find, issues by writing code. We invented practices like unit testing to make sure that what we’re building is the thing we wanted to build. And in the last decade or so, we introduced DevOps practices or agile practices, code review, pull request, pair programming, continuous integration and deployment, CI/CD, code, and secret scanning. And so if you tie this now to AI, it’s actually fascinating. We’ve built the safety network within software development to leverage generative AI to its fullest potential. We all know that those models, those large language models, are not always right and that they have something called hallucinations. They think they have the answer, and they’re confident in what they’re saying, but it’s wrong. And with all these practices that software developers have, we have the safeguards in place to make sure we can work with a model suggestion and either take it and modify it or take it and then figure out in code review that is not exactly what we want to do. You could argue we built DevOps with the aspiration that in the future, there will be a moment like ChatGPT, where we can unlock more productivity, more creativity in developers to ultimately become realize even bigger ideas. I think that’s ultimately what this is all about.

And at GitHub over two years ago, now — in 2020, we started working on Copilot, which is one of the first AI pair programmers. It sits in your editor, and when you type as a developer, it suggests code to you and can complete a line, but it can also complete whole methods — multiple lines of code, lots of boilerplate, import statements and Java and whatnot, test cases, complex algorithms. And it’s not always right, but developers are used to that. They type in the editor, and it shows the suggestion. And if that’s not what I want, well, I can just keep typing, and if it’s close enough to what I want, I’d press the tab key, and I can use that and modify it. And that’s no different than copying code from Stack or from GitHub and then modifying that. Almost never, you know, you find a snippet on the internet that’s exactly what you want.

The generative AI-powered developer experiences gives them a way to be more creative. And, I mentioned DevOps earlier. I think DevOps is great because it has created a lot of safeguards, and it has made a lot of managers happy because they can monitor the flow of the idea all the way to the cloud and they can track the cycle time. And they have a certain level of confidence that developers are not just SSHing into a production server because, they are some safeguards in place, but it hasn’t actually made developers more happy. It hasn’t given them the space to be creative. And so, by bringing AI into the developer workflow by letting developers stay in the flow, we are bringing something back that got lost in the last 20 years, which is creativity, which is happiness, which is not bogging down developers with debugging and solving problems all day but letting them actually write what they want to write. I think that is the true power of AI for software developers.

Aseem: I remember my days of writing code in an Emacs editor, and that was just like slightly better than Notepad because it had a few color schemes and whatnot. Two things that you mentioned that I latched onto. One is productivity, and the second is creativity. And I think those two certainly are top of mind for developers. What are some of the things that developers should be excited about, and what are some of the areas that you guys have doubled down in and will continue to double down in?

GitHub CEO Thomas Dohmke: Yeah. I mean, let me take you on a bit of a history lesson. In the summer of 2020, GPT-3 came out, so that’s almost three years ago, and back then, you know, our GitHub next team that team within GitHub that looks into the future asked themselves can we use GPT-3 to write code? And, we looked into the model, and we came up with three scenarios. It’s fascinating now in 2023 to look at these three scenarios because there was text to code. So that’s what Copilot does today, right? You type text, and it suggests code to you. Code to text, which is like you, you ask the model to describe what the code is doing. And we just announced that as part of Copilot X, where you can have Copilot describe a pull request to you. And if you’re a developer, you know what that’s like. You’re working all day on a feature, and you’re submitting a pull request, and now you have to fill out all these forms and its title and the body, and like, ah, I know, I know what I did today. And it’s all obvious to me because I build all this code. I don’t want to spend too much time describing that to others. And so, with copilot for pull requests, we are helping people to just do that for them. And it describes the code, but it’s not only about the pull request, it helps you to describe code that you might be reading from a coworker and the editor. It might just help you to remember what that was. And it might help people to describe old code. This old COBOL code that some banks are still running and its code that’s from the ’60s, running on mainframes where the people that wrote that code back then are long in retirement, I hope. And so, the expertise is gone. And then the last one was conversational coding. And we didn’t build that at the time because we felt the model was not good enough to have these kind of conversations. And clearly now, with ChatGPT 3.5 and, and now GPT-4, we have reached the point where those chat scenarios are useful. And more often right than wrong. Back in 2020, we explored these three scenarios, and the way we validated, that this is good enough for us and that we can build a product on top of that was we asked our staff and principal engineers to submit coding exercises, things we would use in an interview loop — a description and a method declaration and a method body. And so we got about 230 or so of these exercises, and we stripped out the body and basically gave only the declaration and the description to the model. And we gave the model 150 attempts for each exercise to solve the exercise and get close enough to the solution. And what we figured out from this experiment that 92% of those exercises could be solved by the model back then in 2020. Even then, already the model was good enough for a lot of these coding exercises. And so, we took that as inspiration to build Copilot and ship Copilot to the world.

On March 22nd, we announced Copilot X. So, then the next generation of Copilot, of really bringing the power of these AI models into all parts of the developer experience, whether it’s coding in your IDE, whether it’s chat scenarios where you can explore ideas. The example I tried first was I ask it how to build a snake game in Python. You know, the game that we were playing on cell phones before they had touchscreens. And it starts showing an explanation of how you do that, and then you can just ask it to “Tell me more on step one,” and it shows you some code, and you can start building with that. I think that’s the true power here is that you can rediscover your love for programming if you lost it. Or you can explore a new programming language, or you can just, you know, ask the chat agent to fix a bug in your code or fix the security issue, like to remove that SQL injection that you accidentally put there. We announced Pull requests. I’ve already mentioned that describing pull requests. And soon enough, we will also have test generations. So, the pull requests will check whether you actually wrote or the tests you’re supposed to write and then generate those tests for you. And then the other cool thing that we announced is Copilot for docs. And so, we built a feature that basically lets you ask questions about the documentation for React, Azure, and a couple of other projects.

And so, the model has a cutoff date until it was trained. And the training is a really expensive process. It takes, you know, weeks on a supercomputer to train the model again. The current GPT-4 has a cutoff date of September 2021. And it actually will tell you that, if you ask questions about things that happened since then. And so it doesn’t know about changes to open source projects in their documentation that happened in the meantime. And, you know, September 2021 to, we are recording this in March 2023, is a long time for APIs of open source projects. What we’re doing is basically we are collecting that data from those open source projects, and we are feeding them into the prompt, so they’re becoming part of the prompt of the part that you’re not seeing as the person asking the question, so can answer up to date questions on those projects.

Aseem: I am so excited about Docs, right? Like, I go back to my days as a developer, and so much time was spent on going and reading up docs and pulling up from different places, and it was just a productivity suck. So, congrats and kudos. And I do want to point out that I think GitHub created this notion around Copilot, which is now injected all across Microsoft, and now there’s a copilot for Office as a copilot for Teams. I couldn’t be more excited to see where this goes. Shifting gears, a little bit, Thomas, one thing that gets me excited, especially in the world of venture, is that our startup founders and teams can now go from zero to production very quickly. What advice do you have for somebody starting out, like building a business or creating a team? What should they be bullish on? What should they be worried about?

GitHub CEO Thomas Dohmke: I think you know a lot about creativity is to stay in the flow and not get distracted from all the things that are happening around you. And oftentimes, you know, we, we are like gravitating to those things, whether it’s the browser or whether it is social media and whatnot. And so, I think my first advice to startup founders is, you know, stay focused and leverage the time of the, day when you’re actually creative because that time is so limited. Like, you know, our creativity is infinite, but the time when we are actually creative during a day, when we have the energy to build cool things, is fairly limited. And, for some people, it’s early in the morning. For me, it’s usually after my first cup of coffee, that’s when I’m the most creative. And then I always want the second cup of coffee to have that same impact, and it doesn’t, right? It never works that way. am also creative at the end of the day when it’s dark outside, and I’m a bit of a night owl as well. And so I think, you know, as a founder, you have to find those moments during the day and keep that energy ultimately flowing.

We live in this, you know, world right now, whether you call it a recession or not, I think we are in a complicated macroeconomic environment, to say it more, politically correct. But I think those times are always challenging and opportunities at the same time. And we saw this in the last downturn in 2008 — many of the startups that are now part of our lives, like Airbnb, Uber, Slack, or Netflix, they were founded around that same time. And they’re now part of life. And they, or Shopify actually, is another great examples of these that was founded during a downturn, building the technology, and then as we came out of this everybody wanted to have an e-commerce store or, and buy from these stores. And I think that’s the opportunity that we have now and today or this year, it’s leveraging generative AI, as like the foundational layer. And many startups will build on top of that, and they will have to find differentiation and defensibility of their idea. And I think, you know, we’ll see a. cool ideas building on top of ChatGPT or GPT-4, and a lot of these are really cool, but they’re also probably not going to survive as a company on their own because it’s a small idea that, you know, summarizing your emails in Gmail. I would think Google will build that into the product and then you really have to push hard to make that a paid product that’s customers will pay for if they have that already built into Google.

Aseem: I couldn’t agree more. We’ve always talked about do more with less, but I think the, the AI or the capabilities that we are seeing pop up is all about doing much more with much, much less. And that’s, I think, the beauty of the pace of innovation that we are seeing all around ourselves. Thomas, I know that you are, you’re deeply plugged into the startup ecosystem. You see a lot of these open-source projects come to life. Are there any projects or startups that you are really, really excited about?

Thomas: I’m, I’m staying bullish on ChatGPT and OpenAI, and I think we are at GitHub very excited about the future of Copilot. I mentioned earlier things like Stable Diffusion and Midjourney, which make me really excited. I’m, I’m not an artist at all. I can’t draw, and I certainly cannot paint something that looks like a Monet. And if you take that a step further, I’m really bullish and excited about a startup called Runway that lets you generate videos from images, from video clips, but also from text prompts. And I think, you know, there’s going to be a moment where you can just write a script into a text field, and it generates a full animated video for you. And that will allow us to take the stories that we heard as kids from our parents or even grandparents and turn them into little video clips that we can show to our kids. And I think that will be so cool if you can basically tell the stories for me now, two or three generations ago in little videos to the next generation. I think you and I both sit on a board of a company called Spice AI that explores AI from a different perspective, which is not about large language models or image models. It’s about time series AI and finding anomalies in time series data. And it allows you to query that data, and they started with blockchain and Web3, and you can write your own queries and quickly figure out what’s Bitcoin doing. But you can also run AI on top of that and find things that are interesting, find alerts, or find price changes. And in the future, I think there’s a lot of huge space in there. You can apply this to your server data, your server monitoring maybe your Kubernetes clusters. There are all kinds of time series data that affect us every day — weather is ultimately also time series based, right? Like it’s cold in the night and warm in the day. And so, I’m excited about that. In general, you know, the AI and mL space is super exciting for me. There are so many startups I could list here. There’s Replicate, a startup that’s based in Berkeley. They’re letting you run machine learning models with just a few lines of code. And you don’t actually have to understand how machine learning works. There’s OctoML based in Seattle that uses machine learning to deploy machine learning models to the cloud and find the most efficient version, you know, the right GPU type, and the right cloud provider for your model. But I think you know the ML AI space is super exciting, and I’m sure we are going to see lots more ideas that nobody thought is possible and and nobody thought about right now. And. Similar to, ChatGPT in hindsight, seems so obvious. But until it came and conquered the world, nobody else had built it. So, I think I couldn’t be more excited about that future.

Aseem: Yeah. And I echo that sentiment. We at Moderna are really excited about being able to help Runway, OctoML, and Spice AI in their journey of building out for the future. And I think it’s always interesting to see the future getting accelerated in a way that we can, that we can’t even imagine, to be honest. And yes, there is scenarios around hallucination, et cetera, that we’ve all got to watch out for. And I think you said it well, which is, it’s a start. There’s still going to be a developer or human in the loop, at least for the short term, until it gets to a point of high confidence.

Thomas one other interest. Notion that, that I wanted to sort of also pick your brain on is if I’m a startup founder, what should I look forward to in the distant future? I mean, we talked about all these modalities, but one of the challenges that founders have is developers are hard to come by and top talent is very hard to come by. And, there’s this notion around, tools being built to go tackle the low-code, no-code space or democratize development. What’s your view on that from a GitHub perspective?

Thomas: You know, I think there’s this, slogan, fake it till you make it. And that’s true for so many founders as well. You know, you don’t have to have a perfect solution right from the start. You can combine all these AI tools that are available to you now to stitch something together really fast. Whether it’s copilot, whether it’s Stable Diffusion, whether it’s some of the other tools that help you just by AI — help you write your marketing copy. Embracing those things as much as possible and adjusting your style to it. I think what will happen to developers is that the developer will learn how to leverage AI to its best. Andrej Karpathy tweeted about this recently where he basically says I changed my programming style and by writing a bit more commentary and a bit more declarative, statements I can get Copilot or aI to actually synthesize more code for me. And I think that’s kinda like what we are going to learn and where I’m bullish on building AI in the open and having those models out there, and building with them and learning how to use them as early as possible before we get to AGI and there’s a certain amount of scare about this and what we can do. But you know, today, those models are, not sentient. They’re not actually creative. They’re predicting the next word. And if you wanna switch ’em off, you can just go, to an Azure data center and switch, switch it off. And I think, but, so we need to build this in the open and we need to learn from where’s the model good and how can we use the model to help us as humans. And we also need to learn where’s the model bad and where make does it make mistakes or makes wrong predictions.

And actually, I think, the model itself will be able to correct itself. I think, there was recently an example from Ben Thompson’s blog, Stratechery, where basically somebody on social media posted, I think, four paragraphs of a blog post from Ben into ChatGPT and then asked it who wrote this. And it basically detected that this is a blog post from Ben Thompson without telling it that information. And I think, in the same way, we will be able to use AI to detect something that was wrongly written by AI. And so, the technology works with each other. And I think by building this in the open, we are preparing for that future where AI plays a bigger role for us on this planet.

Aseem: Hey Thomas, I know we are out of time. Thanks so much. This has been a blast, and I’m sure our startup founders, our listeners, are taking so much away from this discussion with GitHub CEO Thomas Dohmke. And I couldn’t thank you enough. Thanks for being on with us. And we’re excited to be able to partner and work together.

Thomas: Yeah. Thank you so much for having me on this podcast.

Coral: Thank you for listening to this episode of Founded & Funded. If you’re interested in learning more about what’s going on at GitHub, check out their blog at Github.blog. Thanks again for listening, and tune in in a couple of weeks for our next episode of Founded & Funded with the founders of MotherDuck and DuckDB.