It was a pleasure hosting the third annual IA Summit in person on October 2, 2024. Nearly 300 founders, builders, investors, and thought leaders across the AI community and over 30 speakers dove into everything from foundational AI models to real-world applications in enterprise and productivity. The day featured a series of fireside chats with industry leaders, panels with key AI and tech innovators, and interactive discussion groups. We’re excited to share the recording and transcript of the panel “Searching for Answers: The New Era of Enterprise Productivity” with Glean Founder and CEO Arvind Jain, Smartsheet President and CEO Mark Mader, Luminance CEO Eleanor Lightbody, and moderated by Mark Nelson, a Madrona Venture Partner and former CEO of Tableau.
TLDR: (Generated with AI and edited for clarity)
In a world where generative AI is touted as the ultimate game-changer for enterprise productivity, how much of it is hype, and how much is reality? At our recent IA Summit, Madrona Venture Partner Mark Nelson led a panel to cut through the noise with CEOs Eleanor Lightbody (Luminance), Arvind Jain (Glean), and Mark Mader (Smartsheet). They tackled the big questions head-on: Are companies really seeing productivity gains from AI? Can we trust AI with critical tasks? And what does the future hold for the workforce as AI continues to evolve?
From real-life success stories to the challenges of building trust and integrating AI into everyday workflows, this session provided a no-nonsense look at what AI can — and can’t—do for your business today.
- AI’s Early Impact on Enterprise Productivity: The panel explored how AI is already saving time across industries, with specific examples of how companies like Glean are improving case resolution times and Luminance is automating legal processes. However, there’s a consensus that while productivity gains are real, we are only scratching the surface of AI’s potential.
- Building Trust and Adoption in AI: Trust remains a key challenge, especially in high-stakes environments like legal and enterprise settings. The panelists emphasized the importance of transparency and gradual adoption, where AI acknowledges uncertainty and users are given time to build confidence in its outputs.
- AI’s Role in Workforce Transformation, Not Replacement: While AI is driving efficiencies, the panelists agreed that rather than replacing workers, AI will transform roles, enabling people to focus on higher-value tasks. They predict that AI will create new opportunities, shifting how we define productivity and work.
Mark N: So I’m Mark Nelson. I’m a venture partner here at Madrona, and I get the pleasure of leading a discussion around enterprise productivity and searching a little bit for not just the vision that we’ve heard a lot about, but a little bit of the reality as well. And just to set the stage a little bit, since Gen AI has exploded on the scene, we’ve heard a lot about the magic world of productivity that’s coming to us and heard a lot of statements. So we’ve had the CEO of Klarna saying he’s going to get rid of 50% of his staff and he’s going to fire Workday and Salesforce. Then you’ve got Marc Benioff renaming Dreamforce to Agentforce and saying, “You’re not going to fire Salesforce. All you need is Salesforce and a bunch of agents that are going to do everything for you. It’s all going to be great.”
You’ve got the leader of DeepMind saying this is going to be great for knowledge workers because AGI is going to be here in the next three years, and we should all get our reading lists ready because we’re going to be sitting on the beach hanging around with a bunch of leisure time. Meanwhile, we’re all trying to get our jobs done and wondering where the productivity is going to come from. That’s the setting of the stage. We love and are thankful that we have three world-class experts here to talk about what’s real today and maybe a little bit of that vision. So with that, I’ll just ask you each to go down the line and introduce yourself and how you fit into this discussion.
Eleanor: Hi, everyone. I’m Eleanor Lightbody, and I’m the CEO of Luminance. Luminance is an AI company that helps businesses with every single interaction that they have with their legal contracts. If you think about drafting a contract, if you think about negotiating a contract, and if you think about finding things within your contract, our AI will automate a lot of that and it will also augment a lot of that. Founded in 2015 before AI was as cool as it is today, and our customers include the likes of AMD or Staples or BCG Consulting.
Arvind: Awesome. Yeah. Hi, my name is Arvind Jain. I’m the CEO of Glean. You can think of Glean like as Google or Chad GPT, but inside your company. We help employees quickly get answers to questions that they have or help them with some of their tasks using AI and using all the knowledge and data inside the enterprise that Glean is connected to in addition to all the world’s knowledge. So we are actually an enterprise productivity company. When we started, we started with the goal of helping save people time. One third of all working time people spend just looking for information, and so our job was to actually make that easier and faster for people. So that’s what Glean is about.
Mark M: Mark Mader, CEO of Smartsheet. We help companies all over the world, specifically business teams, better manage programs, projects, process. And how we intersect with this, Mark, is how we’re incorporating folding AI into making that much easier for people who are asking, how the hell do I fit into this? Because a lot of people are on the outside looking in. A lot of people at this conference are hyper fluent in all the details, the low-level models, and the tuning. Most of humanity’s like, “I don’t get it.” And we’re trying to connect that dot for them.
Mark N: Which is awesome, which is a great tee into the first question, where are we? I mean, we know we’re early, but is any of this real? Is what is real what we’re going to see three years from now? What do you think and what have you seen really being productive today. Whoever wants to dive in first?
Arvind: Yeah, I can talk a little bit. I mean, we just saw the last presentation where a factory was saving engineers four hours of time every week with Glean. We’ve been deployed to many, many large enterprises in the world, and our customers do the same thing. They actually survey their employees and try to actually figure out how much time does AI save them. We also get similar numbers, I think the self-reported numbers from users, which is three to four hours of time saved every week across different teams, whether you’re a developer, whether you’re in tech support or sales.
So there’s a lot of real value that is already being added, but in how far we are in terms of where it could be, I feel that half of our work is not going to be done by us within a few years. So I think that time savings needs to be more like 20 hours a week, and we’re still quite far away from where we can be and where we will be.
Eleanor: Yeah, I would agree with you. I think, let’s take a step back, AI and productivity, it’s a massive spectrum, isn’t it, at the moment? We’ve been seeing productivity gains from machine learning for a while now, but Gen AI has just accelerated that, and we’re still very much I think in early days. And the conversation from where I stand has moved on. 18 months ago, it was like, “Wow, this can do everything and I’m so excited and where should I point it at?” And now it’s like, “Wait a second, what’s the value?” And someone earlier was talking about proof of values, and I think that that’s something that we should really hold onto, which is, the concept’s been proven, but how are we proving the value to each organization?
And at the moment, I think a lot of enterprise productivity is focused on specific use cases. So you can see time savings, you can see cost savings, you can find things you might not even know existed within your landscape, but it’s going to broaden out massively. And we’re definitely just at the start when it becomes beyond just one use case and it all interacts with each other.
Mark M: Yeah, Mark, I think it was about last February the euphoria was building and a lot of people couldn’t make sense of it. So we really tried to fight the temptation to go broad and get really, really small and really, really concrete. And rather than doing pass-through AI in our experience, which was, “Hey, summarize something in Smartsheet,” we said, “Let’s find where there’s a lot of foot traffic, where there’s a lot of pain.” And I found it’s really easy, especially for a business user, to enroll them in something they’re very familiar with that has been very painful and then magically make it a lot easier where they can articulate the benefit. So an example would be if I said, “Mark, here’s our massive store manifest for build out and I’d like you to do some analysis on it,” you’d be like, “Okay, how do I do a Smartsheet widget on a dashboard and how do I get my axes sorted?” Or you can ask the question.
And the first time I showed that to a business user and they asked that question in natural language and it produced the chart, they’re like, “What just happened? At first, there was disbelief, and that person was able to recite exactly the benefit. So rather than trying to make this massive aperture, get it really, really small. And then other pieces in our app, we do a lot of business logic and it comes in the form of formulas, and it’s our number one vector for support. 40% reduction in that vector. And often, when we think about productivity, we think about how much faster or better can I do something? I think what people often forget is there’s a whole audience out there that is boxed out. It’s not about getting faster or better. They used to not do.
Arvind: Just enable.
Mark M: And for me, it’s more of an unlock than it is a, how much better can we produce something? And that’s pretty exciting, because I think, again, most of humanity is not participating right now.
Mark N: Yeah. For sure. And maybe building on that just in the enterprise, because there’s also this level of trust. Like you said, the first time they see it, it seems like magic. And then you go, “Is it real? Did I get the right answer?” And again, it’s one thing in the commercial space, it’s another thing in the enterprise. What have you seen in terms of the level of willingness to adopt this new modality and trust that it’s actually getting the right answer?
Eleanor: So we work with lawyers and, as you can imagine, trust is probably the most important thing, trust and accuracy. And so the way that we built our, well, we use lots of different AI models, but fundamentally we’ve built it to a different end goal, which is, if the AI doesn’t know the answer or if it’s not sure of the answer, tell the human it’s not sure because lawyers would appreciate that. And fundamentally, we let people try it before they buy it, which is so important. And a good example, as I said, it works across all different types of contracts and pretty much any interaction that you’re having with a contract. But we say, “Let’s focus on the super repetitive stuff. Let’s focus on the high-volume things like reviewing an NDA.” Imagine if you could just press a button and the AI will renegotiate the whole NDA to be more aligned with your standards and your company’s positions.
How do you trust that? You read it. You read the red lines. You don’t necessarily have to totally hand it all over to the AI, but the more times that you start pressing that button, you start reviewing the red lines, and you start seeing, yeah, that’s exactly how I would negotiate, the more that you trust it and the more you start to adopt it for more complex contracts. So again, mine’s very specific, but focus on the super things that are high, low-risk. Get those off of your plate, and then you can start building that trust.
Arvind: And to answer your question on, are people willing to try it out, there’s no question, I think all enterprises are trying to stay ahead on this, make sure that they evaluate tools, and see what works for them and what doesn’t. There is also very little, I would say, belief right now in AI products. Actually, there’s hardly any enterprise I would say that will let AI do some work and I won’t see what it did. Putting a human in charge, making it evaluate and assess what AI is doing is definitely what companies are doing right now. But I think the way I think about this, this is a different question, AI I think is a fundamentally different technology and I actually feel it doesn’t need to be accurate. Actually, you have to think of AI as being just like a human.
How you work with a human, we have certain ways of doing that, how you work with a machine being trained to actually use it in a different way. And I think what we’re hearing right now is that, “Well, I have the expectation of both the human and the machine from this AI system.” And I think that’s the education and training that needs to happen where I think the companies that actually make the best out of AI today, if they think of it as a new type of technology which is imprecise, which can be incorrect, but it’s actually creative and you can work with it and it can actually help you do something. That’s what I feel is what needs to happen, AI education for the entire workforce and making people be okay with imprecise responses.
Eleanor: You obviously haven’t worked much with lawyers, but I agree with you totally across the board.
Mark N: And Mark, you see such a broad spectrum of workloads, where have people really embraced it across Smartsheet?
Mark M: I think the demand that people have on getting data into systems is very high and how I almost view people, they almost feel like they’ve accomplished something when the input is complete. It’s almost like this store, forget mindset. And like what you’re doing, Arvind, you’re unlocking the ability to retrieve, and when I think about changing people’s mindset to, your job’s not done when you store it. What are you doing with it now? And I think people, in a sense, we need to be reprogrammed into a different mental math. So if we’re dramatically driving down the cost of inquiry, you need to change. The question isn’t that expensive anymore.
And I think what’ll be quite interesting is, as we talk about the impact to workers and we as employers, I think what AI will allow us to do is to actually distinguish between the people who really can move quickly and those who don’t. I think today it’s actually sometimes elusive. Is that person really contributing max effort? I think that separation will grow now. So I think people are trying to. The demand we’re hearing from right now, I think we’ve progressed a lot in the last year and now people are much more rigorous around the actual value something produces. The whole jazz hands that carry the day a year ago, less so now, and they’re really looking for that concrete return.
Mark N: So maybe if we turn to the way we think about this enterprise productivity and the returns exactly there. Should we think about it like the CEO of Klarna, who was like, “I want to get 50% of people gone”? Should I think about it as my people are this much more productive? Will people really get replaced or is this a matter of making the people we have much more productive? What do you think?
Arvind: So one, we have a lot of customers who use Glean to make their customer service agents more productive. And our statistics show that we are able to improve case resolution time by 40%. And these are big numbers. In the past, companies would be really, really happy if you could get 2%, 3% reduction in these, how fast you can service your customer requests. It translates into ROI. So there’s a big impact and I think the CFO, of course, always wants that to translate into a lower head count for the next year. But that’s where I think right now things are not exactly there. Even these customers that we have who are seeing big success, they can actually see the metrics, they can see the case resolution times going down, but headcount plans are not changing. And I think everybody has the mindset of, well, I would like to do more with what I have as opposed to reducing my bottle line.
Mark M: I think it’ll be less abrupt than some people say. I think we will grow into this. I mean, much like I remember vividly 18 years ago in San Francisco, somebody said, “If you don’t do this in 30 days, you’re done.” I’m like, “Well, okay, see me in a decade.” And some things do take longer. I think in this case we have this juxtaposition of things moving at incredible pace, and then you have this inertia which is part of enterprises. So when I think about the scenario that some people brought up, it’s like a person and the model and that will be the future company. So let’s talk about the resilience of your business and, what happens if that one person leaves? Again, extreme example.
But I do think business operators will get much smarter about where they allocate capital to people and we will expect a lot more from our people, and I just think the cut line is going to be much more tightly managed than it once was. So I think that makes me a little nervous just from a society standpoint. I think a lot of people won’t clear that hurdle and I think people are going to have better metrics to distinguish between those who do perform and who don’t. So I don’t have the answer there, but I think smart operators will become much more ruthless on that front.
Mark N: Yeah.
Eleanor: I’m not sure that there’s a dichotomy between efficiencies and replacement today, so I would agree with you. I think that, look, if you go back in time, when the tractor arrived, the way that farming happened, that really, really revolutionized everything. Since the [inaudible 00:15:35] we’ve been seeing this huge leap forwards, and that’s exactly what Gen AI has introduced today, but we also have a good way of finding other roles for ourselves. And so, yes, there might be certain roles that will get accelerated and might even be automated, but then I think there’ll be the creation and introductions into roles that we don’t even know today, really.
Mark M: Mark, one thing to add. When I talk to other software CEOs, we often chuckle in that people think that there’s some finite number of things we want to build. You realize that list is never-ending. So this notion of everyone’s going to become more productive, we’ll run out of stuff or we’ll cut half our teams, no, we’re just going to build a lot more stuff. There’s this huge demand out there and that hasn’t changed over 20 years. I think that demand will likely remain very high.
Mark N: Which is great. There’s no end of problems to solve, and if we all get more productive, then in theory we’ll all solve more problems. So maybe talk a little bit about the interface to what all this looks like. Brad was up here talking about enterprise GPT. Do we really think we’re going to sit in some experience, I certainly hope it’s not a chatbot all day, or do we think it’s still like today where there are specialized tools, maybe there are agents? What do you think that looks like in the future? Are we still hopping between tools? Do we really have a personal work agent that we’re spending all day interacting with and sending off to do tasks for us? What do you think on specialized versus general experience and what enterprise productivity looks like?
Eleanor: Look, I think today to build that trust that you were talking about earlier, we’ve got to meet end users with where they’re comfortable and where they’re used to. Because a whole piece that we don’t talk about is the change management to these Gen AI models. And so if you can meet them in Microsoft Word or in Excel, wherever it is, then you’re encouraging adoption. In the future, however, I think that the world’s going to be a place that, even five years time, those portals will look very different.
Mark N: Yeah. And to correlate to that question as you go through, what do you think that means for established vendors? Not to pick on Salesforce, but is Salesforce doomed or is Salesforce just part of this? What do you think?
Eleanor: What do I think? I think it’s interesting, isn’t it? Normally, with these legacy technology companies, something new comes along and it shifts the ground underneath them and it’s super disruptive. In this case, it’s going to be really interesting because they have the data. Data’s one part of the conversation. There’s the engineering behind it, there’s the tagging the data first and then the engineering, and it’s not just about, as we know, large language models. And so if they can integrate with their existing infrastructure well and they can find that startup mentality, which we know gets harder as you build bigger businesses, so if you can be that wonderful, word and agile and flexible and you can drop barriers, then yeah, there’s a huge opportunity for them. But I think there’s a lot of work that needs to be done as well.
Arvind: See, we are the enterprise GPT where we’re connected with all of your enterprise information and we give you this conversation interface. You can treat Glean like a sidekick, work with it, talk to it, and get your work done. But I don’t think that’s the way work is going to happen in the future. That’s not the only way. You’ll have a personal sidekick, an AI companion that can help you as an executive assistant helps an executive today, but I think a lot of work will still continue to happen in those purpose-built applications that are out there.
One thing that will change is that all these applications that are trying to charge for being smart, they won’t be able to do that in the future. There’ll be basic expectation that if I’m using a video conferencing tool, if I’m using a CRM, those smart capabilities are just ingrained as part of the product that I buy. And yes, I’ll still use those systems. I’ll interact with them. They will all have amazing AI capabilities as well in addition to me having a sidekick next to me that I can converse with and to work with.
Mark M: I think the power of context is really important too. We were talking, I think, yesterday, Mark, around, I go in and I have a question and I find it. And let’s say I asked that through a prompt, I get the answer. Well, what I didn’t ask was, there also a really big issue right now that maybe the CRM system could have presented to me. The context is so powerful. So when I think about the inputting and the retrieval of information, that frame within which it sits, whether it’s my HRS system or my ERP system or my Salesforce system or a Smartsheet system, that’s super powerful.
We talked about Tableau. It’s the, “I’m looking for my chart. Oh, I didn’t know I could filter on that dimension. Hadn’t even thought of that. It wasn’t even part of my prompt.” So I think that’s one really powerful reason for why they’re going to remain very relevant provided they stay modern. I mean, you can’t just rest on your laurels. They have to modernize. But I think if you’re in the business of retrieving and aggregating, yeah, you’re probably going to be really disrupted.
Mark N: Awesome. So I’m going to do one more question, and then hopefully, we’ll have time to throw it open to the audience and maybe I’ll throw this to Mark first because this came up and when we were talking in the prep about, so with all this productivity, is it like the Google Mind guy and we’re all going to get time to stay on the beach? Or are we just getting on a treadmill that’s going to run faster and faster and faster, and we’re going to be little gerbils going (round and round)?
Mark M: I think we’ll be closer to the latter.
Mark N: That’s depressing.
Mark M: Jamie, my wife, will be thrilled. No. I think there’ll be this temporary moment of, wow, what a huge gain, and then we benefit from it. I don’t think you head to the beach, but you’re like, “Wow, that was a huge win,” and then that void gets filled really quickly. So I think the pace, the expectation’s going to go up. I think there’ll be a lot more innovation, there’ll be a lot of benefits coming from it, but I don’t think people clip the coupon and sit. I don’t think so.
Eleanor: I would agree with that. I think that we’re very good at keeping ourselves busy, even if we’re trying not to. So I think that, yeah, some of the jobs that we are used to today or that we see today, those will change dramatically. But as I said, there’ll be the creation of other things, and there’ll always be problems to solve.
Arvind: Yeah. I mean, I don’t disagree with either of those statements. I do feel, though, that I think the future is a bit farther than today in anything. We’re close to having all of our work done. Right now AI is still very much like the human is the master and needs to be in control and use it. In some ways, get 15%, 20% productivity gains for now. But I think the time always gets filled up. I think looking at my own engineering team, now with all these tools they’re getting. Maybe our committed is going up, but that means the product team is actually asking them to get more things done.
Mark N: Do more.
Mark M: I think you feel much of it, Mark, will also be driven by someone’s personality type. I almost feel bad for the people who are super curious, because they’re going to have a very unsettled life. You’re going to be asking a lot. You have so much power at your disposal and there’s this insatiable appetite for. Those people who don’t exhibit that, they might lead a more peaceful life.
Mark N: If you can, put your phone down because now your phone’s going to be hounding you instead of you looking at it. Awesome. Well, thanks for all that. We have time for a couple of questions from the audience. Who’s got a question? Over there, Ellie.
Question: I love the jazz hands comment and Salesforce is now Agentforce, so everything’s there. You all talked a lot about the experience of AI in your products, building trust, and that’s great for existing customers. As you’re going to market against all the crazy hype and magic and Shazam out there, how do you break through or are you finding it possible to break through on a better quality of AI than just the jazz hands?
Mark M: Look at our Head of Product here in the audience. We just turned on AI for every single trial user. So we have about, I don’t know, 300,000 people a quarter coming in and trialing new. And rather than reserving it for the enterprise plan, it’s like put it out there. So if you want to create, add some business logic to your workflow, you can use an AI agent and you need to invite people. And I think also the degree to which you can nudge someone and really put it in front of them, because not a natural motion. I think that’s we as providers have that duty to really try and not force people, but make them know it’s available.
Eleanor: Yeah, we’re the same as Mark. I think if you’ve got conviction in your technology, give it to someone for free for a short period of time, whether it’s a week, two weeks, three weeks. But have conviction that you can be put alongside anyone else out there and that you’ll show your return on investment and why you’re different. I couldn’t agree with you more.
Mark N: Awesome. One more question. Moe, you got your question in last year.
Question: I did. I’m trying to monopolize. So quick question from a business model perspective. Does the panel think it’s a zero-sum game? Is AI making building a software company much more expensive? How do you see that?
Eleanor: I don’t think it is, but it depends. It depends on the models that you’re using, if you’ve built your own models internally. It depends on how you commercialize the product. And again, I might say something quite controversial here, but it’s not just about the LLMs, is it? You can be super efficient if you have, we use a panel, we call it a panel of judges, it’s a panel of experts approach to things, and some models are more expensive than others. We need to use those for certain tasks, but we can use other models for other tasks. And I think also, as we heard this morning, the cost of tokens is going to go down. I think a lot of this is going to be commoditized. So my answer is no, I don’t think so.
Arvind: I’m not sure if I fully understood the question, but I think AI is absolutely creating opportunity for many entrepreneurs who start software companies. And I think it’s also creating opportunities for every company to build much better products than what they had before. I think folks who execute, folks who can build great products for them, it’s actually absolutely a game changer and a big opportunity.
Mark N: Awesome. Thank you so much to our panelists. Come back again next year, and we’ll see how productive we are.