Kicking off Madrona Engineering Community Meetups with a Side of DevOps

If there is one topic that is on every engineering leader’s mind, regardless of the size of the company, it is how to ship more code more often and how to run that code reliably and efficiently. So, it was a no-brainer that we chose to kick off the first of our Madrona Engineering Community (MEC) meetups with an event on DevOps and Agile practices.

MEC meetups foster ongoing knowledge sharing and networking among engineering leaders across the Madrona portfolio. Our members are VPs of Engineering and CTOs across our portfolio companies.

As far as first events go, we started off with a bang! We had a packed house with over fifty of our engineering leaders attending as our panel of experienced speakers dove into the current and future for DevOps. Our panel was:

  • Tom Casey, SVP Engineering at DocuSign
  • John Ludeman, SVP Engineering at Skytap
  • Adam Johnson, Founder/CEO of IOPipe
  • Moderator, Joe Duffy (Founder/CEO of Pulumi

The make up of the panel was by design — to cover companies at different scale. DocuSign, for example, serves hundreds of millions of users and more than 1.5 million documents per day. Skytap is a mid-stage company that has seen its usage quadruple over the last two years, deploying up to 45,000 VMs and serving up to 2 petabytes of storage a day. IOPipe, in contrast, is in the early days of building its product with a distributed team churning out multiple releases every day! The DevOps goals and plans of each of these companies are different, which led to an interesting discussion under the expert probing of our masterful moderator Joe Duffy.

DevOps = ownership + accountability

While Joe did his best to instigate the speakers in taking opposing positions, he had no luck when it came to how they thought of DevOps. To each, DevOps was about aligning engineering teams with creating customer value and achieving business goals. The key, as each speaker emphasized, was to give developers end-to-end ownership (and the accountability that comes with it). Developers must be able to see the value they are delivering to the customers, or not. And, they must be provided with all the tools and processes, and more importantly, the organization and cultural support needed to succeed.

DevOps ≠ shipping more features

In the fog of war, there is often a tendency to add every feature request to the scrum. Shipping, however, is not a good measure of success. Adoption, utilization, learning, and direct revenue are much better measures of value creation. When it comes to DevOps, reemphasizing value creation and not just features shipped is important. DocuSign, for instance, made driving adoption (of their product) a company wide priority because, direct revenue is typically a trailing indicator of feature value. To keep everyone focused, they evaluated every shipped feature in light of adoption. It mostly worked for them.

DevOps is always a work in progress

No matter how small or big the company is, DevOps is never done. John from Skytap, for example, shared how, after operating for a few years, they realized that their organizational structure was getting in the way of shipping code quickly and efficiently. So, they reorganized and refocused the engineering teams to make sure that they stayed agile as the business scaled rapidly. Tom from DocuSign shared a similar experience, done at a greater scale in a more gradual manner. And, they both fully expect to redo their DevOps process and organization multiple times as their businesses continue to grow quickly.

No “no-ops” unless you also believe in unicorns

In this day and age of serverless, the topic of “no-ops” inevitably came up. To much of Joe’s delight, it elicited some strong reaction from the speakers. Long story short, the verdict was that no-ops does not work at scale. Every SaaS company, once it reaches a certain scale, needs specialized ops to ensure it can deliver the best user experience without blowing up its cost budget. DocuSign has “devs who op,” and every company could do with more of them. Similarly, the topic of SRE did not find a lot of love from the panel. The view was that, for most SaaS applications, it is impossible — or, at least not advisable — to insulate the developers from running the application.

Telemetry trumps testing

The other interesting insight that emerged from the discussion was that, no matter how much one invests in testing, telemetry is indispensable and if anything, more important. Skytap, for example, believes investing more in application telemetry than in running exhaustive regression tests. Similarly, DocuSign puts some of its best devs on telemetry. The emphasis on telemetry helps to speed up release cadence and detect unforeseen bugs.

Looking forward …

While most modern companies have adopted DevOps to a certain degree, most also struggle with several issues. First, balancing agile development with predictability (a must to run a business) is always hard. Hence, every engineering leader clamors for effective planning tools. The other issue is how to measure engineering productivity. Lines of code written or number of bugs fixed could be great vanity metrics, but they do not serve the real purpose. Finally, as we get ready for applications running on multiple clouds, we do not yet fully know the new challenges that will bring to DevOps.

Maybe, that’s where we will pick up at the next MEC meetup!

Re:Invent: 2017 Preview & Predictions

AWS is another year older, bigger and more diverse and so will be the 6th Annual Re:Invent conference. Over 40,000 attendees are expected to attend the event reflecting the success of AWS and the cloud movement that the company kick-started. If AWS was a standalone company, it would be recognized as the software company that hit a $20 billion annual revenue run rate in the shortest amount of time. From a branding perspective, AWS appears focused on courting “builders” including business leaders, product managers and developers who want to create, or recreate in the cloud, solutions that solve real world problems. From a thematic perspective, I anticipate five broad areas to be highlighted:

  1. Modern services for modern cloud apps
  2. ML/AI everywhere!
  3. Hybrid workloads go mainstream
  4. Enterprise agility exceeds cost savings
  5. Customer focus balanced with competitive realities

Modern Services and ML/AI

The first two themes – Modern services and ML/AI are targeted at the grass roots builders and innovators who have long been associated with AWS. Modern services include containerized or “serverless” workloads that work individually or in conjunction with other microservices and event-driven functions like AWS Lambda functions. These technologies deliver greater flexibility, interoperability and cost effectiveness for many applications. And, they can be used to either build new applications or help modernize traditional applications. I have spoken to several smaller businesses and small teams at larger companies who are leveraging these capabilities to build more responsive and cost-effective applications.

Credit to @awsgeek, Jerry Hargrove

At Re:Invent we expect to see AWS embracing community standard like Kubernetes for orchestrating modern containers like Docker. Above is a visual highlighting AWS Elastic Container Service and the use of related services on AWS. AWS will also highlight innovative approaches in the cloud and at the edge that build on Lambda functions to ingest data and automatically produce a functional output. I wouldn’t be surprised to see a “developer pipeline” for building, testing and developing these types of event-driven applications.

ML/AI will likely be broadly highlighted in both Andy Jassy’s Day One keynote and the second keynote on Thursday. This category is where the most disruptive innovation is taking place and the fiercest platform competition is occurring. AWS will feature enhancements or new offerings at four levels.

At a platform level, they are expected to highlight Ironman as a unifying layer to help developers ingest and organize data and then design, test and run intelligent (ML/AI powered) applications. This platform leverages MXNet, which is a machine and deep learning framework originally built at the University of Washington, which has properties similar to Google’s Tensorflow framework. Ironman will leverage a new developer tool framework called Gluon that AWS and Microsoft recently launched.

At a core services level, AWS will continue to enhance AWS ML services and infrastructure processing services like GPU’s and FPGA’s that support the data scientists who can build and train their own data models.

For teams that need more finished ML/AI services, AWS will highlight improved versions of ReKognition, Lex and Polly. I also expect new finished services that leverage pre-trained data models the existing offerings to be announced.

The fourth area of ML/AI will be in the context of leveraging other services either built by AWS or AWS partners that deliver solutions to customers. AWS will likely focus on a combination of running cloud services (AWS, Non-AWS) as well as simplifying ML/AI at the edge. For example, third parties are increasingly building security services like Extrahop’s Addy or Palo Alto Network’s cloud firewall and SAAS security services on AWS. Other services using data stored or processed in AWS, often in data warehouses like Snowflake or Redshift, are rapidly growing for vertical markets and for specific use cases like customer personalization, fraud detection or health recommendations. Seeing what AWS and partners announce in ML/AI powered services across the platform, core services, finished services and solutions layers is likely to be the most exciting area of news at Re:Invent this year.

Matt McIlwain-Madrona Venture Group

“Seeing what AWS and partners announce in ML/AI powered services across the platform, core services, finished services and solutions layers is likely to be the most exciting area of news at Re:Invent this year.”

Hybrid Workloads and Enterprise Agility Solutions

While there are pockets of enterprise innovation in ML/AI and “serverless”, the biggest areas of enterprise focus are going to be hybrid applications and enterprise solutions. These areas also highlight some intriguing partnerships between AWS and other technology companies like VMWare, Microsoft and Qumulo.

Last year AWS on VMWare announced a major partnership where AWS created a dedicated, “bare metal” region for VMWare hypervisors, management tools and more running on AWS. This offering has been in beta all year and appears to be gaining strong enterprise traction. It simplifies moving VMWare-based workloads to AWS and enables hybrid workloads when a portion is on AWS and another portion remains on-premise. Customer examples and new capabilities will likely be announced for this partnership. We don’t expect major announcements around bare metal offerings outside VMWare, but enterprise customers are asking for them to be launched in 2018.

While AWS and Microsoft compete for cloud customers on many levels, there has also been a spirit of partnership between the two companies driven by both enterprise customer demand and competitive realities. Microsoft Windows operating system and applications (SQL Server, ActiveSync Directory, Sharepoint and more) are common applications on AWS in addition to their substantial on-premise installed base. AWS is increasingly enabling their defacto cloud standards like S3 object store and EC2 compute instances to run on-premise. AWS has a service called CodeDeploy that enables EC2 instances to run on-premise or for hybrid workloads (https://aws.amazon.com/enterprise/hybrid/). This enables AWS standard services to work with other Microsoft products on-premise. These examples highlight the growing customer demand for hybrid workloads and services across public cloud and on-premise. And, combined with services like Gluon and the Amazon/Microsoft Voice Assistant partnership, the two Seattle-based technology giants are finding ways to work productively together (often to counteract Google).

Beyond the technology giants, smaller companies like Qumulo will be highlighting hybrid workload flexibility and use cases. Qumulo offers a universal, scale-out file system that allows enterprise customers to scale across on-premise and cloud infrastructure. Technology sectors such as storage where Qumulo is focused, application management where New Relic, DataDog and AppDynamics run, along with databases and security and networking will all see “hybrid” highlighted at Re:Invent.

Beyond individual services and workloads, enterprises continue to look for solutions that help them embrace the agility and cost-effectiveness of cloud computing while mitigating the technology and compliance risks and skill-gaps they may face. AWS will continue to highlight their own professional services as well as a cloud-native solution providers like 2nd Watch and Cloudreach and established “systems integrators” like Accenture and CapGemini. But, I expect AWS will emphasize the growing role of the AWS Marketplace this year as a place to find, buy, deploy and bill first and third-party services from. Finally, more software services will be delivered on AWS in a “fully managed” mode. These modern “managed software services” like the aforementioned cloud data warehouse, database/datastores or storage services will help enterprises embrace cloud native applications.

Balancing Customer Focus with Competitive Realities

All four above themes are driven by customer needs and real technological innovations. But, there are also embedded competitive realities across these themes. Microsoft’s Azure adoption continues to grow rapidly. They have also successfully moved customers to Office365 pulling key services like Azure Active Directory and mobile device management with them. In addition, Microsoft is leveraging their on- premise advantage with hybrid solutions and Azure Stack. These offerings help enterprises embrace agility while cost effectively managing legacy hardware and software. Microsoft also continues to invest and promote their ML/AI and serverless capabilities.

Google has emphasized their ML/AI strength with both the Tensorflow open source adoption as well as leveraging differentiated data sources to build and offer data models “as-a-service”. These image, translation and text recognition models have the opportunity to be strategically disruptive for years to come. Of course, Google also operates broadly adopted cloud apps like Gmail and Google Docs where AWS does not. And, the defacto standard for serverless container orchestration and management, Kubernetes, was created inside Google.

These competitors, as well as other enterprise software and hardware incumbents like Oracle, VMWare/Dell, IBM and Salesforce.com and emerging Chinese competitors like Alibaba will continue to invest and challenge AWS in the years ahead as the enterprise gets more fully engaged in the cloud. While I am confident that AWS will remain the clear market leader for years to come, even they will need to continually “re:Invent” themselves to meet growing customer needs and competitive realities. I will be looking for clues about AWS’s future strategy and approach to emerging competition this week.

Note: Extrahop, Qumulo, 2nd Watch and Snowflake are portfolio companies for Madrona Venture Group, where Matt McIlwain is a Managing Director.