Tag: Intelligent Applications

AWS re:Invent 5th Anniversary Preview: Five Themes to Watch

The 5th Annual AWS re:Invent is a week away and I am expecting big things. At the first ever re: Invent in 2012, plenty of start-ups and developers could be found, but barely any national media or venture capitalists attended. That has all changed and today, re:Invent rivals the biggest and most strategically important technology conferences of the year with over 25,000 people expected to be in Las Vegas the week after Thanksgiving!

So, what will be the big themes at re: Invent? I anticipate, from an innovation perspective, they will line up with the 3 layers of how we at Madrona think about the core of new consumer and enterprise applications hitting the market. We call it the “Future of Applications” technology stack shown below.

future-of-applications
Future of Applications (Madrona Venture Group Slide, November 2016)

The Themes We Expect at 2016 re:Invent

Doubling Down on Lambda Functions

First is the move “beyond cloud” to what is increasingly called server-less and event-driven computing. Two years ago, AWS unveiled Lambda functions at re:Invent. Lambda quickly became a market leading “event-driven” functions service. The capability, combined with other micro-services, allows developers to create a function which is at rest until it is called in to action by an event trigger. Functions can perform simple tasks like automatically expanding a compute cluster or creating a low resolution version of an uploaded high resolution image. Lambda functions are increasingly being used as a control point for more complicated, micro-services architected applications.

I anticipate that re:Invent 2016 will feature several large and small customers who are using Lambda functions in innovative ways. In addition, both AWS and other software companies will launch capabilities to make designing, creating and running event-driven services easier. These new services are likely to be connected to broader “server-less” application development and deployment tools. The combination of broad cloud adoption, emerging containerization standards and the opportunities for innovating on both application automation and economics (you only pay for Lambda functions on a per event basis) presents the opportunity to transform the infrastructure layer in design and operations for next-generation applications in 2017.

Innovating in Machine and Deep Learning

Another big focus area at re:Invent will be intelligent applications powered by machine/deep learning trained models. Amazon already offers services like AWS ML for machine learning and companies like Turi (prior to being acquired by Apple) leveraged AWS GPU services to deploy machine learning systems inside intelligent applications. But, as recently reported by The Information, AWS is expected to announce a deep learning service that will be somewhat competitive with Google’s TensorFlow deep learning service. This service will leverage the MXNet deep learning library supported by AWS and others. In addition, many intelligent applications already offered to consumers and commercial customers, including AWS stalwarts such as Netflix and Salesforce.com, will emphasize how marrying cloud services with data science capabilities are at the heart of making applications smarter and individually personalized.

Moving to Multi-Sense With Alexa, Chat and AR/VR

While AWS has historically offered fewer end-user facing services, we expect more end-user and edge sensors/devices interactions leveraging multiple user interfaces (voice, eye contact, gestures, sensory inputs) to be featured this year at re:Invent. For example, Amazon’s own Alexa Voice Services will be on prominent display in both Amazon products like the Echo and third party offerings. In addition, new chat-related services will likely be featured by start-ups and potentially other internal groups at Amazon. Virtual and augmented reality use cases for areas including content creation, shared-presence communication and potentially new device form factors will be highlighted. Madrona is especially excited about the opportunity for shared presence in VR to reimagine how people collaborate with man and machine (all powered by a cloud back-end.). As the AWS services stack matures, it is helping a new generation of multi-sense applications reach end users.

Rising Presence of AWS in Enterprises Directly and With Partners

Two other areas of emphasis at the conference, somewhat tangential to the future of applications, will be the continued growth of enterprise customer presentations and attendance at the conference. The dedicated enterprise track will be larger than ever and some high-profile CIO’s, like Rob Alexander from Capital One last year, will be featured during the main AWS keynotes. Vertical industry solutions for media, financial services, health care, and more will be highlighted. And, an expanding mix of channel partners, that could include some surprising cloud bedfellows like IBM, SAP and VMWare, could be featured. In addition, with the recent VMWare and AWS product announcements, AWS could make a big push into hybrid workloads.

AWS Marketplace Emerging as a Modern Channel for Software Distribution

Finally, the AWS Marketplace for discovering, purchasing and deploying software services will increase in profile this year. The size and significance of this software distribution channel has grown significantly the past few years. Features like metered billing, usage tracking and deployment of non “Amazon Machine Image (AMI)” applications could see the spotlight.

Over the years, AWS has always surprised us with innovative solutions like Lambda and Kinesis, competitive offerings like Aurora databases and elastic load balancing, as well as customer centric solutions like AWS Snowball. We expect to be surprised, and even amazed, at what AWS and partner companies will unveil at re: Invent 2016.

Evolving the Application Platform from Software to Dataware

Every decade, a set of major forces work together to change the way we think about “applications.” Until now, those changes were principally evolutions of software programming, networked communications and user interactions.

In the mid-1990s, Bill Gates’ famous “The Internet Tidal Wave” letter highlighted the rise of the internet, browser-based applications and portable computing.

By 2006, smart, touch devices, Software-as-a-Service (SaaS) and the earliest days of cloud computing were emerging. Today, data and machine learning/artificial intelligence are combining with software and cloud infrastructure to become a new platform.

Microsoft CEO Satya Nadella recently described this new platform as “a third ‘run time’ — the next platform…one that doesn’t just manage information but also learns from information and interacts with the physical world.”

I think of this as an evolution from software to dataware as applications transform from predictable programs to data-trained systems that continuously learn and make predictions that become more effective over time. Three forces — application intelligence, microservices/serverless architectures and natural user interfaces — will dominate how we interact with and benefit from intelligent applications over the next decade.

In the mid-1990s, the rise of internet applications offered countless new services to consumers, including search, news and e-commerce. Businesses and individuals had a new way to broadcast or market themselves to others via websites. Application servers from BEA, IBM, Sun and others provided the foundation for internet-based applications, and browsers connected users with apps and content. As consumer hardware shifted from desktop PCs to portable laptops, and infrastructure became increasingly networked, the fundamental architectures of applications were re-thought.

By 2006, a new wave of core forces shaped the definition of applications. Software was moving from client-server to Software-as-a-Service. Companies like Salesforce.com and NetSuite led the way, with others like Concur transforming into SaaS leaders. In addition, hardware started to become software services in the form of Infrastructure-as-a-Service with the launch of Amazon Web Services S3 (Simple Storage Service) and then EC2 (Elastic Cloud Compute Service).

Smart, mobile devices began to emerge, and applications for these devices quickly followed. Apple entered the market with the iPhone in 2007, and a year later introduced the App Store. In addition, Google launched the Android ecosystem that year. Applications were purpose-built to run on these smart devices, and legacy applications were re-purposed to work in a mobile context.

As devices, including iPads, Kindles, Surfaces and others proliferated, application user interfaces became increasingly complex. Soon developers were creating applications that responsively adjusted to the type of device and use case they were supporting. Another major change of this past decade was the transition from typing and clicking, which had dominated the PC and Blackberry era, to touch as a dominant interface for humans and applications.

Software is programmed and predictable, while the new dataware is trained and predictive.

Matt McIlwain

In 2016, we are on the cusp of a totally new era in how applications are built, managed and accessed by users. The most important aspect of this evolution is how applications are being redefined from “software programs” to “dataware learners.”

For decades, software has been ­programmed and designed to run in predictable ways. Over the next decade, dataware will be created through training a computer system with data that enables the system to continuously learn and make predictions based on new data/metadata, engineered features and algorithm-powered data models.

In short, software is programmed and predictable, while the new dataware is trained and predictive. We benefit from dataware all the time today in modern search, consumer services like Netflix and Spotify and fraud protection for our credit cards. But soon, every application will be an intelligent application.

Three major forces underlie the shift from software to dataware which necessitates a new “platform” for application development and operations and these forces are interrelated.

Application intelligence

Intelligent applications are the end product of this evolution. They leverage data, algorithms and ongoing learning to anticipate and improve interactions with the people and machines they interact with.

They combine three layers: innovative data and metadata stores, data intelligence systems (enabled by machine learning/AI) and the predictive intelligence that is expressed at an “application” layer. In addition, these layers are connected by a continual feedback loop that collects data at the points of interaction between machines and/or humans to continually improve the quality of the intelligent applications.

Microservices and serverless functions

Monolithic applications, even SaaS applications, are being deconstructed into components that are elastic building blocks for “macro-services.” Microservice building blocks can be simple or multi-dimensional, and they are expressed through Application Programming Interfaces (APIs). These APIs often communicate machine-to-machine, such as Twilio for communication or Microsoft’s Active Directory Service for identity. They also enable traditional applications to more easily “talk” or interact with new applications.

And, in the form of “bots,” they perform specific functions, like calling a car service or ordering a pizza via an underlying communication platform. A closely related and profound infrastructure trend is the emergence of event-driven, “serverless” application architectures. Serverless functions such as Amazon’s Lambda service or Google Functions leverage cloud infrastructure and containerized systems such as Docker.

At one level, these “serverless functions” are a form of microservice. But, they are separate, as they rely on data-driven events to trigger a “state-less” function to perform a specific task. These functions can even call intelligent applications or bots as part of a functional flow. These tasks can be connected and scaled to form real-time, intelligent applications and be delivered in a personalized way to end-users. Microservices, in their varying forms, will dominate how applications are built and “served” over the next decade.

Natural user interface

If touch was the last major evolution in interfaces, voice, vision and virtual interaction using a mix of our natural senses will be the major interfaces of the next decade. Voice is finally exploding with platforms like Alexa, Cortana and Siri. Amazon Alexa already has more than 1,000 voice-activated skills on its platform. And, as virtual and augmented reality continue to progress, voice and visual interfaces (looking at an object to direct an action) will dominate how people interact with applications.

Microsoft HoloLens and Samsung Gear are early examples of devices using visual interfaces. Even touch will evolve in both the physical sense through “chatbots” and the virtual sense, as we use hand controllers like those that come with a Valve/HTC Vive to interact with both our physical and virtual worlds. And especially in virtual environments, using a voice-activated service like Alexa to open and edit a document will feel natural.

What are the high-level implications of the evolution to intelligent applications powered by a dataware platform?

SaaS is not enough. The past 10 years in commercial software have been dominated by a shift to cloud-based, always-on SaaS applications. But, these applications are built in a monolithic (not microservices) manner and are generally programmed, versus trained. New commercial applications will emerge that will incorporate the intelligent applications framework, and usually be built on a microservices platform. Even those now “legacy” SaaS applications will try to modernize by building in data intelligence and microservices components.

Data access and usage rights are required. Intelligent applications are powered by data, metadata and intelligent data models (“learners”). Without access to the data and the right to use it to train models, dataware will not be possible. The best sources of data will be proprietary and differentiated. Companies that curate such data sources and build frequently used, intelligent applications will create a virtuous cycle and a sustainable competitive advantage. There will also be a lot of work and opportunity ahead in creating systems to ingest, clean, normalize and create intelligent data learners leveraging machine learning techniques.

New form factors will emerge. Natural user interfaces leveraging speech and vision are just beginning to influence new form factors like Amazon Echo, Microsoft HoloLens and Valve/HTC Vive. These multi-sense and machine-learning-powered form factors will continue to evolve over the next several years. Interestingly, the three mentioned above emerged from a mix of Seattle-based companies with roots in software, e-commerce and gaming!

The three major trends outlined here will help turn software applications into dataware learners over the next decade, and will shape the future of how man and machine interact. Intelligent applications will be data-driven, highly componentized, accessed via almost all of our senses and delivered in real time.

These applications and the devices used to interact with them, which may seem improbable to some today, will feel natural and inevitable to all by 2026 — if not sooner. Entrepreneurs and companies looking to build valuable services and software today need to keep these rapidly emerging trends in mind.

I remember debating with our portfolio companies in 2006 and 2007 whether or not to build products as SaaS and mobile-first on a cloud infrastructure. That ship has sailed. Today we encourage them to build applications powered by machine learning, microservices and voice/visual inputs.

This post was originally published by TechCrunch

 

Takeaways from the 5th Annual Data Science Summit

The 2016 Data Science Summit just wrapped up in San Francisco and it was bigger and better than ever. With over 1,300 attendees over two days, the conference combines business and academic leaders in a broad mix of machine learning areas – bringing together the latest in research with the state of the art in the industry. Many of the speakers are both leaders at key technology companies and involved with the top research institutions in the U.S.

Carlos Guestrin, with both Turi (previously Dato) and University of Washington, framed the world of intelligent applications including the opportunities for automating machine learning processes, creating online, closed-loop systems and increasing trust in machine learning applications.

Pedro Domingos, author of The Master Algorithm and also a UW professor, outlined the five schools of machine learning, their underlying philosophical approaches and the types of problems they best address.

Jeff Dean from Google highlighted their powerful new service TensorFlow along with its rapid adoption and independent forks in the open source community. Jeff emphasized that TensorFlow has potential beyond the deep learning area as an end-to-end system for Machine Learning applications.

While Jeff highlighted several Google ML use cases, Robin Glinton from Salesforce.com and Jure Leskovec from Pinterest (and Stanford University) impressed the audience with detailed examples of how to build and continually improve intelligent applications.

Stepping back, there are several observations from this conference that generally confirm and expanded upon learnings from Madrona’s recent AI/ML Summit in Seattle.

  1. Deep Learning is both real and overhyped. Deep learning is very well suited for image recognition problems and is growing in areas like speech recognition and translation. However, deep learning is only one branch of machine learning and is not the best approach for many intelligent application needs.
  1. Greater agility is required for intelligent applications in production. Agility comes in many forms, including automating development processes like data munging and feature engineering. It also applies to model training and ongoing model iterations for deployed intelligent apps. Automated, end-to-end pipelines that continually update production applications are rapidly becoming a requirement. These applications, like the ones consumers experience with Netflix and Spotify recommendations are increasingly referred to as “on line” applications due to their agility in both making real time recommendations and bringing data back to update models.
  1. “Closed” loops and “humans-in-the-loop” co-exist. Many intelligent applications become business solutions by involving humans to verify, enhance or act on machine outputs. These “humans-in-the-loop” cases are expected to persist for many years. However, intelligent applications increasingly require automated, closed-loop systems to meet narrow business requirements for performance and results. For example, product recommendations, fraud predictions and search results are expected to be more accurate and relevant than ever and delivered in milliseconds!
  1. The strategic value of differentiated data grows by the day. Intelligent applications are dependent on data, metadata and the models this data trains. Companies are increasingly strategic about the data they collect, the additional data they seek and the technologies they use to more rapidly train and deploy data models. Google’s internal use cases leveraging data like RankBrain are expanding. And, their decision to “open source” data models for image and speech recognition built on TensorFlow is a leading example of engaging the outside world to enhance a model’s training data.

Overall, I found the conference extremely energizing. There was substantial depth and a diversity of backgrounds, ideas and experiences amongst the participants. And, the conference furthered the momentum in moving from academic data science to deployed intelligent applications.

Machine Learning and AI. Why Now?

Trying to go to the moon, but today we’re at the top of a tree

You can hardly talk to a technology executive or developer today without talking about artificial intelligence, machine learning, or bots. Madrona recently hosted a conference on ML and AI bringing together some of the biggest technology companies and innovative startups in the Intelligent Application ecosystem.

One of the key themes for the event emerged from a survey of the attendees. Everybody who responded to the survey said that ML is either important or very important to their company and industry. However, more than half of the respondents said their organizations did not have adequate expertise in ML to be able to do what they need to do.

Here are the other top 5 takeaways from the conversations at the summit.

Every application is going to be an intelligent application

If your company isn’t using machine learning to detect anomalies, recommend products, or predict churn today, you will start doing it soon. Because of the rapid generation of new data, availability of massive amounts of compute power, and ease of use of new ML platforms (whether it is from large technology companies like Amazon, Google, Microsoft or from startups like Dato), we expect to see more and more applications that generate real-time predictions and continuously get better over time. Of the 100 early-stage start-ups we have met in the last six months, 90%+ of them are already planning to use ML to deliver a better experience for their customers.

Intelligent apps are built on innovations in micro-intelligence and middle-ware services

Companies today fall into two categories broadly – ones that are building some form of ML/AI technology or ones that are using ML/AI technologies in their applications and services. There is a tremendous amount of innovation that is currently happening in the building block services (aka middle-ware services) that include both data preparation services and learning services or models-as-a-service providers. With the advent of microservices and the ability to seamlessly interface with them through REST APIs, there is an increasing trend for the learning services and ML algorithms to be used and re-used as opposed to having to be re-written from scratch over and over again. For example, Algorithmia runs a marketplace for algorithms that any intelligent application can use as needed. Combining these algorithms and models with a specific slice of data (use-case specific within a particular vertical) is what we call micro-intelligence that can be seamlessly incorporated into applications.

Trust and transparency are absolutely critical in a world of ML and AI

Several high profile experiments with ML and AI came into the spotlight in the last year. Examples include Microsoft Tay, Google DeepMind AlphaGo, Facebook M, and the increasing number of chat bots of all kinds. The rise of natural user interfaces (voice, chat, and vision) provide very interesting options and opportunities for us as human beings to interact with virtual assistants (Apple Siri, Amazon Alexa, Microsoft Cortana and Viv).

There are also some more troubling examples of how we interact with artificial intelligences. For example, at the end of one online course at Georgia Tech, students were surprised to learn that one of the teaching assistants (named Jill Watson after the IBM Watson technology) they were interacting with throughout the semester was a chat bot and not a human being. As much as this shows the power of technology and innovation, it also brings to mind many questions around the rules of engagement in terms of trust and transparency in a world of bots, ML and AI.

Understanding the ‘why’ behind the ‘what’ is often another critical component of working with artificial intelligence. A doctor or a patient will not be happy with a diagnosis that tells them they have a 75% likelihood of cancer, and they should use Drug X to treat it. They need to understand which pieces of information came together to create that prediction or answer. We absolutely believe that going forward we should have full transparency with regards to ML and think through the ethical implications of the technology advances that will be an integral part of our lives and our society moving forward.

We need human beings in the loop

There have been a number of conversations on whether we should be afraid of AI based machines taking over the world. As much as advances in ML and AI are going to help with automation where it makes sense, it is also true that we will absolutely need to have human beings in the loop to create the right end-to-end customer experiences. At one point, Redfin experimented with sending ML-generated recommendations to its users. These machine-generated recommendations had a slightly higher engagement rates than a users’ own search and alert filters. However, the real improvement came when Redfin asked its agents to review recommendations before they were sent out. After agents reviewed the recommendations, Redfin was able to use the agents’ modifications as additional training data, and the click-through rate on recommended houses rose significantly. Splunk re-emphasized this point by describing how IT Professionals play a key role in deploying and using Splunk to help them do their jobs better and more efficiently. Without these humans in the loop, customers won’t get the most value out of Splunk. Another company Spare5 is a good example of how humans are sometimes required to train ML models by correcting and classifying the data going into the model. Another common adage in ML is garbage-in, garbage-out. The quality and integrity of data is critical to build high quality models.

ML is a critical ingredient for intelligent applications. But you may not need ML on day one.

Machine learning is an integral part and critical ingredient in building intelligent applications, but the most important goals in building intelligent apps are to build applications or services that resonate with your customers, provide an easy way for your customer to use your service, and continuously get better over time. To use ML and AI effectively, you often need to have a large data set. The advice from people who have done this successfully before is to start with the application and experience that you want to deliver, and in the process think about how ML can enhance your application and what data set you need to collect to build the best experience for your customers.

In summary, we have come a long way in the journey towards every app being an intelligent app, but we are still in the early stages of the journey. As Oren Etzioni, CEO of the Allen Institute for AI said in one fireside chat, we have made tremendous progress in AI and ML, but declaring success in ML today is like “Climbing to the top of a tree and declaring we are going to the moon.”

Previously published by TechCrunch.

The Intelligent App Ecosystem (It’s not just bots!)

Intelligence App thumbnail
Click to view the full image PDF (1.8 MB)

Today we interact with many intelligent applications like the Google and Bing search engines, Spotify and Netflix media services, and the Amazon shopping experience. The machine learning technologies that power these services are becoming mainstream and setting the stage for the Intelligent App Era.

Application intelligence is the process of using machine learning technology to create apps that use historical and real-time data to make predictions and decisions that deliver rich, adaptive, personalized experiences for users.

We believe that every successful, new application built today will be an intelligent application. The armies of chat bots and virtual assistants, the ecommerce sites that show the right recommendations at the right time, and the software that detects anomalous behavior for cybersecurity threats, to name a few, are all built to learn and create continuously improving experiences. In addition, legacy applications are becoming more and more intelligent to compete and keep pace with this new wave of applications.

We believe that every successful, new application built today will be an intelligent application.

S. Somasegar & Daniel Li

Now is an exciting time to be investing in the broader intelligent app ecosystem because several important trends are coming together in application development:

  • The availability of massive computational power and low-cost storage to feed machine learning models
  • The ease with which developers can take advantage of data sources and machine learning techniques,
  • The adoption of microservices as a development paradigm for applications, and
  • The proliferation of platforms on which to develop applications, and in particular platforms based on “natural user interfaces” like messaging and voice

We have spent time thinking about the various ways Intelligent Apps emerge – and how they are built. This Intelligent App Stack illustrates the various layers of technology that are crucial to the creation of Intelligent Apps. (Please send us feedback on this world view! @SSomasegar @danielxli )

As investors we like to think about the market dynamics of major industry shifts, and the rise of intelligent apps will certainly create many new opportunities for startups and large technology companies alike. Here are some thoughts on the key implications for companies operating at various layers of the intelligent app stack:

“Finished Services”: Applications will define the end user’s experience with machine learning
At the application layer there will be two primary classes of applications: net-new apps that are enabled by application intelligence and existing apps that are improved by application intelligence.

somasagar-gridNet-new apps will need to solve the tough problem of determining how much end users will pay for “artificial intelligence” and how to ensure they capture a portion of the value delivered to users. More broadly, it will be interesting to see if our thesis that the value proposition of machine learning will primarily be a revenue generator comes true.

Also because of the importance of high-quality, relevant data for machine learning models, we think that use-case specific or industry-specific applications will be the most immediate pockets of opportunity at the Finished Services or application layer. Today, we see the main categories of use-case specific applications as autonomous systems, security and anomaly detection, sales and marketing optimization, and personal assistants. We are also seeing a number of interesting vertically focused intelligent applications especially serving the retail, healthcare, agriculture, financial services, and biotech industries.

The killer apps of the last generation were built by companies like Amazon for ecommerce, Google for search and advertising, Facebook for social, Uber for transportation, and Netflix for entertainment. These companies have a significant head-start in machine learning and user data, but we believe there will be apps that are built from the ground up to be more intelligent that can win in these categories and brand new categories that are enabled by application intelligence.

Interfaces: New interfaces will transform applications into cross-platform “macro-services”
As we think about how new intelligent applications will be developed, one significant approach will be the transformation of an “app” to a service or experience that can be delivered over any number of interfaces. For example, we will see companies like Uber build “services” that can be delivered via an app, via the web, and/or via a voice interface.

It will also be easier for companies to deliver their services across platforms as they design their apps using a microservices paradigm where adding a new platform integration might be as simple as adding a new API layer that connects to all of the existing microservices for authentication, product catalog, inventory, recommendations, and other functions.

The proliferation of new platforms such as Slack, Facebook Messenger, Alexa, and VR stores will also be beneficial for developers because platforms will become more open, add features that make developers lives easier, and compete for attention with offerings such as investment funds.

Finally, at the interface layer, we see the “natural interfaces” of text, speech, and vision unlocking new categories such as conversational commerce and AR/VR. We are incredibly optimistic about the future of these interfaces as these are the ways that humans interact with one another and with the world.

Building Blocks and Learning Services: Intelligent building blocks and learning services will be the brains behind apps
As companies adopt the microservices development paradigm, the ability to plug and play different machine learning models and services to deliver specific functionality becomes more and more interesting. The two categories of companies we see at this layer are the providers of raw machine intelligence and the providers of trained models or “Models as a Service.”

In the first category, companies provide the “primitives” or core building blocks for developers to build intelligent apps, like algorithms and deployment processes. In the second category, we see intermediate services that allow companies to plug and play pre-trained models for tasks like image tagging, natural language processing, or product recommendations.

These two categories of companies provide a large portion of the value behind intelligent apps, but the key question for this layer will be how to ensure these building blocks can capture a portion of the value they are delivering to end users. IBM Watson’s approach to this is to provide developer access to its APIs for free but charge a 30% revenue share when the app is released to customers. Others are charging based on API calls, compute time, or virtual machines.

li-gridThe key differentiators for companies in this layer will be the ability to provide a great user experience for developers and the accuracy and performance of machine leaning algorithms and models. For complicated, but general problems like natural language understanding, it will likely be easier and more performant to use a pre-built model from a provider who specializes in generating the best data, models, and processes. However, for specialized, business-specific problems, startups and enterprises will need to build their own models and data sets.

Data Collection and Prep: The difficult and boring tasks of data collection and preparation will get smarter
Before data is ready to be fed into a machine intelligence workflow or model, it needs to be collected, aggregated, cleaned, and prepped. Sources of data for consumer and enterprise apps include photos and video, websites and text, customer behavior data, IT operations data, IOT sensor data, and data from the web.

After applications are instrumented to collect the right pieces of raw data, the data needs to be transformed into a machine-ready format. For example, companies will need to take unstructured data like text documents and photos and transform it into structured data (think of rows and columns) that is ready for a machine to review.

The important part of this step is realizing that the quality of a model is highly dependent on the quality of its input data. Creating bots or ‘artificial intelligences’ without high quality training data can lead to unintended consequences (see Microsoft’s Tay), and the creation of this training data often relies on semi-manual processes like crowdsourcing or finding historical data sets.

The other area of this space to keep an eye on is the companies that have traditionally served as “dumb” pipes for data sources like clickstream data or application performance logs. Not only will they try to build predictive and adaptive features, they will also see competition from intelligent services that draw insights from the same data sources. This will be an area of innovation for finance, CRM, IT Ops, marketing, HR, and other key business functions that have traditionally collected data without receiving immediate insights. For example, HR software will become better at providing feedback for interviewers and highlighting the best candidates for a position based on historical data from previous hires.

Data Infrastructure: Intelligent apps will be built on the “Big Data” infrastructure
The amount of data in the world is doubling every 18 months, and thanks to this explosion in big data, enterprises have invested heavily in storage and data analysis technologies.

Projects like Hadoop and Spark have been some of the key enablers for the larger application intelligence ecosystem, and they will continue to play a key role in the intelligent app stack. Open source will remain an important feature for choosing an analytics infrastructure because customers want to see what is ‘under the hood’ and avoid vendor lock in when choosing where and how to store their data.

The amount of data in the world is doubling every 18 months, and thanks to this explosion in big data, enterprises have invested heavily in storage and data analysis technologies.

S. Somasegar & Daniel Li

Within the IaaS bucket, each of the major cloud providers will compete to run the workloads that power intelligent apps. Already we are seeing companies open source key areas of IP such as Google’s TensorFlow ML platform, in a bid to attract companies and developers to their platform. Google, in particular, will be an interesting company to watch as they give users access to their machine learning models, trained on some of the world’s largest data sets, to grow their core IaaS business.

Finally, hardware companies that specialize in storing and managing the massive amount of photos, videos, logs, transactions, and IOT data will be critical to help businesses keep up with the new data generated by intelligent applications.

There will be value captured at all layers of this stack, and there is the opportunity to build significant winner-take-all businesses as the machine learning flywheel takes off. In the world of intelligent applications, data will be king, and the services that can generate the highest quality data will have an unfair advantage from their data flywheel – more data leading to better models, leading to a better user experience, leading to more users, leading to more data.

Ten years from now, all applications will be intelligent, and machine learning will be as important as the cloud has been for the last 10 years. Companies that dive in now and embrace intelligent applications will have a significant competitive advantage in building the most compelling experiences for their users and as a result, the most valuable businesses.

This post was previously published on TechCrunch.com

 

McIlwain and DeVore Share their Thoughts on the Seattle Startup Scene

Will a new crop of Seattle startups rise to world domination? Will virtual or augmented reality applications create the next powerhouse tech company?

Those are two of the overriding hopes and ambitions emerging from the Seattle segment of my informal cross-country survey of leading investors as I seek to unearth core investment themes and premises as we dive into 2016.