For business executives, staying relevant in this time of innovation and rapid technological advancement can be challenging. Thankfully, we can leverage consumer and marketplace data to help our companies optimize products and services.
One of the premier analytical reports on emerging business-related trends, Mary Meeker’s annual Internet Trends 2018, was just released. While the more than 290 slides in the full report come packed with useful information for driving your company forward, here are 12 important and actionable insights for business executives.
Amazon has jumped ahead of search engines as the jumping-off point for product searches. Nearly half of all product search queries (49 percent) start on Amazon instead of on the traditional search engines like Google and Bing (36 percent each) S62. Searches that start on Amazon are typically fulfilled on Amazon, whereas searches that start on search engines are typically filled elsewhere. S63-S64
Source: 2018 Internet Trends Slide 62
Social sharing is having a significant influence on purchasing decisions, with 55 percent of consumers queried saying they purchased products online after discovering them on social platforms. Facebook leads in discovery, with 78 percent of respondents having discovered a product on this platform, followed by picture-heavy Instagram and Pinterest, both with 59 percent. S71
In addition, purchases from social media links are increasing, making up 6 percent of today’s referrals S72. Facebook’s e-commerce click-through rates grew to 3 percent, from just 1 percent in 2016. S74
Source: 2018 Internet Trends Slide 71
Mobile was the only medium to see an increase in time spent, and accordingly, was the only medium that saw an increase in advertising spend. All other mediums were down or flat. S96
Source: 2018 Internet Trends Slide 96
Past Mary Meeker reports have correctly predicted the explosion of video, gamification and personalization. The 2018 report highlights the expansive growth of video and messaging. Voice-driven searches, now at 95 percent accuracy, have spurred confidence and growth in this segment, with the voice-driven Amazon Echo device today found in 30 million households S26. As for messaging, there are now more than one billion monthly active users on each of the most popular messaging apps, including WhatsApp, Facebook Messenger, and We Chat. S22
E-commerce acceleration continued at a 16 percent growth rate in the past year S45 and now represents 13 percent of all sales transactions s46. Easy-to-use tools including mobile payments, cloud computing, hosting and analytics makes it easy for anyone to open an online store. Amazon Web Services (AWS), which provides on-demand cloud computing platforms to individuals, companies, and governments, expanded from one service in 2006 to 140+ services in 2018 S183. As a result, the number of online merchants is growing rapidly S52, but Amazon is the share leader, with 28 percent of e-commerce gross merchandise. S47
Source: 2018 Internet Trends Slide 45
Subscription-based services are seeing significant adoption. Year-over-year subscriptions have increased significantly for powerhouses like Netflix, the New York Times and Dropbox. In fact, 45 percent of Spotify’s monthly active users are subscribers vs. 0 percent in 2008, the year it launched. Key reasons, say respondents, include access, selection, price, experience and personalization. S81-82
Source: 2018 Internet Trends Slide 81
Costs per clicks are increasing, so businesses are placing increased emphasis on Customer Lifetime Value (CLV) as a key performance measure of ad spending performance. S76
Source: 2018 Internet Trends Slide 76
The innovation of personalization and convenience of business data results in an overall gain for end users. Therefore, the end user will continue to share their personal and collective data to provide better experiences. However, experts predict there will be more regulations in data collection. As a result, businesses will increasingly need to demonstrate responsible data use. S204-S205
Artificial Intelligence (AI), a machine’s ability to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages, is expected to be the one of the most important general-purpose technologies of our era. AI is becoming widely available, and Chief Information Offers (CIOs) say networking equipment and Artificial Intelligence projects will enjoy some of the largest increase in IT spending this year. S198-S202
While it is known by many names, including “On-Demand,” “Freelance,” “Fractional,” and “Temp,” the “Gig Economy” is growing – and will constitute a significant portion of what employment will look like in the future. In 2017, 8.1 percent of growth occurred in the freelance workforce, versus 2.5 percent of growth in the total workforce S163. Technology improvements make the gig economy easy and inexpensive to participate in.
The uptick is occurring in part because this style of work provides flexibility, which is cited among the most desired non-monetary benefits for workers S162. Other advantages include increased income and asset utilization S161-S176. Examples of freelance-friendly organizations include Uber, Airbnb, Etsy, Upwork, 99 designs, and Fiverr.
Source: 2018 Internet Trends Slide 163
Given the speed of change and new technologies, staying competitive means constantly staying ahead of trends and “lifelong learning.” Chiefmartec.com estimates there are now 6,800 marketing technology solutions alone. How do today’s enterprises keep smart, strong talent relevant? Unfortunately, the time needed to run the company operationally hampers internal employees’ abilities to stay up to date on innovations. S232-S232
Better tools and greater access are helping with continuous learning, but speed of innovation is one of the reasons the gig economy will continue to increase. We can expect businesses to keep product/company expertise in house while outsourcing more of the functional expertise. In just the past six months, 55 percent of freelancers participated in skill related training, versus 30 percent for non-freelancers. YouTube streams 1 billion hours of education video per day, and 70 percent of the platform’s users access it to help solve work, school, and hobby problems. Educational channels like Asap Science, Crash Course, TED-X, and Khan Academy now have millions of subscribers. S234-S236
During the last 30 years, shelter, insurance, and healthcare have consumed a larger proportion of household budgets, while consumer spending on food, transportation, entertainment, and apparel has declined S105. Some of the decrease in spending is due to price declines from increased competition, like Walmart, as well as e-commerce. S111
Business leaders can expect price declines initiated online to expand to brick and mortar stores, as the technology that made “online shopping more effective with lower prices, more selection and increased conveniences” continues to move to offline shopping. S110-S114
Source: 2018 Internet Trends Slide 112
Capitalizing off Critical Internet Data
Technology continues to quickly change the landscape for both Business to Business and Business to Consumer enterprises. Mary Meeker’s presentation provides insights and data that we have seen historically shed light on emerging business trends. It is one of the premier publicly-available resources helpful in developing and modifying strategies and tactics for business growth, and I hope in spotlighting some of the most business-critical findings, these topics help to shape discussion about the present state and future aspirations of your business
One of the big challenges managers face when planning Industrial IoT (IIoT) projects is choosing the right architecture or approach for solving business problems. Managers don’t simply decide to buy some IoT technology one day and then install it. Instead, they look at an issue, such as how to reduce energy on a production line or how to lower maintenance costs on wind turbines, and then apply IoT technologies. And that is the problem: too many IoT options but too few case studies that provide best practices. This situation is starting to change, however. A couple of examples follow.
First, the Industrial Internet Consortium (IIC) has produced a new white paper that offers practical guidance for deploying IIoT solutions within the concept of edge computing—a hot topic in the industry. The 19-page white paper defines edge computing architectural functions and underscores some key use case factors. The IIC white paper is aimed at technical people who have to implement IoT solutions, but this is the sort of document and shared learning necessary to drive wider adoption.
I spoke recently with two of the authors of the white paper, Mitch Tseng of Huawei and Todd Edmunds of Cisco, who pointed out that defining where the computing takes place in IIoT is less important than harnessing the technology in the right way to achieve valuable business outcomes. They also noted that both edge and cloud computing are important to many IIoT use cases, and the key is in orchestrating the various resources to optimize the outcomes.
Their work is not finished, by any means. The next step calls for the IIC to produce a more technical report that addresses in greater detail how to implement an IIoT architecture that is managed, orchestrated, trustworthy, and secure. Engineers who need to deploy IIoT solutions should benefit greatly from the collective thinking in that yet-to-be-published document.
Mark Venables recently noted the complexity in IIoT and the challenge to provide new tools in his online piece about Thing Query Language (TQL). He highlights Atomiton, a company founded 5 years ago that developed TQL as an operating system for enabling machines, equipment, or devices to talk to each other and that can be programmed, similar to Microsoft’s Visual Basic Programming language. The software is currently used in oil & gas, smart cities, agriculture, and industrial automation settings, but could be applied in other sectors as well. Atomiton was founded by Jane Ren, one of the original founders of GE’s digital arm.
Atomiton is not the only technology vendor working to smooth the pathway for IIoT implementations, of course. Other examples of companies providing valuable IIoT products or solutions include PTC, OSIsoft, Siemens, AWS, Microsoft, Oracle, and C3 IoT.
Solving industrial problems with IIoT solutions is still in its early phase. No one company has the full stack of products or services to meet the corporate demand. A group of vendors working together or through an integrator has proven successful. As the Navigant Research Leaderboard: IoT Platform Vendors report noted, there are hundreds of firms offering solutions, which makes for a complex and sometimes confusing ecosystem. So, when efforts to simplify or provide valuable or tested approaches in using IIoT technology become widely known, it helps drive adoption and reduce wasted efforts. I’m all for that.
Most people didn’t notice last month when a 35-person company in San Francisco called HoneyBook* announced a $22 million Series B.
What was unusual about the deal is that nearly all the best-known Silicon Valley VCs competed for it. That’s because HoneyBook is a prime example of an important new category of digital company that combines the best elements of networks like Facebook with marketplaces like Airbnb — what we call a market-network.
Market-networks will produce a new class of unicorn companies and impact how millions of service professionals will work and earn their living.
“Networks” provide profiles that project a person’s identity and then lets them communicate in a 360-degree pattern with other people in the network. Think Facebook, Twitter, GoodReads*, Meerkat*, and LinkedIn.
What’s unique about market-networks is that they:
An example will help: let’s go back to HoneyBook, a market-network for the events industry.
An event planner builds a profile on HoneyBook.com. That profile serves as her professional home on the Web. She uses the HoneyBook SaaS workflow to send self-branded proposals to clients and sign contracts digitally.
She then connects the other professionals she works with like florists and photographers to that project. They also get profiles on HoneyBook and everyone can team up to service a client, send each other proposals, sign contracts and get paid by everyone else.
This many-to-many transaction pattern is key. HoneyBook is an N-sided marketplace — transactions happen a 360-degree pattern like a network, but they come here with transacting in mind. That makes HoneyBook both a marketplace and network.
A market-network often starts by enhancing a network of professionals that exists offline today. Many of them have been transacting with each other for years using fax, checks, overnight packages, and phone calls.
By moving these connections and transactions into software, a market-network makes it significantly easier for professionals to operate their businesses and clients to get better service.
AngelList* is also a market-network. I don’t know if it was the first, but Naval Ravikant and Babak Nivi deserve a lot of credit for pioneering the model in 2010.
On AngelList, the pattern is similar. The CEO of the startup creates her own profile, then prompts her personal network of investors, employees, advisors and customers to build their own profiles. The CEO can then complete some or all of her fundraising paperwork through the AngelList SaaS workflow, and everyone can share deals with everyone else in the network, hire employees, and find customers in a 360-degree pattern.
In 2013, when I met Oz and Naama Alon, two of the founders of HoneyBook, they were building a beautiful network product — a photo-sharing app for weddings. We sat down and I walked them through the new idea of a market-network. They embraced it immediately, and have taken it to a whole new level – from the design and workflow to the profile customization and business model.
Houzz* is a third good example. Houzz connects homeowners with home improvement professionals and with products they can buy for their home. They have a product that is very nearly a market-network. The company raised $165M in its last round.
Joist is another good example. Based in Toronto, it provides a market-network for the home remodel and construction industry. Houzz is also in that space, with broader reach and a different approach. DotLoop in Cincinnati shows the same pattern for the residential real estate brokerage industry.
Looking at AngelList, Joist, DotLoop, Houzz and HoneyBook, the market-network pattern is visible.
Seven Attributes Of A Successful Market-Network
In the last six years, the tech industry has obsessed over on-demand labor marketplaces for quick transactions of simple services. Companies like Uber, Lyft*, Mechanical Turk, Thumbtack, DoorDash* and many others make it efficient to buy simple services whose quality is judged objectively. Their success is based on commodifying the people on both sides of the marketplace.
However, the highest value services – like event planning and home remodels — are neither simple nor objectively judged. They are more involved and longer term. Market-networks are designed for these.
With complex services, each client is unique and the professional they get matters. Would you hand over your wedding to just anyone? Your home remodel? The people on both sides of those equations are not interchangeable like they are with Lyft or Uber. Each person brings unique opinions, expertise, and relationships to the transaction. A market-network is designed to acknowledge that as a core tenet and provide a solution.
Collaboration happens around a project
For most complex services, multiple professionals collaborate among themselves—and with a client—over a period of time. The SaaS at the center of market-networks focuses the action on a project that can take days or years to complete.
Pleasing profiles with information unique to their context give the people involved a reason to come back and interact here. It captures part of their identity better than elsewhere on the Web.
Market-networks bring a career’s worth of professional connections online and make them more useful. For years, social networks like LinkedIn and Facebook have helped built long-term relationships. However, until market-networks, they hadn’t been used for commerce and transactions.
In these industries, referrals are gold, for both client and service professional. The market-network software is designed to make referrals simple and more frequent.
By putting the network of professionals and clients into software, the market-network increases transaction velocity for everyone. It increases the close rate on proposals and speeds up payment. The software also increases customer satisfaction scores, reduces miscommunication, and makes the work pleasing and beautiful. Never underestimate pleasing and beautiful.
First we had communication networks like telephones and email. Then we had social networks like Facebook and LinkedIn. Now we have market networks like HoneyBook, AngelList, DotLoop, Houzz and Joist.
You can imagine a market-network for every industry where professionals are not interchangeable: law, travel, real estate, media production, architecture, investment banking, personal finance, construction, management consulting, and more. Each market-network will have different attributes that make it work in each vertical, but the principles will remain the same.
Over time, nearly all independent professionals and their clients will conduct business through the market-network of their industry. We’re just seeing the beginning of it now.
Market-networks will have a massive positive impact on how millions of people work and live, and how hundreds of millions of people buy better services.
I hope more entrepreneurs will set their sights on building these businesses. It’s time. They are hard products to get right, but the payoff is potentially massive
What should our product team work on today? The answer is usually on your product roadmap. It contains a list of features, ideas, tasks to do, small and significant projects and multi-team efforts that must be achieved. Along with that, all of them have a deadline. So everyone can grab a task from the roadmap and not worry about what to do next. This means we can define a roadmap as:
The product roadmap is a prioritised list of features and projects with an end date.
Usually, a product roadmap comes from the management team or the product manager. Both parties, have reasonable answers on why they want one and why you should follow it. The most common reasons are:
These are right answers to the question why do we need a product roadmap. But we miss one thing that stayed in the darkness for too long:
Roadmaps are the cause of most waste and failed efforts in product organisations. They are a graveyard for more than half of listed features and ideas that will never succeed.
The harsh truth is that, with all our best intentions, half of our product ideas will not work. More realistic teams expect that number to jump up to 75%. Because we all know that things almost never go as planned when it comes to building a product. If being more specific, some of the reasons are:
You can embrace these truths and you ride along with them or you can ignore them and experience the effects on yourself.
So I embarked on a mission to find out how weak and strong product teams cope with those truths listed above. What are they doing differently from each other? And I found out that:
Month after month, they follow all the tasks on the roadmap and make sure that everything is done on time. And once something goes wrong they start blaming the management for poor goal setting. After that, they ask for more time to redesign the “feature” and improve it. And if they had enough time and money, they would eventually solve the problem and achieve the desired solution, but that is rarely the case.
The ideal state is not to ignore these truths but work around them and adapt to the ever-changing needs from a different perspective. Strong teams do not start putting a list of features on the roadmap, but they start with a product discovery stage first.
The problem with roadmaps is that we put on it features to build, but it should have business problems to solve. What is the underlying problem? And not just deliver a feature.
A product discovery is a stage when the team tackles: 1) what are the business goals? 2) what are the measurable KPI’s? 3) all the risks and problems that are to overcome, 4) and are fast at iterating the ideal and effective solution for the main business problem they have predefined.
The problem with putting a list of features on the roadmap is that once it’s out, people will interpret it as a commitment. No matter how many disclaimers you attach or notes you leave, people will still have a fixed goal in mind — design or build that feature. The team will start developing and delivering the next task on the list, even when it does not solve a problem.
On the other side, yes, we do need a list of features and hard dates to build and deliver stuff. But what I am suggesting is to take a step back. And transition from being a feature building team, to a business problem-solving team. So I don’t recommend removing the roadmap but improving it with an extra step before committing to anything.
I would like to make it clear that the problem lies in committing too early to a goal before you even established any business objectives. People make commitments before they even know if this is something they could deliver, and most important, will it solve the customer’s problem. Having a list of features to add with a deadline is crucial. But I am speaking of a broader context — product vision.
Adding a discovery and delivery model in advance is changing everything. Discovery is all about an intense collaboration between product management, UX and engineering. The purpose of discovery is to separate the good ideas from bad quickly. The output of product discovery is a validated product backlog. This means getting answers to these questions:
Let’s suppose that it requires two weeks to onboard a big client. But for the company to scale we need to reduce that amount of time from two weeks to a couple of hours. That’s an excellent example of a great business goal: “Reduce the amount of time required to onboard new big size customers.” And for a measurable objective, it would be: “Onboarding time less than one hour.”
Now imagine having a product roadmap with only business goals on it. This kind of roadmap opens room for creativity and innovation to come in. People feel more open and safe sharing their ideas. So now we are not limited by a set of features.
First, we ask our key stakeholders to give us a little bit more time for product discovery, to investigate the solution and execution plan. We tell them that we need time to understand if this goal resonates with our customer and whether it is going to improve the overall experience. Then we talk with engineers to ensure that it can be built and iterate on the findings and ideas from a technical point of view. After that we communicate with our stakeholders or management team to see if the solution found achieves our business goals.
Now once we aligned all three parties — customers, engineers and management team — we can start making the execution plan. This plan includes our business goal, measurable KPI and the rest of the details, together with an end date.
It is essential for companies to make this kind of commitments from time to time and get comfortable with switching to it entirely. Because this way we can indeed improve our products by listening to customers and committing to things that are critical to our company rather than doing incremental changes without seeing the bigger picture. And even if you do it rarely, it is better than not doing it.
Industrial digitalization is transforming manufactured products into smart products. Automobiles and aircraft, for example, are quickly becoming computers with mobility. Moving human involvement from direct navigation to remote operation (drones, self-driving cars) is the most obvious result of how digitalized, smart products change the operational model.
Digitalization in product development is also relocating direct human involvement in maintenance and operations. Built-in IT allows products to perform self-analysis and to report out valuable operational data, changing human involvement to a more executive role of overseeing the automated process of deciding when a part is under stress, needs maintenance or should be replaced. Such analytic parts must be designed for connectivity and sharing—which opens up management and control issues between vendors and product owners.
Just as operational models are changing, so too are the business models that deliver products and services. Is an aerospace vendor obligated to provide IT support for an airplane? If a vendor wants to sell services around product data retrieval, does that mean the vendor is claiming ownership of the data? Should products be sold as physical product only, with full digitalization only available for an extra fee? Right now there are no simple answers.
Take the case of the U.S. Air Force as an example. It wants to use operational data to predict when its cargo and refueling aircraft need maintenance or part replacement. Their newer aircraft have the capability to monitor and send the necessary data, but they can’t do it under existing contracts. The Air Force owns the jets, but the rights to diagnostic information are controlled by the primary vendors of airframes and engines. “I want to own the data,” says Gen. Carlton Everhart, who leads Air Mobility Command. “Our airplanes have the capability right now; all I have to do is put it on contract,” the general said at an Air Force Association conference last year. “We have the capability; we just didn’t buy the capability.”
In the case of the Air Force, the contractors are Boeing and Lockheed Martin for the airframes, and Pratt & Whitney and General Electric for the engines. There is no question the vendors want to work with the military; it is just a matter of contractual agreements and determining what both sides consider as fair compensation. Because it is for military use, there are special complications, the most obvious of which is security. Any data passing between aircraft and operations must be secure. Everhart thinks it could be several years before he is able to gather the money required to move real-time information to the ground.
The cost of full access to aircraft data will be worth it, Everhart says. He cited the example of a commercial aircraft “telling on itself” by reporting out that it would need maintenance in 40 hours. So the engine was changed out before it flew again. It was only after the swap that a nick was discovered in a fan blade. The estimated cost savings of changing the engine before it failed was $50 million.
The European Union is watching how data ownership and access is changing industrial relationships. An April 2018 study examines the notion that the value of data increases with its aggregation level. Contractual agreements are already in use to control how product data is gathered and used, but the EU is concerned contracts alone may not lead to “broad and fair access to data held,” especially when smaller companies buy products from very large vendors. Small companies want to “valorize” their product data—create value from its use. But relying on contracts alone, the report claims, does not settle many issues. There would be no standards on the definition of intellectual property (IP) and trade secrets as they impact data analytics.
The potential rewards of fully digitalized product data are high, which means there is no lack of companies bringing solutions to market. C3 IoT was created by Thomas Siebel, best known for Siebel CRM Systems, a large database software company acquired by Oracle in 2005. C3 IoT offers data analytics for many industry segments including aerospace and defense. It now has product trials underway, analyzing data from flight records, maintenance logs, sensors and crew observations. “Based on initial results, we think we can improve aircraft availability by at least 25%,” C3 IoT President Ed Abbo told Bloomberg Government. The 2018 Air Force budget for operations and maintenance (O&M) is approximately $45 billion. Every percentage point of improved aircraft availability would trim hundreds of millions of dollars from operational costs.
A recent Accenture Consulting study on data ownership and access models in aerospace and defense reveals two predominant streams of thought in industry. The first is coordination among multiple players (original equipment manufacturers, supply chain and IT providers). The second is central management of data as a comprehensive digital thread. In a survey of A&D companies, 87% see supplier and IP management for both hardware and software as a key aspect of creating and using digital thread technology in the next three years. Accenture sees this current state of data usage as raising more questions than it answers, specifically surrounding data standardization, sharing, IP control and incentives for compliance.
The Accenture survey revealed only 27% of A&D firms have shared ownership of the digital thread and or the digital twin across their IT and business functions. “Of all the enterprise systems and processes, this is an area where a true symbiotic commitment is fundamental,” the Accenture report notes. “While there are clear technical components involved in developing the right systems and protocols, insights that result in effective monetization will necessarily come from the business.”
Accenture is not the only company that suggests data issues transcend engineering. David Knight is founder and CEO of Terbine, a startup for curating industrial physical data from manufacturing to consumer use. “As a rule of thumb, whomever holds title to the data producing platform, likely owns the data,” notes Knight. But there are many uncertainties. “The common denominator is well-crafted contractual language that both protects consumer interests and feeds a growing data ecosystem.”
In a study prepared for IEEE, two Bell Laboratories (Alcatel-Lucent) scientists outline how they see multiple models of data ownership evolving. The first is pay per use. The product becomes part of a provided service agreement and the contract allows the manufacturer data access and use rights in exchange for charging for use of the product based on usage data. The second data ownership model is data market. Those who purchase a product offer the data it generates as a commodity, either to the original manufacturer, a vendor or other parties. The third data ownership model is open data model, for situations like driving, where the data creation takes place in a public space. Such a space would include a notion of intention integrity, meaning device owners must have equal access to the data with manufacturers and public agencies, and the use of the data must be explicitly defined
I’d like to let you in on a secret: when people say ‘machine learning’ it sounds like there’s only one discipline here. There are two, and if businesses don’t understand the difference, they can experience a world of trouble.
Imagine hiring a chef to build you an oven or an electrical engineer to bake bread for you. When it comes to machine learning, that’s the kind of mistake I see businesses making over and over.
If you’re opening a bakery, it’s a great idea to hire an experienced baker well-versed in the nuances of making delicious bread and pastry. You’d also want an oven. While it’s a critical tool, I bet you wouldn’t charge your top pastry chef with the task of knowing how to build that oven; so why is your company focused on the equivalent for machine learning?
Are you in the business of making bread? Or making ovens?
What they don’t tell you is that all those machine learning courses and textbooks are about how to build ovens (and microwaves, blenders, toasters, kettles… the kitchen sink!) from scratch, not how to cook things and innovate with recipes.
If you build machine learning algorithms, your focus is general purpose tools for others to use. (Kitchen appliances, if you prefer the analogy.) This business is called machine learning research and is typically done by places like academia or Google.
When it comes to machine learning, many organizations are in the wrong business.
You need quite a lot of education to be in this line of work, because there’s a long history here. Some popular algorithms have been around for centuries. For example, the method of least squares for regression, was published in 1805. Trust me, humanity has come a long way in 200 years.
Today, there are some pretty sophisticated appliances out there… how are you going to build a better microwave if you don’t know how this one works? Of course you need all that immersive study! Becoming a researcher takes years and there’s a good reason that the 101 course starts with the basics of calculus.
Most businesses just want to get cooking — to solve their business problems. They have no interest in selling microwaves, and yet often make the mistake of trying to build those appliances from scratch. It’s hard to blame them — the current hype and education cycle dominantly focuses on research, instead of application.
If you’re innovating with recipes, don’t reinvent the wheel. Those microwaves exist already. You can get them for free from many places. And if setting up your own machine learning kitchen sounds like a chore, providers like Google Cloud Platform let you use theirs, complete with appliances, ingredients, and recipe books.
If you’re innovating in the kitchen, don’t reinvent the wheel.
For most applications, your team doesn’t need to understand the mathematics of backpropagation in neural networks any more than a chef needs to know the wiring diagram for a microwave. But there’s a lot that you do need to know if you’re planning on running an industrial-scale kitchen, everything from curating your ingredients to checking that your dishes are good before you serve them.
Unfortunately, I see a lot of businesses failing to get value from machine learning because they don’t realize that the applied side is a very different discipline from the algorithms research side. Instead, leaders try to start their kitchens by hiring those folks who’ve been building microwave parts their whole lives but have never cooked a thing. What could possibly go wrong? If that works out, it’s because you got lucky and accidentally hired an engineer who is a great chef.
But usually you’re not lucky. There are only so many hours in one lifetime, and if you spend them learning how a microwave is wired, you’ve got fewer to devote to mastering the art of pastry or business. Where — and when! — would your PhD-trained artificial intelligence researcher have gained the skills required for applied machine learning? If you set your heart on the hybrid who’s an expert in both, no wonder you’re complaining about the talent shortage!
Whom should you hire instead? Just like in an industrial kitchen, you need an interdisciplinary team with leadership that understands this space. Otherwise, projects fizzle and go nowhere.
If you’re selling cutting-edge appliances, hire researchers. If you’re innovating in recipes to sell food at scale, you need people who figure out what’s worth cooking / what the objectives are (decision-makers and product managers), people who understand the suppliers and the customers (domain experts and social scientists), people who can process ingredients at scale (data engineers and analysts), people who can try many different ingredient-appliance combinations quickly to generate potential recipes (applied ML engineers), people who can check that the quality of the recipe is good enough to serve (statisticians), people who turn a potential recipe into millions of dishes served efficiently (software engineers), people who keep the interdisciplinary team on track (project/program managers), and people who ensure that your dishes stay top notch even if the delivery truck brings you a ton of potatoes instead of the rice you ordered (reliability engineers).
While these needn’t be separate individuals, be sure you’ve got each role covered. And before you fling your rotten tomato at me for providing such an incomplete caricature, I’ll freely admit that there’s much more to say about hiring for applied machine learning. Let’s outsource that to a future post.
Speaking of outsourcing, if your team has tried all existing tools and can’t make a recipe that meets your business objectives, it makes sense to think about adding skills in building appliances (researcher). Whether or not you hire that person to your permanent staff or outsource the job to an experienced algorithms research firm depends on the scale and maturity of your operation.
Another reason to connect with researchers is that your prototype is so successful that using custom-built appliances makes sense at the massive scale you’re lucky enough to operate at. (What a great problem to have!)
Experts should be talking about this, but they aren’t. They’re not owning up to the fact that there’s really two machine learnings here, and so the world is training people in building all these algorithms but not in using them.
My team is working to fix that. We’ve created a new discipline to cover the applied side and we’ve already trained over 15,000 staff members in it. We’re calling it decision intelligence engineering, and it spans all the applied aspects of machine learning and data science.
To put it another way, if research machine learning is building microwaves and applied machine learning is using microwaves, decision intelligence engineering is using microwaves safely to meet your goals and using something else when you don’t need a microwave.
When it comes to applied machine learning, the hardest part is knowing what you want to cook and how you plan to check it before you serve it to your customers. That part is actually not that hard — just don’t forget to do it.
As for the rest, solving business problems with machine learning is far easier than most people think. Those gleaming kitchens are waiting for you to come play in them. Dive in as you would in a real kitchen. Start tinkering! Every time I meet someone who believes they need to take a traditional machine learning algorithms course — or, goodness! a whole degree — in order to get started, I can’t help but imagine them refusing to use microwaves until they built one themselves. Don’t fall for the lie that says you need a PhD to do amazing things with machine learning. Instead, what you really need is a bit of human creativity. Good luck and have fun!