Every Product Manager should be an AI Product Manager
Artificial Intelligence is heading towards explosive growth and we are looking at a 400 billion dollar industry in the next 5 years. Today more than a third of the companies around the globe are using AI in some capacity. So naturally, product managers with AI and machine learning skills are in high demand.
Four years ago, while building an enterprise collaboration product at Cisco, I ventured into machine learning and AI. The business problem at hand was to improve the first touch experience and reduce day 1 churn. My team and I explored different ways to increase user stickiness. One of our findings was that users are more likely to come back if they have instant gratification i.e. immediately chat or call someone. That’s when the idea of building an onboarding chatbot came up and we started prototyping and experimenting with the idea. Since then, I have built a number of AI products like voice assistants and price recommendation engines for real-world customers.
While AI is cool, let me point out some of the gotchas. The most successful AI products are not the ones that are built for the sake of AI as a tech initiative, but rather the ones which solve a real business problem and need machine learning to deliver the last mile. You can quickly detect the fail switch when a product touted as a pathbreaking AI product does not manage to find customers or revenue. The decision to use machine learning should grow on the team organically to solve problems that cannot be solved by traditional statistical analysis. That’s why I strongly believe every product manager today should be well versed in the nuances of machine learning and data management. Every product manager should be an AI product manager.
Do you know more than half of the AI initiatives fizzle out in the pilot state? How can product managers help steer the ship in the right direction and increase the success rate of AI initiatives? What are the key skills that you need to be an AI product manager?
Align the team to take a business-first approach
There no doubt that AI is a shiny new technology that excites executives, data scientists, and the rest of the organization. As a product manager, it’s extremely important to clearly define the business problem, have a clear success metric, and align everyone on the value that you are creating. Then work with the team (your designer, engineers and data scientists) to figure out the optimal way to solve the problem. Many times you won’t need to take the Ferrari to a derby.
For e.g. when you are building a chatbot should you start with a rule-based bot with predefined rules or do you need a self-learning bot with natural language understanding? The former is simpler to build and can in fact address most use cases. A self-learning bot with deep learning techniques takes more time to train but is a must-have if you expect the bot to be exposed to a lot of new scenarios which is impossible to design with rule-based approach.
Use data to grok the data infrastructure
You cannot build a successful AI product without a well-defined data strategy. As a Product Manager, you need to be super conversant with data types, volumes, pipelines, and associated tooling. Work closely with your data science team to define the use cases, accuracy goals, and corresponding data requirements. Be ready to solve the hard problems of data availability, data procurement, training data creation, and AI data workflows. Also partner closely with legal and security teams to evaluate security, privacy, and governance aspects of the data early in the development process not as a checklist for launch.
Be ready to operate under uncertainty
In traditional software products, you can define all the use cases and scenarios, your QA team can test according to spec and you can confidently provide acceptance of features. The outcome is more or less predictable. Compared to traditional software development written with well-defined data sets, AI systems are built with tens and thousands of real-world data set scenarios, which are not known when you are writing requirements and it’s a continuous learning process for the models as well as you as the product manager. So an AI product manager needs to navigate through a lot of unknowns; the only realistic path to success is to conduct an iterative evaluation of the capabilities and the limitations to strategize what you can ship in the product launch.
Create a culture of experimentation and intelligent risk-taking
You will learn about new data use cases every day. To move fast and be nimble enough to absorb ever-changing requirements, build a culture of frequent experimentation and risk-taking. But how can you do that without risking a degraded user experience?
Advice #1 – “Don’t try to boil the ocean”. Also use strategies like launching to a selected cohort of users, set expectations with the end-user on the limitation of the model, have a fallback option with humans reviewing the model output, get transparent feedback from users, etc. These real-world learning and setbacks are what you need to build a high performing AI experience. So always take the risk to experiment and learn.
Advice #2 – Try simulated offline experimentation to quickly iterate through different solutions and course correct. In many cases, offline model evaluation is not sufficient and you have to A/B test models in production.
Advice #3 – Continuously track progress towards the business KPI e.g. increase revenue, reduce churn, improve user engagement etc. While tracking the model output attributes like accuracy, execution time, and precision are important; the ultimate outcome that you are driving are the business results.
Be curious and become a quick learner
In many ways, AI is still in its nascent stages of growth. The technology, tools, and applications are evolving fast. As a product manager, you have to build trust with your users, engineers, data scientists and executives. It’s ok to say you don’t know.
If you have to provide valuable feedback and challenge your engineers to build the most scalable and optimal product, you have to get into the weeds and acquire deeper domain expertise. Start with gaining familiarity with different machine learning algorithms (regression, clustering, neural network), model evaluation techniques, accuracy measures (accuracy, precision, recall, F1), etc.
Stay curious and keep yourself abreast of current trends, tooling, and changing market landscape. I hope to work with you on your next AI project !!!