Background

Machine Learning and AI are being infused into more software products than before in 2022, and there is, without a doubt, a lot of momentum around the language models, and generative machine learning for text, images, music and many other kinds of data. There is a distinct difference between the useful search, AI, ML and data algorithms being used for most use cases, and the cutting edge of advanced ML as it were, and this means there are upsides for new adopters of the more advanced models. Along with these opportunities come many risks and challenges of using ML systems or subsystems in software products. This post is an endeavor to discuss and cover some of the ground in the space of building machine learning products in general.

A New Landscape of User Needs for AI Products

The Basic Needs: Full Product and an Inviting User Experience

How does one characterize broadly the needs of the users of AI products in 2022? While it is clear that the AI product landscape is evolving and changing, some things do remain common, constant needs. A full fledged product is a requirement. Any AI product team should at the very least be able to deliver a user-facing component and other components, and have the ability to deliver and maintain the application on a frequent basis. What this means for the tech team building and developing the product, is that you need a contemporary application in the form of a service, with a front-end and a back-end to deliver the underlying software product to the end user. The nature of the consumption may change and evolve, leading to a stronger characterization of the end user interactions and requirements. However, this need to have a product end-to-end hasn’t gone away, and can’t be expected to.

Another important need in AI product development as of 2022, is to imagine the user’s experience fully and use this to drive the development process. What constitutes value for the end user, and how will AI or data capabilities deliver this? How are they inclined to obtain this value through your app or product, and what interactions would make sense in this context? Some of these fundamental questions, and some of the common guidelines on building responsive interfaces can help build a compelling experience for end-users.

Although this may not be evident to AI and Data teams at first, the user requirements and the interactions we design for them tend to drive strongly how and what we build in terms of data and AI value in the core of the product.

Evolving AI-centric Needs

Customers and end-users are more aware in 2022 than before about data and data security, the impact of good and big data on models. The tech-aware consumer’s footprint continues to grow, and many tech-aware consumers will be early adopters of AI-centric products and tools. Such end users are likely to evaluate and gauge the quality of the models they consume from the product. They may also demand transparency into how their data is used, and would want to be abreast of how regulations and standards are being followed and observed by the AI products they use. Many users are also likely to ask how the results they receive as part of a product ought to be interpreted, and why certain kinds of results are produced at certain times by their product.

This changing set of requirements has made necessary the need to build more trustworthy AI. This is a broad objective which perhaps encompasses a few different sets of capabilities:

1. Good datasets used to build the machine learning or AI models in question
2. Meaningful and regulation-adherent ways handling of user data in a manner that is transparent to the end-user and aligned with regulation
3. Explainable results from AI models, which implies the ability to interpret the results (inferences) from machine learning models

Let’s break these down even further.

Data quality and Data Management

Outcomes in machine learning and data science are heavily dependent on the quality of the data used to perform statistical inference, or analysis leading up to model training and deployment. If we’re unable to understand the data in question, we’re unable to be effective in building machine learning and data science models effectively. This may sound simple, but is hard to put into practice. Data collection can become a messy process, especially in processes that don’t have rigorous measurement processes defined. Measurement system analysis, a common enough set of approaches in traditional industries such as manufacturing, or in engineering processes, tend to not be well-established for industries such as technology where data from people, processes and business systems is hard to instrument and hard to measure. KPIs may in some cases be tangible – in industries such as logistics, e-commerce and other domains – but even in these cases, there may be substantial ambiguity in how the data are stratified, how they’re counted and measured, and what the implications of variability are for measurement systems.

Another related consideration is how the data is handled and managed – especially where the data concerns the end-user. Some user data needs to be stored and managed to provide a good experience to users, but where does one draw the line? Additionally, how do the data that get stored, get managed over long periods of time? It is good to have a plan in place as a product team way before the product iterations begin, and certainly well before any release of the product.

Explainable results for end users

There is greater user awareness in 2022 than ever before about the importance of being able to explain the results of AI/ML models. To the end user, these considerations may come up as specific product features, because machine learning systems are being implemented in code as software product features. For example, a loan application app, job-candidate matching app, may invite a certain kinds of scrutiny from end users. The end user may ask, “why did X get matched to a certain job while I didn’t?”, or “why is the interest rate on my loan so high despite the prevalent rates for such loans in my area?”. Many users of social networking applications have asked what is being done with their data – we’ve seen broad and strong debate on this topic as well. Especially where machine learning models are used in a user and recommendations context, we invite such questions. Over the few years that I’ve followed this topic, my own understanding of the space of explainable AI has been challenged by Zach Lipton’s paper on model interpretability, which has invited this recent rejoinder on the communications of the ACM.

The practical experiences I have had indicate a preference on the part of end-users to have case based explanations for the interpretability of the results of their entire product. The reasons for this are two-fold, a) the specific context of the end-user and the data they submit is paramount in importance to the inference or result in question, and b) Even if the model’s explainability can be understood by treating the model alone a black box, there are many business rules and validations that are implied in the process of model deployment.

Implications for Data Scientists, Engineers, Architects and Product Managers

The modern data scientist, data engineer and ML engineer tend to be much more cross functional than before, requiring awareness and the ability to implement models of different kinds, and take some of the more complex models to production. In my observation, some of the common tasks have become a lot easier because of improved tooling and the availability of high-level frameworks for implementing some of these more advanced features.

Data Science and ML Model Development Implications

ML model training, development and iteration is commonly understood process in 2022 at the software and code level, but the questions around whether we have used good data or merely big data still remain. There should be more questions asked around what the business problem is, what the data and hypotheses are, and what models to build. It is easy to train models, but hard to undo the results of bad models once deployed especially in production. The availability of powerful frameworks and tools makes life better for those who can carefully think through, hypothesize and execute their model training process in more deliberate ways. This is the differentiating factor in 2022 between teams of data scientists that merely train models, and teams that are able to deliver truly differentiated value to end-users, based on soundly constructed hypotheses. Exploratory data analysis never gets old – it is always important to check assumptions, understand the underlying statistical properties of the data we use, and go back even further to where and how the data was collected, and how it needs to be processed. The availability of foundational models has created a gulf between teams building narrow AI at scale (like an AI/ML startup that may be building a product for processing invoices), teams building specialized narrow AI (like the teams building Alpha Fold or Google’s or Meta’s billion-parameter model teams), and teams researching broad AI (like the teams researching reinforcement learning like with Gato). The achievements of teams that are building narrow AI in highly specialized areas is quite staggering. Teams building more boring AI for routine, run-of-the-mill use cases in banks, telecom companies or manufacturing firms are likely to be doing equally impactful, if less glamorous work, and often have the same hierarchy of needs such as good data, assumptions checking and a rigorous process to train and validate models before deployment. The truth of the matter is that in 2022, not all AI model problem statements are equal – you can get outsize impact for the AI models you’re building if you’re building the right model for the right industry.

Data and ML Engineering Implications

Data and ML engineering in 2022 is a more sorted landscape than it was a few years ago. The changing user needs described above do have implications on how data and ML engineers execute their work. Data engineers are familiar now with tools to prepare, automate and implement data workflows such as ETL and ELT. All big cloud providers have such capabilities baked into their respective ecosystems as well. The focus has probably shifted today in data engineering to the careful management and versioning of datasets required for training machine learning models. Additionally, Data governance, data profiling and quality are the important talking points today, and rightly so, because the mere transfer of data from some source into your data lake doesn’t constitute business value anymore. In some cases, that may even mean the addition of cost and effort for business teams who have to manage, maintain and interpret this data.

On the ML engineering front, there are many opportunities and challenges for engineers who need to build applications which scale. Kubernetes and scalable deployments of endpoints have become common, but the complexity of managing distributed systems like Kubernetes, and doing so with a significant-sized DevOps effort involving infrastructure-as-code is a learning curve for most organizations. Managed services for handling Kubernetes deployments are helpful. There is an evolving discussion on the security of ML APIs that transcends common considerations around APIs such as HTTPS endpoints, or the tradeoff between synchronous and asynchronous endpoints.

While often not discussed, the QA function has become more important in data science than ever before, at least in my experience. It has always been important to write well-tested code, arguably, and the time for doing so and doing so well is now, as far as AI products are concerned. Not only is there a need for data science and ML engineering teams to have good strategies to test their code and ensure good coverage, but there is a strong need for user-focused testing and end-user testing.

Implications for Data and AI Architects

From an architecture perspective, there is certainly a trend towards event-driven systems in the machine learning product landscape. Using event-based architectures centered on technologies like Kafka or other message queues simplifies some of the orchestration around product workflows, and better enables on-demand provisioning and use of resources, in a cost-effective manner. I personally think that architectural decisions need to be made more deliberately and more frequently. The use of event-based architectures can simplify some of the workflows around user data, model inference and logging and observability, which are all parallel considerations alongside the changing user needs discussed above, such as data quality, model explainability, and so on. I recollect an insight from an earlier job when Terraform and infrastructure-as-code were gaining ground in the telecommunications AI space, that it is possible to rearchitect systems more frequently with IaC. This is perhaps an important insight for Data/AI product teams in 2022 on the architecture front – that it is possible and often necessary to iterate often even at the architecture level, as we build up from the basic product needs to more advanced and nuanced needs of end-users.

Implications for Product Managers

Product Managers in AI have a harder time in 2022, compared to earlier years, because of the dual pronged effect of greater regulation and evolving and changing user requirements from AI products. The landscape of the product features has shifted to more pragmatic performance needs, and feature prioritization seems to follow this trend. If there’s one piece of advice I have for product managers in 2022, it is to understand the capabilities and limitations of the underlying technologies behind data and AI systems. Most product managers are not technical and don’t understand the hierarchy of needs ranging from data, models and engineering to DevOps and MLOps as well. These are important for PMs to grok in 2022 and beyond if they’re to be effective at planning and delivering world class AI products.

There are implications for profitability from AI applications and products from these new and changing needs. As AI infused products are built more often and served up to end users, the bottom line is going to be influenced by the architecture and the optimizations of the product, while the top-line is going to be influenced by the scale of the product. Product managers will have to gauge user needs and build products to the appropriate scale for the phase of the product that’s being built – this is crucial, because the developmental costs of AI/data models and products is significant, and there’s a leverage that comes with these, which if used well can help build significant product impact.

Concluding Remarks

We’ve discussed both the evolving and new sets of requirements from end-users of AI products as of 2022, and the implications this has for different sets of stakeholders. From a product envisioning and development perspective, these changes surely imply a maturation of the needs of the market for AI and data infused products. From an inchoate market with aspirational ideas on making data, AI and machine learning work for end-users even just a few years ago, we see a greater tendency in AI product teams to want to build on-point and focused products for end-customers. More product teams are building end-to-end software applications which infuse AI/data capabilities in specific features or as a core feature, but this has come with implications for the end user, who is now more aware and more demanding than before. The resulting challenge for product managers is to characterize these changing needs of the product and incorporate such needs into their product plan. For engineers, this means building new kinds of value in the form of data and model provenance. For data scientists, it means prioritizing good data rather than merely big data.