AI Products: The New User Needs Landscape And Implications for Product Teams

Background

Machine Learning and AI are being infused into more software products than before in 2022, and there is, without a doubt, a lot of momentum around the language models, and generative machine learning for text, images, music and many other kinds of data. There is a distinct difference between the useful search, AI, ML and data algorithms being used for most use cases, and the cutting edge of advanced ML as it were, and this means there are upsides for new adopters of the more advanced models. Along with these opportunities come many risks and challenges of using ML systems or subsystems in software products. This post is an endeavor to discuss and cover some of the ground in the space of building machine learning products in general.

A New Landscape of User Needs for AI Products

The Basic Needs: Full Product and an Inviting User Experience

How does one characterize broadly the needs of the users of AI products in 2022? While it is clear that the AI product landscape is evolving and changing, some things do remain common, constant needs. A full fledged product is a requirement. Any AI product team should at the very least be able to deliver a user-facing component and other components, and have the ability to deliver and maintain the application on a frequent basis. What this means for the tech team building and developing the product, is that you need a contemporary application in the form of a service, with a front-end and a back-end to deliver the underlying software product to the end user. The nature of the consumption may change and evolve, leading to a stronger characterization of the end user interactions and requirements. However, this need to have a product end-to-end hasn’t gone away, and can’t be expected to.

Another important need in AI product development as of 2022, is to imagine the user’s experience fully and use this to drive the development process. What constitutes value for the end user, and how will AI or data capabilities deliver this? How are they inclined to obtain this value through your app or product, and what interactions would make sense in this context? Some of these fundamental questions, and some of the common guidelines on building responsive interfaces can help build a compelling experience for end-users.

Although this may not be evident to AI and Data teams at first, the user requirements and the interactions we design for them tend to drive strongly how and what we build in terms of data and AI value in the core of the product.

Evolving AI-centric Needs

Customers and end-users are more aware in 2022 than before about data and data security, the impact of good and big data on models. The tech-aware consumer’s footprint continues to grow, and many tech-aware consumers will be early adopters of AI-centric products and tools. Such end users are likely to evaluate and gauge the quality of the models they consume from the product. They may also demand transparency into how their data is used, and would want to be abreast of how regulations and standards are being followed and observed by the AI products they use. Many users are also likely to ask how the results they receive as part of a product ought to be interpreted, and why certain kinds of results are produced at certain times by their product.

This changing set of requirements has made necessary the need to build more trustworthy AI. This is a broad objective which perhaps encompasses a few different sets of capabilities:

  1. Good datasets used to build the machine learning or AI models in question
  2. Meaningful and regulation-adherent ways handling of user data in a manner that is transparent to the end-user and aligned with regulation
  3. Explainable results from AI models, which implies the ability to interpret the results (inferences) from machine learning models

Let’s break these down even further.

Data quality and Data Management

Outcomes in machine learning and data science are heavily dependent on the quality of the data used to perform statistical inference, or analysis leading up to model training and deployment. If we’re unable to understand the data in question, we’re unable to be effective in building machine learning and data science models effectively. This may sound simple, but is hard to put into practice. Data collection can become a messy process, especially in processes that don’t have rigorous measurement processes defined. Measurement system analysis, a common enough set of approaches in traditional industries such as manufacturing, or in engineering processes, tend to not be well-established for industries such as technology where data from people, processes and business systems is hard to instrument and hard to measure. KPIs may in some cases be tangible – in industries such as logistics, e-commerce and other domains – but even in these cases, there may be substantial ambiguity in how the data are stratified, how they’re counted and measured, and what the implications of variability are for measurement systems.

Another related consideration is how the data is handled and managed – especially where the data concerns the end-user. Some user data needs to be stored and managed to provide a good experience to users, but where does one draw the line? Additionally, how do the data that get stored, get managed over long periods of time? It is good to have a plan in place as a product team way before the product iterations begin, and certainly well before any release of the product.

Explainable results for end users

There is greater user awareness in 2022 than ever before about the importance of being able to explain the results of AI/ML models. To the end user, these considerations may come up as specific product features, because machine learning systems are being implemented in code as software product features. For example, a loan application app, job-candidate matching app, may invite a certain kinds of scrutiny from end users. The end user may ask, “why did X get matched to a certain job while I didn’t?”, or “why is the interest rate on my loan so high despite the prevalent rates for such loans in my area?”. Many users of social networking applications have asked what is being done with their data – we’ve seen broad and strong debate on this topic as well. Especially where machine learning models are used in a user and recommendations context, we invite such questions. Over the few years that I’ve followed this topic, my own understanding of the space of explainable AI has been challenged by Zach Lipton’s paper on model interpretability, which has invited this recent rejoinder on the communications of the ACM.

The practical experiences I have had indicate a preference on the part of end-users to have case based explanations for the interpretability of the results of their entire product. The reasons for this are two-fold, a) the specific context of the end-user and the data they submit is paramount in importance to the inference or result in question, and b) Even if the model’s explainability can be understood by treating the model alone a black box, there are many business rules and validations that are implied in the process of model deployment.

Implications for Data Scientists, Engineers, Architects and Product Managers

The modern data scientist, data engineer and ML engineer tend to be much more cross functional than before, requiring awareness and the ability to implement models of different kinds, and take some of the more complex models to production. In my observation, some of the common tasks have become a lot easier because of improved tooling and the availability of high-level frameworks for implementing some of these more advanced features.

Data Science and ML Model Development Implications

ML model training, development and iteration is commonly understood process in 2022 at the software and code level, but the questions around whether we have used good data or merely big data still remain. There should be more questions asked around what the business problem is, what the data and hypotheses are, and what models to build. It is easy to train models, but hard to undo the results of bad models once deployed especially in production. The availability of powerful frameworks and tools makes life better for those who can carefully think through, hypothesize and execute their model training process in more deliberate ways. This is the differentiating factor in 2022 between teams of data scientists that merely train models, and teams that are able to deliver truly differentiated value to end-users, based on soundly constructed hypotheses. Exploratory data analysis never gets old – it is always important to check assumptions, understand the underlying statistical properties of the data we use, and go back even further to where and how the data was collected, and how it needs to be processed. The availability of foundational models has created a gulf between teams building narrow AI at scale (like an AI/ML startup that may be building a product for processing invoices), teams building specialized narrow AI (like the teams building Alpha Fold or Google’s or Meta’s billion-parameter model teams), and teams researching broad AI (like the teams researching reinforcement learning like with Gato). The achievements of teams that are building narrow AI in highly specialized areas is quite staggering. Teams building more boring AI for routine, run-of-the-mill use cases in banks, telecom companies or manufacturing firms are likely to be doing equally impactful, if less glamorous work, and often have the same hierarchy of needs such as good data, assumptions checking and a rigorous process to train and validate models before deployment. The truth of the matter is that in 2022, not all AI model problem statements are equal – you can get outsize impact for the AI models you’re building if you’re building the right model for the right industry.

Data and ML Engineering Implications

Data and ML engineering in 2022 is a more sorted landscape than it was a few years ago. The changing user needs described above do have implications on how data and ML engineers execute their work. Data engineers are familiar now with tools to prepare, automate and implement data workflows such as ETL and ELT. All big cloud providers have such capabilities baked into their respective ecosystems as well. The focus has probably shifted today in data engineering to the careful management and versioning of datasets required for training machine learning models. Additionally, Data governance, data profiling and quality are the important talking points today, and rightly so, because the mere transfer of data from some source into your data lake doesn’t constitute business value anymore. In some cases, that may even mean the addition of cost and effort for business teams who have to manage, maintain and interpret this data.

On the ML engineering front, there are many opportunities and challenges for engineers who need to build applications which scale. Kubernetes and scalable deployments of endpoints have become common, but the complexity of managing distributed systems like Kubernetes, and doing so with a significant-sized DevOps effort involving infrastructure-as-code is a learning curve for most organizations. Managed services for handling Kubernetes deployments are helpful. There is an evolving discussion on the security of ML APIs that transcends common considerations around APIs such as HTTPS endpoints, or the tradeoff between synchronous and asynchronous endpoints.

While often not discussed, the QA function has become more important in data science than ever before, at least in my experience. It has always been important to write well-tested code, arguably, and the time for doing so and doing so well is now, as far as AI products are concerned. Not only is there a need for data science and ML engineering teams to have good strategies to test their code and ensure good coverage, but there is a strong need for user-focused testing and end-user testing.

Implications for Data and AI Architects

From an architecture perspective, there is certainly a trend towards event-driven systems in the machine learning product landscape. Using event-based architectures centered on technologies like Kafka or other message queues simplifies some of the orchestration around product workflows, and better enables on-demand provisioning and use of resources, in a cost-effective manner. I personally think that architectural decisions need to be made more deliberately and more frequently. The use of event-based architectures can simplify some of the workflows around user data, model inference and logging and observability, which are all parallel considerations alongside the changing user needs discussed above, such as data quality, model explainability, and so on. I recollect an insight from an earlier job when Terraform and infrastructure-as-code were gaining ground in the telecommunications AI space, that it is possible to rearchitect systems more frequently with IaC. This is perhaps an important insight for Data/AI product teams in 2022 on the architecture front – that it is possible and often necessary to iterate often even at the architecture level, as we build up from the basic product needs to more advanced and nuanced needs of end-users.

Implications for Product Managers

Product Managers in AI have a harder time in 2022, compared to earlier years, because of the dual pronged effect of greater regulation and evolving and changing user requirements from AI products. The landscape of the product features has shifted to more pragmatic performance needs, and feature prioritization seems to follow this trend. If there’s one piece of advice I have for product managers in 2022, it is to understand the capabilities and limitations of the underlying technologies behind data and AI systems. Most product managers are not technical and don’t understand the hierarchy of needs ranging from data, models and engineering to DevOps and MLOps as well. These are important for PMs to grok in 2022 and beyond if they’re to be effective at planning and delivering world class AI products.

There are implications for profitability from AI applications and products from these new and changing needs. As AI infused products are built more often and served up to end users, the bottom line is going to be influenced by the architecture and the optimizations of the product, while the top-line is going to be influenced by the scale of the product. Product managers will have to gauge user needs and build products to the appropriate scale for the phase of the product that’s being built – this is crucial, because the developmental costs of AI/data models and products is significant, and there’s a leverage that comes with these, which if used well can help build significant product impact.

Concluding Remarks

We’ve discussed both the evolving and new sets of requirements from end-users of AI products as of 2022, and the implications this has for different sets of stakeholders. From a product envisioning and development perspective, these changes surely imply a maturation of the needs of the market for AI and data infused products. From an inchoate market with aspirational ideas on making data, AI and machine learning work for end-users even just a few years ago, we see a greater tendency in AI product teams to want to build on-point and focused products for end-customers. More product teams are building end-to-end software applications which infuse AI/data capabilities in specific features or as a core feature, but this has come with implications for the end user, who is now more aware and more demanding than before. The resulting challenge for product managers is to characterize these changing needs of the product and incorporate such needs into their product plan. For engineers, this means building new kinds of value in the form of data and model provenance. For data scientists, it means prioritizing good data rather than merely big data.

The Value of “Small Data” in ML and AI

This is a comment from LinkedIn.

I wish we paid more attention to “small data”. Models that are built from small data aren’t necessarily bad – it depends on the data generating process you’re trying to model. More data doesn’t necessarily imply better models, especially if the veracity of the data is questionable. Data-centric AI is a discussion that’s being had now in this context. However, when you don’t need large scale ML models are are (prudently) content building statistical tests and simple models, these small data problems become important.

What decision makers shouldn’t forget is that the essential nature of decision making won’t change just due to the size of the data – ultimately it is the insight that models provide (based on many factors) that are the commodity we consume as decision makers. Consequently there should not be an aversion towards “small data” problems but a healthy curiosity. Like all efficiency movements that came before, small data paradigms are innately attractive – if I can verifiably build better models by doing less work, that should logically be a point of value.

MLOps Capabilities, Outcomes and Opportunities for Enterprise AI

Machine Learning Operations (MLOps) has come to be an important push for enterprises in 2021 and beyond – and there are clear reasons why this paradigm shift in Enterprise AI is upon us. Most enterprises who have begun data science and machine learning programs over the last several years have had difficulties putting even their promising machine learning models and proof of concept exercises into action, by deploying them meaningfully in production environments. I use the term “meaningfully” here, because the nuances around deployment make all the difference and form the soul of the subject matter around MLOps. In this post, I wish to discuss what ails enterprise AI today, sources of the gaps between production and proof-of-concept, expectations from MLOps implementations and the current state of the discourse on MLOps.

Note and Acknowledgement: I have also discussed several ideas and patterns I've seen from experiences I've had in the industry, not necessarily in one company or job, but going back all the way to projects and programs I've been in over the last seven to ten years. I don't mention clients or employers here as a matter of principle, but I would like to acknowledge mentors and clients for their time and energy and occasionally their guidance as well, in the synthesis of some of these ideas. It is a more boundaryless world than before, and great conversations are to be had regardless of one's location. I find a lot of the content and conversations regarding data science on Twitter and LinkedIn quite illuminating - and together with work and clients, the twain have constituted a great environment in which to discuss and develop ideas. 

What ails Enterprise AI today?

Surely, with the large scale data pipelines companies have access to, the low cost of cloud native solutions, and the high level frameworks for building machine learning models, things should have become easier? Enterprises still seem to be failing in their efforts to build AI programs for many reasons despite these upsides. For one thing, building models has become easier than before. It takes less time to take (good enough or clean enough) data and build prototype models with this data. Regardless of how many hypotheses you have as a business leader or data scientist, you’re more likely in 2021 to be able to collect data and build prototype models with this data, than you were able to in previous years. In the past, you may have had to go through several organizational hoops to get your data, and then prepare this data and then build models. All of these processes have become a bit simpler in 2021, thanks to enterprise data stores maturing, frameworks for building ML models become better known, and greater numbers of data scientists being available to build models. While things are still quite complex for the uninitiated, those on the growth curve in data science have found this phase to be adding productivity to their prototyping efforts.

What hasn’t changed, though, is the process of taking these models to production. The model is largely seen as a software asset, and productionization of the model has been seen in this limited context. As we will discuss, it is important to challenge this mindset if we’re to build effective machine learning systems for production. The gap, therefore, between proof-of-concept models like we’ve discussed above, and production scale implementations of such models, is large. Real world implementations are more complex and tedious, and often, the hypotheses we want to build models for are a bit more well defined – this necessitates extensive data processing, profiling and monitoring. But the complexity doesn’t end there, even though there has been an effort on the part of MLOps practitioners to build end-to-end pipelines. You’ll note that none of these are ground-breaking realizations. MLOps is a practical field, thus far, intended to make all these models work for enterprises – but as we will see below, the practical nature of this field encompasses a number of domain, statistical, cultural, architectural and other considerations.

I wish to suggest before diving deeper into this post, that this trend towards MLOps adoption represents a noteworthy change in how enterprises see ML system architecture in 2021, as opposed to the previous decade. In a manner of thinking, represents a move towards the “plateau of productivity” in enterprise machine learning.

Considerations for Enterprise AI – from MLOps, Data Science and Data Engineering

Domain Considerations Matter in Data Science, Data Engineering and MLOps

I wrote several years ago on this blog that domain knowledge is an important element in doing data science. Back then, as a data science neophyte learning from early experience in pure data science roles, I had made several observations about the impact of domain understanding on how quickly we can arrive at hypotheses for formulating data/AI problems. Looking back, this was an important lesson, because I now acknowledge the importance of domain knowledge every time I work on a data science project, or each time that I enable a data science team to be successful. Whether this is my own domain knowledge or that of SMEs, I am grateful for it, because without it, we could build anything, and it wouldn’t ultimately matter to anyone. Domain knowledge gives purpose to data and AI efforts. Without speaking to the domain experts and SMEs in various projects (finance, manufacturing, retail, energy and other industries), there would be little to no chance of timely and cost-effective success in characterising, ideating about and solving these problems.

It may not be immediately evident, however, that domain considerations matter in MLOps (and DataOps). Without an understanding of data generating processes, data formats, sources, rates, types, and data organization patterns, data fields, tables and even some of the process characteristics, we cannot understand data generation or transformation processes in enterprise data pipelines. We can also not understand how models are to be implemented, and what deployment means in different enterprise or customer contexts. When building and architecting machine learning systems, we end up needing to discover these details if we haven’t already. MLOps therefore cannot be ideated about in a vacuum, without consideration to the domain of the problem, or without consideration to the unique challenges of deploying models that domain. MLOps in logistics and supply chain problems, therefore, will be quite different from MLOps in manufacturing, retail or banking domains.

For instance, if we were building a classification model to sort defective parts from good ones on a manufacturing shop floor, we may need a real-time deployment system, with consideration to latency, edge based deployments of models, opportunities to inspect models as downstream processes or metrics may indicate process failure modes, and so forth. These considerations may not exist if we were building a system for enhancing ad revenue in a platform software company. The considerations there around uplift from pushing ads to new customers may require edge based deployments of a different kind, or federated learning needs, that may be unnecessary in the manufacturing example we discussed. To use an analogy, deployments are like different flavours of ice-cream, each requiring a different kind of appreciation. A failure to realize this may lead to difficulties in enterprises that may inadvertently underestimate the complexity of MLOps, of their own domain processes, or both.

Simplistic, Linear Pipelines Don’t Get Us Over the Line

The current thinking around MLOps is somewhat simplistic and linear, and I mean this in a specific way. There is a lot of discussion around data workflows and pipelines, metadata generation and management, and the metrics around model training and model performance. These are discussions around the management, transformation and profiling of data. Datasets are important to MLOps pipelines, and inasmuch as agility in data science is concerned, I’d even say that they are primary.

However, this notion of thinking only about the software and application-level implications of models and their deployment doesn’t address some of the needs from MLOps pipelines for enterprises. Notably, model interpretability and explainability, managing a diversity of deployment patterns (edge, batch, real-time or near-real-time), and the need to build repeatable pipelines or reproduce results. These problems cannot be broken down into just software applications, and require statistical rigour and attention to changing domain patterns. In fact, there is sometimes a desire on the part of ML engineering or MLOps practitioners to see these more statistical needs of MLOps as “not software engineering” and perhaps therefore “not easy to build for” – both of which may not be true, especially as the space of tools and implementations of statistical models for interpretability/explainability expands just as ML implementations have expanded.

Imagine that you have built an MLOps pipeline to build a dataset for a specific use case, and deployed it and the model eventually, and all’s well. If there’s a need for a new use case, you’re likely to begin back at square one, and build new pipelines, especially if you don’t have a clear and unified data model. As we will discuss in a later section on architecture, this is important to consider in ML engineering – more than one use case may require your data pipeline. This also means that simplistic and linear pipelines can only serve a limited purpose when you’re required to build many such pipelines across enterprise workloads.

For instance, it is possible to build SHAP scores for models given a specific dataset, and for companies with regulatory needs, there may be a reason to deeply analyze and publish results such as these. Therefore, MLOps shouldn’t only be about building simplistic DAGs or workflows in your YAML engineering tool of choice, or building and deploying metadata-tracked machine learning training/inference workflows. These are necessary, but insufficient for good MLOps implementations – chiefly because there are many other statistical and probabilistic considerations around MLOps which also deserve attention.

Data Architecture Before MLOps, but Business Needs First

There was an interesting discussion here recently around the theme of “Data before models, but problem formulation first”. The interesting article in question describes the specific challenges of thinking about data science problems based on business problems, and being “data-driven” in thinking about and building models for our hypotheses. I posit that a similar paradigm applies to MLOps. Data architecture understandably matters a great deal for success MLOps implementations, because it encompasses very foundational organizational processes and needs around data collection, storage and management, governance, security and quality, access patterns, ETL/ELT, sandboxes for analytics, connections to BI and reporting systems, and so on. Ultimately, this complex web of processes and technologies (because data architecture is more than just storing and retrieving data) is meant to perform some function of the business. As W. Edwards Deming said, “Data are not collected for museum purposes” – they are collected for a decision to be made, or for some end use. In the world of MLOps, we enable such decisions to take place on top of the data provided to us through an enterprise data architecture such as this one described above.

While typical enterprise data architectures are driven by the capabilities of tools and cloud scale applications more and more (because of the economies of scale of cloud providers, and the low barriers to entry), there is an important set of decisions every enterprise data architect has to answer for, around the specific needs of the organization, and how the architecture in question enables that to happen. Seemingly trivial decisions taken at the design phase of a data lake or data warehouse can have long lasting implications for the delivery of value from analytics, machine learning and MLOps. Data architecture is certainly important for MLOps, but the more fundamental needs of the organization – the kind of data required, the strategic importance of it, the decisions that need to be made across use cases, security and access patterns for data analysis and data science, and many more operational aspects of data – all of these are important and have a bearing on MLOps effectiveness too. So if you’re a data scientist or MLOps practitioner looking to improve your impact and effectiveness in solving problems, understand the underlying data architecture more deeply first. Sometimes, doing this can be hard – especially if there are no stakeholders who can explain it well – but this kind of fundamental understanding and context are highly underrated and have an outsize impact on the success of data science and ML programs eventually.

The Enterprise Model Sanctuary: Many Simpler Models, A Few Complex Models, and Other Combinations

A cursory glance at machine learning and MLOps forums, discussions and content indicate that the thinking around model development techniques is method centric, and not business centric. A large number of the discussions are a consequence of what’s required for companies at scale innovating on a few complex models with huge amounts of data – and these are legitimate and interesting discussions for sure. For example, most MLOps discussions I have come across seem to discuss the deployment of deep learning models. They discuss text and unstructured data processing, and complex image processing pipelines. Whether the use of tools like Kubeflow for training and deploying models in a distributed fashion, or the use of MLFlow for tracking metrics and performance, these are all legitimate considerations that may solve subsets of the ML deployment space. However, machine learning state-of-the-art is rarely required for enterprises looking to get value out of their specific use cases. The large majority of use cases in the industry are for simpler models, though and this is why simpler pipelines could do a large part of the value creation. I say this from experience and with confidence, having seen numerous projects where managers struggle to make sense of ML outcomes for their business, but have less difficulty making sense of data aggregations, summaries and statistics based on the data. The enterprise model ecosystem is more likely to resemble a zoo or even more accurately a sanctuary of different models, where each model may have its own specific needs and requirements.

Model development in mature organizations generally is an afterthought to carefully evaluating data and the evidential findings from it on merit, and then exploring hypotheses subsequently. Enterprises at lower levels of maturity have difficulty getting value from such an approach, however, and many leaders there may still rely on dashboards and reports. Clearly, there is an important and untapped market in business intelligence from big data. There is also a huge market for implementing simpler models based on clearly defined hypotheses. In many cases, enterprises may need many such simpler models, one for each stratified part of a specific use case. For instance, if you’re a market research firm estimating sales in a market segment, you may wish to build many such models for each sub-segment. If you are an equipment manufacturer doing quality checks using machine learning models, you may wish to use attribute based classification models, one for each product line, and perhaps you want to build many of them. The true value of MLOps in these cases is not in managing the complexity of deployment for one complex model, but in enabling many simpler models to be taken to production quickly and efficiently. These simpler models may then provide a baseline with which to build more complex models as needed.

Machine Learning Systems are Stochastic, Not Deterministic

Perhaps I’m stating the obvious, but it needs to be said. The underlying nature of data generating processes and machine learning models is stochastic and not deterministic. Whether we’re talking about manufacturing process metrics, banking and finance transaction data, energy sector data around load, power, usage, and so forth – all of these data are generated from stochastic data generating processes, even if they come from engineered systems. Machine learning models are also never exact mathematical formulations – they are almost always stochastic processes. There is a little to unpack here, so I’ll get into a few instances. What this stochasticity means, is that machine learning models exhibit variability in results from situation to situation, and that this will be quite evident in production. In order to begin building machine learning systems, we need to perform exploratory data analysis prior to training time, prepare features for our hypothesis, check assumptions based on the feature and the model formulation, and then build models and evaluate them. What it also means, is that we need to build safeguards to ensure that these assumptions are valid when doing production scale inference. It means that we may have to reformulate problems, as the underlying conditions of the data generating process changes. In case of deep learning models, sophisticated tensor transformations and training loops are required as part of the normal training loop of deep learning models.

When the model is eventually trained to the required level of performance and rendered, they too represent a solution at a specific point in time. MLOps is not about “train once, deploy everywhere”, but about “routine retraining and redeployment”. This makes ModelOps and the continuous training lifecycle of model development as important a consideration in MLOps as DataOps is. A lot of discussion around MLOps today is centered around data preparation – and the motivation for this, of course, is the fact that there are significant data preparation challenges that data scientists face. However, model training in the real world cannot be wished away by despite the prevalence of AutoML, although AutoML tools are one path for progress. As of 2021, for most use cases, model definition and training is still done manually, even if tuning and optimizing the model are automated. In MLOps lingo, we are referring to the importance of using feature stores, and their impact on data drift and concept drift analysis. While a healthy discussion is in progress on these topics, the instrumentation in actual implementations of data drift and concept drift identification and measurement tends to vary. Some tool chains are ready for this change, and others just aren’t.

More broadly, some MLOps implementations may account for these stochastic and probabilistic characteristics of ML systems, because their data scientists ask the hard questions after training and during/before deployment. On the other hand, it is likely that most MLOps implementations today treat models merely as pieces of software. The latter pattern leads to the unfolding of technical debt of various kinds later in the lifecycle of the system. This technical debt currently represents building additional regulatory checks, doing interpretability analysis, meta-data logging, model performance metrics, and so on – and over time, this set of secondary considerations may grow much bigger.

Changing Skillsets and Roles for MLOps

Companies looking to hire top ML talent as of 2021 are pushing for a greater number of high quality data engineers with MLOps skill sets. This is in contrast to emphasis on data science hiring in the past. Hiring pipelines for data and AI roles (I’ve seen a few different ones over the last few years) tend to emphasize programming, statistics, databases and specific technologies for data science – of late, this is largely SQL, Python, with a smattering of distributed frameworks and tools, and skill sets in deep learning, tabular data analysis and the associated frameworks and tools for solving problems in this space. For data engineering roles, over the years I’ve seen skill requirements specifying systems programming and strongly typed languages such as Java and Scala, experience working on JVM languages, in addition to SQL, databases, and a lot of the back-end software engineering skill sets we see for application developers elsewhere. For data engineers working on big data technologies, there’s very often a need to be familiar with NoSQL databases, or graph databases, depending on the role and use cases, in addition to the Hadoop-and-friends ecosystem, and cloud engineering skills such as AWS or Azure. While the data scientist’s role and skill set has come to include domain considerations, advanced statistical and ML models, cloud-native and large scale data science and deep learning and communication/presentation of data and insights, the data engineer’s role has become broader around systems engineering and design.

Someone said (in fact, in this talk) that data engineers ought to build frameworks, and not pipelines – and this is a fair assessment of how to use this broad and useful skill set in data engineering. There has been a healthy discussion in various forums, talks and the like on ML engineering roles which combine elements of these two different skill sets. All of these conversations around skill sets are important context for where we’re heading in data science and engineering space overall too. MLOps, unlike DevOps before it, should not be constrained by the limited value addition possible outside of data scientist or data engineering roles (the bulk of DevOps roles are administrative in nature). They cannot be construed as or see themselves as configuration file engineers, for lack of a better term. In fact, their role could be much broader – as systems engineers spanning a range of capabilities in both data science and data engineering, while not possessing expertise in any one of these (themselves diverse) areas. MLOps roles should perhaps also emphasize domain knowledge or expertise of some kind – since ultimately, the outcomes here are practical and related to business value from ML. There are many outcomes and opportunities for talent and skill sets for sure, but these stand out as being relevant. What is for sure is that the data scientist’s role has changed (as has the data engineer’s), and the old and unyielding challenges being faced by data scientists are taking on new definitions and manifestations – thereby requiring new mindsets, new skill sets, and new processes to come forth.

In my view this churn in the extant data science and engineering role paradigm is a welcome development because enterprises first want to realize value from DataOps and MLOps simultaneously today. As we will discuss later in this post, while models are important, business managers will continue to derive value from analytics and reports – and perhaps there has never been a better time to build on that need than 2021. Also, the emphasis on data engineering roles as on date is well-founded. From practical experience as a data scientist who worked on a range of problems from relatively simple ML to complex deep learning models, I will happily acknowledge that data engineers I have worked with were indispensable to the success of the projects I succeeded on. However, leaders hiring for ML roles should not think that the role of the data scientist is no longer required. I believe this emphasis on data engineering is a passing trend as enterprises build foundational pieces that enable value from data. The focus will therefore shift once again to business value from data, and that this automatically means that statistical, data science and ML skills will continue to be in vogue through this shift and afterwards.

Don’t Ignore Decision-Making Culture

Organizational culture matters a lot for the success of MLOps, as much as it does for any digital transformation program. MLOps represents, in a way, a desirable end-state or the happy marriage of data science and data engineering in a given enterprise and data architecture context. However, both data science and engineering can only be valuable and effective in organizations whose leaders think about and talk about data and use the data and insights from these data for taking decisions. The latter is a cultural synthesis, and not just a technology adoption process or workflow that one can execute on demand. Being a cultural matter, it has to do with behavioural and attitudinal patterns that ultimately enable data and insights to be used for decision making.

The adoption of data driven decision making represents a shift from thinking about business processes, systems and decisions in terms of rules (“Rules are for lazy managers”, to paraphrase Simon Sinek), to an open-minded thought process around data and AI systems. When leaders stop thinking in terms of rules, and start thinking in terms of systems, they are often imagining situations of change, synthesis, formation and deformation of patterns, structures and interactions. They begin to see their role as an influencer more and as a commander less, and this shift in thinking can enable them to make subtle changes to their managerial approach, driven by data.

In the earlier post I wrote about OODA, and the AI-enabled generalist, there is a point I make about the decision making language of organizations. This kind of development of a decision making language requires a way of thinking about the enterprise’s systems, processes, and also the ML models in new ways. It requires an openness of mind in decision making to adopt models as thinking tools. In a sense, the modern AI-empowered generalist could be seen as a prototype for a supreme pragmatist. Enterprises want rational actors at their helm, at least for the functions that require data driven decision making – and such rational actors can be groomed in a culture that doesn’t shy away from challenging the current rules and norms on decision making, and is willing to look at data and models.

Data/AI Exponents as SMEs and Future Leaders

Organizations come to embrace data, ML or even MLOps so that they can ultimately derive value from data, and this cannot be done without talent that unlocks value from data. Be this talent data science talent or data engineering/architecture talent, there is both a topical / functional need and a strategic value of these roles in enterprises, and this tends to be overlooked in data strategy. This is because of the value such individuals accumulate over time, as they build data pipelines and AI/ML models, accumulating a lot of knowledge about business processes, customers and also domain knowledge in the process. When you have a data scientist in your team who has built a few different models that explain different elements of your business, processes or customer behaviour, they become invaluable assets for both developing further models, and for analyzing customer or business or process behaviour. Such individuals can also become effective leaders and transition to process management roles.

MLOps and DataOps engineers in an organization can therefore themselves be considered Data/AI SME roles – and this is an important source of value that is often overlooked in organizations. A lot of organizations still see data/AI resources as just means to an end, but in fact, many of these roles can become storehouses of domain knowledge. MLOps can potentially enable the tacit knowledge from such individuals to be effectively captured for process management as well – this may be an important opportunity for value creation from MLOps. MLOps can also accelerate the development of data-driven leadership talent. When exposed to the models used to take decisions, and the specific mechanisms of taking such decisions, leadership potential for process leadership is improved.

In an earlier post, I discussed the importance of higher-level decision making languages, the OODA decision making loop, and how AI can enable a new generation of generalists. I would suggest that this is a useful idea to consider in the broader context of building a data-driven decision making culture.

“Data Before Models” also implies “Models After Data”

The purpose of this heading is to draw attention to the fact that the best data pipelines won’t help, if we aren’t doing much with the data we prepare. We have to eventually build models with this data of one or other kind for actually taking decisions. Many recent discussions around MLOps talk about data-centric AI, and above, we have discussed data architecture and other elements of enterprise systems and culture that contribute to MLOps success. We have also discussed the stochastic nature of data generating processes and machine learning systems. There are important implications from the core ModelOps processes as well, and we will discuss them here, finally. The process of developing models, as I have discussed above, has become easier now than ever before, at least in software. The careful formulation and evaluation of model hypotheses, statistical analysis of the input data and features, and the checking of assumptions – these still remain harder, more tedious and less trivial, as they were before. This necessitates the importance of statistical analysis and exploratory data analysis. Without these foundational steps, ML models can be built with high bias or high variance, thereby setting up the use case for higher failure rates and lower effectiveness overall. This bears introspection and repetition, since there seem to be two schools of machine learning and data science professionals – there seems to be a group of professionals who believe strongly that mathematical and statistical thinking are important for doing data science. There’s another group of professionals and practitioners who think otherwise, that the software elements of data science modeling can be learnt by someone without knowledge of statistics or machine learning.

In my experience, the statistical analysis and EDA are fundamentally important for machine learning – they forms an integral and important part of extracting value from the data we have, and making sense of it, before we solve problems. A number of business situations require us to think in terms of data distributions and stochastic processes. To build things that scale within MLOps pipelines, some of us may need to have an open mind about exploring the mathematical underpinnings of things like gradient descent or batch normalization, or activation functions. This open-mindedness is important for a key reason – a lot of MLOps engineers being trained today may assume that the data science is easy, or trivial, because people who don’t know statistics are building models, or because they can, if they just follow a simple workflow. I know this to be patently untrue – if you want to develop a model worth anything in an enterprise, you may have to start from formulating and thinking about the business problem, get to the EDA and statistical analysis and built out tests for assumptions checking, and then experiment with different models. You have to get into the probability and statistical analysis eventually, or you will be forced to rediscover the effectiveness of these mathematical and scientific methods. Even if you manage to build one or a few models, there will be situations where you’re required to explain these models. Not only will ML engineers or data science engineers be more confident when they are able to reason about the mathematics of machine learning, but their ability to build and scale systems for the enterprise improves. Their ability to think about the implications of these models for different related use cases, for different deployment modes, different source data, and different data quality considerations also improves. By checking assumptions on the features, they could stave off big challenges that may arise when the model is implemented in production.

Statistical analysis and machine learning model development have been core and will be core to data science, regardless of the peripheral engineering required for realization of value. Data engineering and MLOps as allied fields help realize this value at enterprise scale. It is the process of data science and model development that ultimately converts data into insights – and insights are the primary purpose of investing in enterprise data and AI projects and programs in the first place. They will therefore continue to be a good bet for practitioners in future – as long as they realize that those skills alone cannot take them over the finish line.

Concluding Remarks

I hope that you’ve benefited by reading this rather lengthy post on MLOps and Enterprise AI. If anything, it allowed me to explore my own experiences, document a few patterns I see in the development of truly enterprise ready AI, MLOps toolchains and capabilities, and also explore sources of value from MLOps for enterprises. If you have questions or ideas, please leave a comment or tweet to me at @aiexplorations.

Further Reading/Listening

  1. Data Science is Different Now, by Vicki Boykis:
  2. Problem Formulation Comes First by Brian Kent on Crosstab.io
  3. Build Frameworks, Not Pipelines – a Data Engineering Talk on PyData
  4. From Model-Centric to Data-Centric AI – a discussion on Enterprise scale AI with Andrew Ng and others
  5. ML Engineering for Production – another discussion on ML for production with Andrew Ng and others

The New Generalist: OODA, Machine Learning and Decision Making Languages

Machine learning operationalization has come to be seen as the industrial holy grail of practical machine learning. And that isn’t a bad thing – for data to eventually be useful to organizations, we need a way to bring models to business users and decision makers. I’ve come to understand the value of machine learning for decision making from the specific context of tactical decision making as fitting into the common OODA loop for taking decisions. At first sight, these seem like specific ideas meant for a business and enterprise context, but further exploration below will reveal why this could be an important pattern to observe in ML-enabled decision making.

Lazy Leaders of a Specific Kind

I recently listened to a Simon Sinek talk in which he made make a rather bold statement (among the many others he’s made) – “Rules are for lazy leaders”. I see this phrase from the specific lens of using machine learning solutions as part of a decision making task – building which is almost existential to the machine learning practitioner’s workflow today. What I mean here, is that leaders who think in terms of systems rarely rely on one rule, but usually think in terms of how a system evolves and changes, and what is required for them to execute their OODA (“Observe-Orient-Decide-Act”) loop. I want to suggest that machine learning can create a generation of lazy leaders, and I mean “lazy” here in a very specific way.

Observe-Orient-Decide-Act

A less-well-known sister of the popular Deming PDCA (Plan-Do-Check-Act) cycle, OODA was first developed for the battlefield, where there is (a) a fog of war – characterized by high information asymmetry, (b) a constantly changing and evolving decision making environment, (c) strong penalties for bad decisions, (d) risk minimization as a default expectation.

These four criteria and other criteria make battlefield decision making extremely tactical. Strategy is for generals and politicians who have the luxury of having a window into the battlefield (whence they collect data) and have the experience to make sense of all this in the context of the broader war beyond the one front or battle. Organizations are much the same for decision makers.

OODA
OODA – Observe-Orient-Decide-Act – Courtesy Opex Society

As of 2021, decision making that uses machine learning systems has largely impacted the tactical roles, where there are decisions made on a routine basis with a days-long, hours-long or minutes-long decision window. Andrew Ng famously summarized this in his “AI is the new electricity” talk – where he discusses the (unreasonable) effectiveness of machine learning for tasks that require complex decision making in a short time frame.

Sometimes, strategic and tactical decisions are made on the same times scales, but this is rarely the case outside of smaller organizations. Any organization at scale naturally sees this schism in decision making scales develop. OODA is ideal for decision makers that are domain experts, in this limited tactical decision making setting. Such decision makers are rarely domain experts, but are adept functional experts who understand the approach to and the implications of success in the environment they’re in.

OODA Loop is an actual street named for the Observe-Orient-Decide-Act context

So where does machine learning operationalization fit into all this? Today’s decision making environment is data-centric and information-centric in all enterprises. What this means for the typical decision maker or manager, is that they are often faced with information overload and high cognitive load. When you’re making decisions that have dollar-value impact on the enterprise, cognitive load can be a bane beyond a point. Being cognitively loaded is absolutely necessary for any activity up until a certain point, when the tedium of decision making under uncertainty can affect decision makers emotionally, causing the fight-or-flight response, or can trigger other set behaviours or conditioned responses.

This brings us to the information-rich nature of today’s decision making environment. When we’re dealing as decision makers with lower level metrics of a system’s performance and are expected to build an intervention or take a decision based on these lower level metrics, we are rarely able to reason well in terms of more than a few such variables. The best decision makers still end up thinking in terms of simple patterns most of the time, and rarely broach complex patterns of metrics in their decision making processes.

Machine learning enables us to model systems by transforming the underlying low-level metrics of their performance into metrics of higher level abstractions. In essence, this is simplification of complex behaviour that requires high cognitive load to process. Crucially, we are changing the language with which we take decisions, thanks to the development of machine learning models. For instance, when we are looking at a stock price ticker and are able to reason about it in terms of confidence interval estimates, we’re doing something a little more sophisticated than thinking in simplistic terms such as “Will the stock go up tomorrow?”, and are probably dealing with a bounded forecast spanning several time periods. When we’re analyzing video feeds manually to look for specific individuals in a perimeter security setting, we ask the question “Have I seen this person elsewhere” – but when doing this with machine learning, we ask the question of whether similar or the same people are being identified at different time stamps in that video feed. We’re consequently able to reason about the decision we ought to make in terms of higher level metrics of the underlying system, and in terms of more sophisticated patterns. This has significant implications in terms of reducing cognitive load, allowing us to do more complex work with less time, and crucially, with less specialized skill or intelligence for executing that task, even if we possess a complex understanding of the decisions we take.

The Limits of our Decision Making Language

I want to argue here in a rather Wittgensteinian vein, that humans are great at picking up the fundamentals of complex systems rather intuitively if the language they use to represent ideas about these systems can be conveyed in a simplistic manner. Take a ubiquitous but well-known complex system – the water cycle. Most kids can explain it reasonably accurately (even if they don’t possess intricate deep knowledge of the underlying processes), because they intuitively understand phase changes in the state of water and the roles of components in the cycle such as trees, clouds and water bodies. They don’t need to understand, say, the anomalous expansion of water, or the concept of latent heat, in order to understand the overall process of evaporation, the formation of clouds and the production of rain and water bodies as a consequence of these.

OODA and other decision making cycles can be amplified by operationalized machine learning systems. ML systems are capable of modeling complex system behaviour in terms of higher level abstractions of systems. This has significant implications for reducing cognitive load in decision making systems. Machine learning operationalization done through MLOps can therefore have significant implications for decision making effectiveness on a tactical basis for data-driven organizations.

Implications and Concluding Remarks

Machine learning and the development of sophisticated reasoning about systems could lead to the resurgence of the generalist and the AI-enabled decision-making savant with broad capabilities and deep impact. For centuries, greater specialization has been the way capitalism and free markets have evolved and been able to add value. This has had both advantages and disadvantages – while it allowed developing societies and countries to rise out of poverty by the development of specialized skill, it also meant decelerating returns for innovative products, and more seriously, led to exploitative business practices that sustained free markets. Machine learning systems if applied well can have far reaching positive implications and free up large numbers of specialists from tedium, enable them to tackle broader and more sophisticated problems, while simultaneously improving their productivity. As a consequence, ML and smart decision enablers such as this may be able to bring even more people from developing nations in Asia, Africa and elsewhere into the industrial and information age.

Assigning Blame and Credit to AI Systems

In the 1950s, Isaac Asimov imagined a world where robots lived and worked in human society, and in one of his short science fiction stories, he discusses the travails and tribulations of a robot “Cal“, who serves a human writer, and “wishes” to be a writer himself, after Cal discovers that he cannot be considered helpful for performing his master’s work – writing. This heartwarming and profound story is a benign example of the future previous generations imagined (perhaps erroneously although charmingly enough).

An Asymptote to the Asimovian Atelier

In recent years, developments in AI and Machine Learning techniques have encompassed generative modeling and algorithms used to generate art, to send control signals to self-driving vehicles to correct themselves on the highway, or to generate music and text for artists and writers alike. There have been forays into AI-based authoring, AI-based composition of music, and AI-based writing of novels. While these algorithms are incipient in their scope and capability, some generative algorithms such as GPT-3 show huge promise. Some years ago in 2016, I was part of a session conducted by Google’s Tensorflow team where we performed simple style transfer tasks using deep learning on convolutional neural networks. The advanced algorithms we have access to in 2020 make these tasks quite trivial and the current state of research makes the possibility of even more advanced algorithms acute. A good technical summary of how to build some of these algorithms – including the DCGANs and CycleGANs that have become so popular is the book embedded below. There is lots of literature for the technically minded and Github has scores of repositories on the subject of generative AI, not to mention other kinds of AI models.

Despite this creeping progress that AI researchers today are making towards the world imagined by Isaac Asimov several decades ago, certain key elements of AI and its impact on our society are likely to be up for discussion and debate well into future decades.

The problem of accountability and blame (or credit) assignment in AI systems is one of these important ongoing discussions, and it promises to upend some of our existing notions about how people and human and natural systems function. We are guaranteed to peer more into the underlying chain of causation in AI-based systems and the cause effect chains that arise due to their interactions with human society at large – in all spheres from science and medicine to engineering and technology, to art and creativity.

Yes, but is it art?

Edmond de Belamy.png
Edmond De Belamy, a piece of art created by a Generative Adversarial Network

This discussion promises to shine new light on agency, free will, decision making and other aspects of societal functioning. Starting with relatively innocuous questions such as the one posed in this fantastic paper – “Who gets credit for AI generated art?”, which is a discussion on Edmond De Belamy. Additionally, it serves as a broader swath of exploration veritably covering our mental models of what reasonable causes and effects are, what phenomena can be considered indivisible or black boxes, and what the meaning and extent of human agency is, – all these will be explored. Let’s start with the obvious – how our media (and by extension most of society) see artificial intelligence – this is the example of the art discussed in the aforementioned paper.

Human interpretations (from the media) of the credit assignment problem posed by the generative AI painter

Operational Domains, System Definitions and Moral Crumple Zones

To understand some elements of how AI based systems can be handled in society, we need (but may not necessarily get) the following:

  1. Acceptable boundaries and a definition of what constitutes an AI agent’s field of operation (domain)
  2. A range of effects and first order characteristics of such effects that are a direct consequence of the agent’s action
  3. An understanding of how second, third and higher order effects may be ascribed to the agent, or to broader systems at play.

In addition to the above, a clear operational definition or description of the AI system in question is important in any discussions that ascribe blame or credit to the system in question. We will see how this is important in a further section of the post.

At the outset, it may seem as though a self-driving car that has killed a person may deserve blame, or its manufacturer may be liable to legal action. It may seem that the person using the DGCAN algorithm in the aforementioned link on AI generated art would be liable to collect the proceeds from the sale of art produced by the algorithm. It may also seem as though the external environment plays no tangible role in the outcomes of the AI systems, since they are “AI systems” after all.

However, things get much more complex when we start looking at the details of such real-world interactions. The acceptable boundaries of an AI agent’s field of operation are not always well defined. For instance, would a self-driving car be to blame, if there was an unexpected event (such as a human-caused accident, or an oil tanker spill) on the road ahead, and because of a limitation of the system, the system flies out of control in this environment, and ends up killing its occupants? Would the creator of the DCGAN algorithm have to be penalized in case the art doesn’t fetch some anticipated price at an auction? One of these questions may sound more serious and the other more innocuous than the other, but in fact, they both tie back to the same boundaries and domain of operation we discussed earlier, and what we accept as the first order effects or characteristics of an AI agent, and what we don’t.

One could group the ambiguities around disproportionate responses by human systems to the results of AI systems, as “moral crumple zones”. Indeed, in the context of the human interaction with the natural world (because the two domains of beasts and men intersect there), we could term these “ecological crumple zones” – where human interest in extracting value of a specific kind from natural systems makes us amenable to cause widespread and disproportionate imbalances in the world – as can be seen in over-fishing or over-whaling, the hunting of animals to extinction and the destruction of the rainforests. In the context of the AI-human interaction question, moral crumple zones (as defined by Epstein et al) constitutes disproportionate responses directed at one facet of an AI system due to agony caused to a human or a group of humans due to an AI’s action.

Our task then, is to reduce the span of these “moral crumple zones” – and a key to this is the operational definition of the AI system mentioned above. This isn’t necessarily a direct or even useful answer to the dilemma posed by the art sale or the self-driving car’s blame assignment problem, but does take us in that general direction.

A Submarine by Any Other Name: Narrow and Broad AI

It is helpful sometimes to think of agents in natural systems in terms of synthetic equivalents, although this may be a form of shoehorning their relevance, value and significance. For instance, one could call a fish, “a natural submersible agent” – although this may capture an aspect of the fish, i.e., its locomotion, it doesn’t necessarily do justice to the role of the fish in the context of the broader ecosystem, as an organism that’s part of the grand natural dance and as a complex organism in its own right, given to a physiological homeostasis of its own. In the same way, we could refer to birds and aircraft as analogues in the natural and synthetic spheres, and although you could argue that both are “evolved” (one by nature, another by the human hand), it is clear that the same limitations of the analogy as discussed earlier apply to this situation too.

Clearly, anthromorphic AI is a long way off. Whatever stories the media spin around us about the coming strong AI, or the fact that “An AI beat Lee Sedol at the Game of Go“, it remains that these are engineered systems, and are not to be confused with agents with human level intelligence. Surely, much of this AI hyperbole exists despite qualifications to the contrary, because it is a sign of our times to ascribe undue credit or undue intelligence to things that are not especially intelligent of their own accord. A DCGAN algorithm, for instance, consists of a generator neural network, and a discriminator neural network which are trained in lock-step in alternative training cycles, seeking to create a generator which just barely manages to fox the discriminator in a generation task. This doesn’t constitute broad intelligence.

Broad AI, in the words of Gary Marcus, is a phenomenon that is the converse of the narrow AI capabilities we have begun to see in the world today – essentially, narrow AI like the DCGAN alluded to above, or the deep learning network that guides a self-driving car, are simplistic, purpose-built decision making systems, which have come to be conflated with the capability of a human over time. Lee Sedol lost to a reinforcement learning model – I’ll admit this is a model of significant complexity, but it is a model nevertheless, and not a full fledged intelligent agent in any sense other than in the narrow domain. While Lee Sedol retains the ability to gaze at the stars, wonder, appreciate art and a good cup of tea, and still pontificate on the ramifications of move #37 made by Alpha Go, Alpha Go itself remains a narrow construct, excellent at taking decisions in a specific domain and a specific context, with no clue of how to solve broader problems in human society.

The broader point here, then, is the fact that anthropomorphicity in terminology used to describe AI can be a hurdle to truly understanding what the AI system is doing in the first place. Just as a submarine and a fish aren’t one and the same thing. Even though they use the principles of hydrodynamics for accomplishing locomotion tasks, one of these is a machine with a limited specification set and clearly defined system and operational parameters, while the other is a complex, evolving lifeform with broad influence and ramifications for the ecosystem it inhabits. The moral crumple zone for an evolving lifeform is different than that for an anthropomorphic robot. The moral crumple zone for the fish is likely to be smaller, and that for the AI agent is likely to be larger, especially if used as a tool by sophisticated creatures such as human beings. The latter is likely (given the nature of progress we’re making in AI systems) to be of far more influence and may wield much more power than the former, which is playing a part in a whole harmoniously and will either be predator or prey in a vast ecological chain of cause and effect. The cyclical nature of the ecological homeostasis of the lifeform lends to itself a termination point, whereas artificial intelligence agents may not function in the same way, and may have much more longevity and influence, especially given that they are tools and ultimately are human means to a human end.

AGI and Alexander the God: A Shift in Agency

It isn’t unfair to suggest, in light of this discussion on narrow and broad AI, that an artificial general intelligence (AGI) may be subject to far more scrutiny than are the narrow intelligence of today. I posit that there is a shift in the agency of a system that extends broadly enough, to the point that while initially, an individual or team may have been culpable for the undesirable side effects of the system (such as blaming the lack of seat belts in a car to a specific individual upon failure of the car to secure a passenger), to a systemic problem that expands to include the whole system itself (a great example of this being the fraudulent, stupid and self-serving financial system that gave us the sub-prime crisis in the late 2000s).

The agency of a narrow AI system can more or less be easily understood and matters of legal and scientific dispute may thereby be more clearly elucidated, than in the case of a broad AI or a general intelligence. In the latter case, we essentially have parts of a whole to blame for different effects on the boundaries of the competence of the system

Another of Asimov’s sci-fi short stories discusses a young protagonist who sets out to become rich by predicting the stock market, and successfully pulls this off by getting his computer to learn how the stock market from the news, the associated changes in the stock market and is able to tailor his trades to world events and phenomena, leaving him a very rich man. This story, like many others in science fiction is saddled with the twin paradox of technical debt and innovation in a vacuum, not to mention the efficient market hypothesis – had Alexander been able to do even a fraction of what is claimed in the story, he would have run up against a wall by doing so himself, compared to if he were “standing on the shoulders of giants who came before him”. In this sense, Alexander’s project in the Asimov story may be likened to a vast, multi-generational capitalist project in a never-ending free market economy (which in turn sounds less and less realistic given the vagaries of the world economy over time, the more I think about it). Such a project would, in all likelihood have expanded its scope from merely a profit engine, to becoming one capable of influencing the world, because after a point, the shortest path to the goal of maximizing rewards would involve direct intervention. Somewhere along this trajectory, Alexander’s AI would stop being narrow AI, and start becoming a more general intelligence. So, if a rainforest disappeared last week due to Alexander’s AI’s need to deforest it for preferentially ticking the stocks up, would you blame that on the profit maximization rule built into the AI, or would you blame it on the greed of the humans who benefit from Alexander’s AI? At this scale, we all realize (as the legislators will) that the AI is a means to an end.

Coming back to the problem of ascribing credit and blame – the story in question is quintessential in how the media and the non-technical world at large see AI – as a looming, foreboding influence in the present that will enable large scale income inequality and that would somehow summon demons out of the fabric of reality. In fact, I’m not ashamed to use the hyperbole here since Stephen Hawking himself ended up using a similar metaphor. The reality, of course, is that we really have a number of narrow AI systems that are interacting with the broader human-created systems such as in trade or industry, which can have an outsize effect on how we interact with the world. How does one characterize agency in this situation? Narrow AI systems may have a specification, but can a broader system with multiple specialization areas (and by extension multiple domains, multiple interaction points with society, greater complexity and lesser interpretability) be considered an agent in its own right for legal and social / cultural / scientific purposes? The moral crumple zones in these broad AIs will also tend to have bigger moral crumple zones – so does that give society the responsibility to treat these AIs as they would treat, say, an air hostess who has to apologize on behalf of the airline to the customer, or a customer service agent who has to take the blame for a product? If yes, does the idea of humaneness become relevant in this context, and what does that entail?

Ascribing credit and blame in case of the widespread use of narrow AI in complex human systems is probably as complex if not more complex as the broader and hitherto unseen problem of legislating an artificial general intelligence.

Concluding Remarks

Credit and blame assignment in AI systems – like speculation and wonder – are innate to human responses to and characterization of phenomena, systems and forces that are beyond our full grasp, but which we perceive to be simpler to understand than they often are. And this isn’t necessarily a bad thing, but an indication of how we think. Short of upending everything we understand and know about agency and pulling us into an infinite regress of causality (“The whataboutery could indeed go all the way back to the Big Bang,” said he humorously), we can devise practical ways and means of characterizing the domains and scope of AI systems and plan for progress. This includes healthy discussions about the scope and limits of algorithms, the kinds of situations in which they may be used (and their due regulation, such as in warfare – including cyberwarfare), and keeping the human stakeholders of these systems informed in advance of the risks of using both narrow AI systems today and broader AI systems in the future. Without planning for progress in AI systems in this manner, we’re likely to inherit a future in which the admixture of human societal systems and AI capabilities will leave us befuddled as a society that cannot maximize outcomes for individuals because of the fact that it has to carry on in a manner that is convenient to the clueless bulk of society, or at the very least, to architects of the mess that it has become.

References

Further reading

Three Elements of the AI Skills Gap and Implications for AI Jobs

It was common in the Data/AI circles to discuss or lament about the skills gap in the market for candidates, around 2015-16. This hasn’t changed in five years, inasmuch as recruiters and hiring managers still have these lamentations when asked about the hiring scene. The challenge is real for organizations, but I also believe that there are multiple reasons for this to be a subject of discussion half a decade down the line.

Perceptions and Reality

Hiring managers in large firms that want to jump start their data and AI projects after having missed the early adopter window in the early 2010s, may come to the realization that the talent pool has since been flooded with candidates – not with the quality promised by the pace of research – but with quality indicative of education and certification opportunities out there. Education in AI has its share of problems – from an emphasis on tools rather than principles, to a lack of focus on business value and operationalization, it doesn’t foster the kind of mindset often required for scalable, enterprise grade data science and AI solution development.

Education in AI also prepares incumbents for roles in research/academia and also industry, and this two-pronged objective rarely yields benefits for recruiters. A greater focus on applied AI/ML has certain disadvantages as well, since it can lead to a lack of depth. There is no silver bullet, but perhaps one approach here is to build multiple levels of skill and provide opportunities to back-fill talent into roles gradually, and build progression as a consequence.

It is also true that AI/ML lateral hires tend to be expensive to hire, especially if they have real world production scale AI/ML experience. This makes the challenge even more acute for those looking to hire, who are on a budget. Upskilling existing employees is obviously the best long term solution. For startups, attracting talent who have an upside in terms of being able to grow into advanced AI/ML roles for niche products, may be an attractive selling strategy – and even if the said startup won’t help them build deep skill in AI, it could give them a shot at success in being part of an AI product journey.

Writing effective and meaningful job descriptions and controlling the hiring and evaluation process well is the key here – without meaningful job descriptions and with ludicrous titles, it is easy to get on a slippery slope.

Overall, whether hiring from university graduate pools, or in the lateral hiring market, recruiters and hiring managers have to manage perceptions and reality for AI hiring.

Inability to Understand the Fast-Changing AI Landscape

The AI landscape changes almost on a daily basis, with ground-breaking research published almost each week. In 2020 alone, we have seen significant new research, the highlight of which has been GPT-3. In earlier years, we reached very high levels of performance in language translation tasks, and significant breakthroughs in generative modeling such as GANs. We are now seeing significant new work in reinforcement learning that promises to make agent based models easier to build and train. Despite being an industry professional and leader in the AI and ML space who’s constantly learning and upskilling, new research can even be hard for professionals like me to keep up with. This is the gap between the research world in AI and practitioners such as myself.

There is another even more worrying gap in understanding – and that is the gap between the practitioners and the managers or recruiters. Managers and recruiters who don’t understand the fundamentals of AI and machine learning, and who aren’t clued into the fast-changing AI landscape will be at a disadvantage when trying to hire good talent.

With lateral hires, the challenge is more nuanced. When bringing onboard data science candidates with prior experience, one needs to look out for a vast range of skills and capabilities that need to match with the organization’s current state and needs. There’s the additional challenge of ensuring that the lateral hire candidates are contemporary and able to solve or at the very least, reason about complex and new problems.

Impostor Syndrome and Gatekeeping

Image courtesy Hackernoon

The flip side of Steve Jobs’ quote, of course is that someone would choose not to hire a smart person at all – because they may be too smart, or too capable.

Impostor syndrome (and its close cousin, the Dunning-Kruger effect) is widely seen in the AI practitioner world – where those new to the space who nevertheless have credentials in building AI solutions or products find themselves falling short of expectations because of the sheer complexity of real-world requirements on the one hand, and because of an inability to retain or sustain the knowledge they have earned credentials for. This sends a signal about their competence which is contrary to their actual skill in a subject. Naturally, this leads to problems in executing on the job for those suffering from impostor syndrome. Ultimately, I believe this stems from an attitude of credentialism (to borrow a phrase from entrepreneur and investor Balaji Srinivasan) – that earning a credential will get you to a destination – rather than a genuine interest in AI and a curiosity and open mind that enables the development of new skill and capability.

Gate-keeping in the AI/ML world is a consequence of Impostor Syndrome, and it is essentially the prevention of good talent from joining an organization in the hiring process, due to the insecurity felt by less-skilled incumbent employees who fear replacement by new, higher skilled employees. This is not new to organizations or unique to the AI/ML hiring space. In areas like data science and AI/ML hiring, it is plausible that hiring managers with limited understanding of a subject may fall prey to gate-keeping. After all, power-poisoning as is seen in a lot of management positions is a strong enabler of mono-cultures and also gate-keeping. Power poisoning is a natural impulse for anyone in management who wants to keep their jobs, especially in a time of rapid change. One way to ensure this doesn’t become a slippery slope, is to build a culture of recognition and reward those who invest time in learning new skills to stay relevant on the job. Another way for leaders to make all feel welcome is to ensure that those on the job are constantly challenged, because this is one way in which additional bandwidth can be justified, and consequently one can justify new roles, positions and new hires to fill these positions.

Concluding Remarks

I hope this discussion on the AI Skills Gap and the implications for AI jobs and hiring provided interesting and important insights from the frontlines in AI and ML. If you’ve faced challenges like this, whether from bosses or hiring managers who experience impostor syndrome, or from recruiters who don’t understand the landscape for AI/ML hiring, you’re likely to resonate with a lot of the ideas here. As with all things in leadership and management, there’s rarely ever one good answer. However, constantly scanning the AI landscape and being clued into the patterns and trends can help build great AI teams and sustainable Data and AI organizations, be it i product development, research, or consulting.

Emphasizing the Basics: Structured Data Science Mentoring

Data science, machine learning and AI are constantly growing and burgeoning fields, with research that’s spilling over at the seams in terms of the sheer volume of it all. Every day, I receive numerous references to interesting papers on my Twitter feed, thanks to Arxiv daily and such accounts there. I also see papers explained with code, and references to ML products and systems in numerous contexts. This is all overwhelming beyond a point for a professional who doesn’t have a specific focus area. Speaking pragmatically, and from the tree of knowledge (which is always bound to be vast), it is a feature of every single human endeavour to exhibit this kind of complexity as we spend more and more time exploring things, farming ideas and understanding new possibilities in these areas.

Data scientists are going to be at different levels of competence and may be differently placed to take on challenges they are asked to face – the role of the mentor (regardless of the type) is to systematically challenge the data scientist to discover new innate potential and develop such potential to increase their overall capabilities and effectiveness.

The Data Science Mentoring Challenge

Mentoring can be a hard task for this reason – a lot of people (understandably) gravitate towards complex models that are meant for specific purposes, without fully understanding the details and the exact mechanisms behind simpler machine learning and statistical modeling methods. The problem with this is two fold – a) reliance on libraries and frameworks with implementations that already exist, and b) inability to characterize, apply and explain common and simpler techniques to actual real world problem statements. Part of the problem here is the sensationalization of research in the media. Open research without borders is important and pivotal for speedy progress in technology areas like ML. But we’re also seeing a lot of misinformation including sensationalization of advanced ML techniques and when some of gets parroted by professionals (some of who may become hiring managers) we see the problem proliferating into the world of work as well. I’ve interviewed my fair share of individuals who understand, say, an LSTM unit’s different gates but aren’t comfortable explaining autocorrelation techniques or ARMA models. This gap probably stems from gaps in mentoring and coaching, which ideally should emphasize basics first.

I’d posit that the role of the mentor has changed, in data science, over the last several years, and I would say it has changed most significantly in the last two years. In the future, data and AI mentoring will look different from what it looked like in the past five years. This is because the nature of the job of a data scientist (or alternatively an AI/ML engineer) has also changed. Despite developments in Automated Machine Learning, we’re inundated with situations in the real world, where we require human expertise to get through data science and machine learning problems. This human expertise manifests in three processes: problem characterisation, problem formulation, and problem solving. We need real, human data scientists (not just an AutoML tool) to look beyond the obvious automations such as hyperparameter or architectural searches, to reason about the nuts and bolts of problems, interpret the problem domain and reason about different kinds of hypotheses and how they make sense.

This makes the process of mentoring for data science different than it was, in certain specific ways. For one thing, mentors today create the field of problems or opportunities that will exist tomorrow. Data scientists today experience an overload of information as can be expected, from different sources. From Arxiv and Springer papers and articles, to new research and code, new books and new frameworks and algorithms, there are plenty of things to learn on a daily basis. However, the broader skill set of the data scientist even today can be characterized into four key areas: basic, business, functional and frontier skills.

Broad Characterization of Skills for Modern Data Science
  1. Technical skills: There’s the need for a strong foundation that enables general effectiveness in a data science role. This includes good skills across statistical analysis fundamentals, leading into the key principles that enable statistical learning models to be built, and a sound understanding of the mechanism behind common algorithms such as regression, tree algorithms, search and optimization methods
  2. Business skills: There is a strong need for data scientists who can reason about business processes and systems, and understand how data may be generated, how it may flow, and what insights may be required of it. Not only is this is a key skill to have fruitful interactions with clients and stakeholders, but it is also important to narrow down to the right level of depth for the job in terms of satisfaction and effectiveness.
  3. Functional skills: There’s the need for effectiveness on the job, at a functional level, which not only includes technical competence at the statistical, mathematical and code levels, but at the level of processes and good practices such as clean code, change management and reproducible research. One could also see more advanced machine learning and feature engineering techniques as being part of the functional skill set.
  4. Frontier skills: There’s research that’s expanding at multiple frontiers, which is hard for even experienced data scientists to keep up with, if they’re really interested in furthering their career beyond the obvious and evident challenges of day-to-day work.

Mentors: Different Levels

The role of mentorship has also become specialized in the last two years, which is, in my view, one of the changes most representative of the maturation of the field of data science. Mentors today can be at different levels of skill and still add value to different kinds of data science and analytics roles. For the sake of this discussion, I’d classify mentors today into two kinds – the “breadth mentor” and the “depth mentor”. While both kinds of mentors possess certain common skills, especially on the interpersonal communication front, they may have different approaches to technical, functional and research level mentoring.

The breadth mentor is an individual with plenty of experience in data science, perhaps in a consulting setting, that can provide generally correct advice to data scientists with the development of broad skill sets, ranging from basic statistical analysis, to advanced algorithms. The nature of the mentoring here is on developing a well-rounded data scientist, rather than an expert in a specific field.

The depth mentor by contrast, is someone who has deep experience in a specific area of industry or technology and has deep experience in bringing this field together with data science. Examples of this kind of data scientist would be an NLP researcher, or a researcher in the field of robotics, both of who may be expert practitioners of data science methods in their specific areas, but without the broader knowledge of consultative data science methods.

Depending on the needs of the business and the data scientists in question, the appropriate kind of mentor has to be chosen – and this shouldn’t be done lightly. For example, bringing a breadth mentor to an AI product firm may have some advantages, but if the firm is solving problems in a specific space, this may not work out so well. Similarly, bringing a depth mentor to a consulting firm can help grow a specific practice (or a new one) but may not benefit the broader data science efforts across different business domains there.

Structuring Mentorship in Data Science

Mentors (and hiring managers) in general should emphasize the importance of the basic skills listed above. In my view, when a data science candidate has the correct understanding of the essential basic statistical ideas and common algorithms, it becomes a lot easier for them to grasp more advanced ideas such as in deep learning, when this is required. Mentors can build better basic skills in data scientists by challenging their technical acumen.

Mentors should also emphasize business skills where relevant, and where the emphasis is on research, they should emphasize some of the frontier skills as well. Mentors in this context are expected to challenge the data scientist with relevant questions, and encourage a habit of systematically breaking down problems and asking the right questions. These business skills are important all the way up to solution architect roles and management, when crucial decisions have to be taken and hard questions will need to be asked often. Mentors can build better business skills in data scientists by challenging their problem understanding and characterisation.

Functional skills are important for effectiveness on the job. It is not okay for data scientists to theoretically understand a specific subject area, only to find themselves handicapped when asked to build a machine learning pipeline. Therefore functional skill mentoring is about challenging the data scientist on problem solving effectiveness.

Finally, frontier skills development depends on both the organizational or research context, and the data scientist’s interests. Mentors can provide helpful markers to enable the exploration of ideas, while emphasizing value from the research, and asking questions that keeps the data science researcher on track. The challenge the mentor can pose here is differentiated solution value and originality.

The Importance of Emphasizing the Basics

This brings me to the importance of emphasizing the basics. I see numerous individuals out there who are getting into data science and machine learning that are interested in getting right to the latest and greatest algorithms. For a while – and this has been a trend on LinkedIn and Twitter – budding data science aspirants post some of their work, where it involves the development of simple scripts or programs around computer vision, translation and such problem statements, thereby delivering an impression to a lot of their audience, that not only are they skilled at those techniques demonstrated, but that they are skilled at different kinds of data science problem statements as well. My own suggestion to data science aspirants is that they will be under pressure to demonstrate some of their more involved skills, not merely the ability to use pre-built libraries to solve problems using one’s own basic skill sets in statistical learning, but, perhaps be able to build such algorithms and systems from scratch. This kind of deeper skill is what differentiates the wheat from the chaff in data science.

Concluding Remarks

In conclusion, I believe the mentor’s role in data science has changed – mentors today have their tasks cut out, when it comes to building deeper skill in their data scientists – they should emphasise technical acumen first and foremost, problem understanding and characterisation next, and problem solving effectiveness after this. This builds up a well-layered skill-set where technical skills can perform a harmonious dance in amalgam, resulting in true value to the data science market.

Getting Data Science Work Done Remotely

Given the current Covid-19 crisis that has led to massive disruptions to how we work, communicate and collaborate, there is an understandable interest in being able to do data science work remotely and effectively. In a sense, this capability has been brewing in the background, because the data science talent crunch experienced for several years before data science skill sets went mainstream, was an opportunity for companies to hire talent around the world, and work remotely.

Despite this, there have been challenges in building excellent remotely managed teams in all technology sectors, including data science and AI- and ML-centric teams. For one thing, remote work is fraught with asynchronous work, meetings and the need for over-communication. Another aspect of remote work is the need for interpersonal interactions and relationship-building between peers and team members. These are undeniable aspects of what makes remote work in itself challenging – sometimes, phone calls are not enough, and video calls done professionally and effectively require discipline and commitment on behalf of the participants. All asynchronous collaboration requires goal-orientation and timeliness in the execution of tasks. These are, of course expectations that you may have from ideal employees and team members. Based on the working style and approach of different individuals, you may have very different reactions from your team members to the same ground rules.

Making Remote Data Science Teams Effective

I can think of some ways in which we can enable better interactions and more productive data science teams across locations, time zones and in complex data science projects:

  1. Knowing your team members well. This piece of advice is not about data science, it is about just being a good team player or leader. There is no substitute for actually building great interpersonal relationships. Humans are not robots – professionals are people and are driven by meaning, reason and have motivations of many kinds, both personal and professional. All of us have challenges when listening and taking feedback that requires us to change, whether we’re leading or contributing. Some of us may have quirks and oddities that make us interesting in some ways and annoying in other ways. There are some good ways to build such relationships in remote ways.
    1. Don’t skip the small talk. Ask about how they are doing when you start a video or audio call. A little small talk never goes to waste, especially at the beginning of a meeting. In these times where people may have vastly different levels of well being, whether because of being affected by Covid-19, or otherwise being impacted because of it, it never hurts to ask.
    2. Empathize and make your team feel wanted and welcome. Ensure you empathize with them in case they come out and say that they’re having a tough time. It doesn’t help to be the “strong and silent” kind of individual when the person at the other end is communicating difficulties that they’re having. Video and audio calls require us to overcommunicate.
    3. Understand your team members’ habits and quirks. Share a joke or two, and understand how your team responds. Determine what makes them tick, and what kind of work assignments interest them.
  2. Setting ground rules and “team level agreements”. One of the biggest enablers of productivity in data science teams is knowing what tasks are meant for who. In real world data science teams, things may not be clear-cut, when it comes to the broad span of tasks that data science team members need to do. Ground rules ease the situation and collaborative tools enable this to happen well. Tools like Teams or Slack are great at building contextual conversations. However, you can lose sight of the bigger picture here, because you’re following along multiple threads under a specific topic. What helps here is setting up Wikis, and doing effective stand up calls.
    1. Wikis and how they can help: Team wikis help by consolidating the key information in one place. They’re great for teams who have been working a certain way in an office setting and have had to transition to remote teams, sometimes with new team members. They can provide a nice, section-wise summary of key tasks and elements of the work stream or the job in question. In the context of data science teams, Wikis can help in the following ways: a) by being project documentation, b) by instructing on specific tasks – be this environment set up for a Python task, or PEP 8 guidelines, class hierarchy for an application, a list of hypotheses to explore in statistical analysis, etc., or c) by being a repository of tribal knowledge about the solution you’re building. Wikis can be created by using documentation tools like Read The Docs, or even by Wiki servers. Where Wikis are too much work to do, simple documents (Google / Word / Confluence) can help. On Confluence, you can attract comments as well, and this can make things easier in some ways.
    2. Stand up calls and doing these effectively. Stand up meetings (typically these are 15 or 20 minutes long) are a great way to start the day and gain some momentum with respect to the features and solutions you’re building. They shouldn’t be long drawn out, but should just focus on the key accomplishments and blockers. Add in a little sugar – use video and do a nice, positive team ritual. Ensure that you do round-robin updates, because this makes sure everyone is involved. When doing data science, you may get updates on how a certain analysis went, or whether new findings were made with respect to some data, or whether a model training step failed or succeeded in some way. All of these are useful points of exploration and problem solving for the day ahead. These initial discussions in the day can be the source of a new synergy – if you are a leader, you generally respond to some of these, or perhaps you bring ideas from the previous day to share and guide the team in a specific way. If you’re a contributor, you’re likely to make key notes of things that happened during your work day and share them in the next day’s stand up call.
  3. Few meetings, but effective meetings. Having spent time in big corporate and in startups, I see a tendency on the part of managers and employees who come from large organizations to gravitate towards meetings to solve problems. The reality is that meetings are rarely effective in and of themselves, and better consensus can be built asynchronously in written communication that can be read, absorbed, digested and then responded to. Something to keep in mind in remote teams:
    1. Data scientists and engineers need to write well. They should be able to articulate thoughts, ideas and complex constructs well, and should be able to respond with the appropriate amount of detail. Without this ability to think critically and express themselves in written form, the organization becomes meeting-driven, and can descend into chaos when these meetings sap team energy
    2. Meetings should have clear outcomes. These outcomes should be decided at least five minutes before the end of the meeting. The agendas for meetings need to be clear before the start of the meeting, which isn’t often enough the case. Finally, the results of the meeting should be circulated over Teams/Slack/Email with clear expectations on the part of those involved.
  4. Remote-sourcing domain expertise. The availability of team members from across the world is a potential benefit from having fully remote teams that companies haven’t fully realized the benefits of yet. If you’re doing data science in the energy sector, for example, you may be interested in building a team with occasional input from an energy sector domain expert you may not have had access to before, because more people have opened up to consulting remotely. Such domain expertise is especially important in building effective teams across borders that understand the domain well enough to be effective as a data science delivery team in a specific industry. Technology, statistical analysis and communication skills together cannot solve a problem that also requires domain knowledge to solve, and this can be done effectively by sourcing such talent or expertise remotely.
  5. Working effectively without borders requires planning and effective documentation. Working across time zones can be hard at times – when we work in synchrony, communication can be easier. However, asynchronous communication and work will become increasingly important in the age of Covid-19 and beyond, as more and more teams become remote and distributed. Working remotely doesn’t equate to being flexible time-wise, but being effective with your tasks. For example, you may be on a call at 9pm with your team that’s in a different time zone, and are retiring for the day soon after – this situation calls for planning your upcoming day and tasks well enough for you to be effective at solving those problems in front of you. This may be a challenge for those with small children and families, but there are probably ways to work around it all. The solution is not stretching long into the night to complete that task. This probably impedes personal health, productivity and other aspects of well-being, both personal and professional. Effective asynchronous work requires good planning and documentation.
  6. Leading across borders requires planning, patience and understanding. When you have team members in other time zones and are delegating tasks to them, you probably have to curb the enthusiasm to do things yourself, or to delegate to someone closer to home – this risks isolating those who work in remote time zones. If you find yourself doing this, consider that you may need to rethink how you are structuring your project and tasks.
  7. Pair programming and digital mentoring can be really effective. When your team members are trying to develop new skills and don’t have anyone to turn to for help, they can benefit greatly from pair programming sessions and digital mentoring. These are not new practices, and there are platforms available to do this across teams – but what’s important is to have regular communication to sort issues out as they happen, and help people correct course as soon as they need to do so. Pair programming enables specific and contextual feedback. Whether statistical analysis or application development, data science mentoring done remotely can be a big enabler of growth and technical accomplishment.
  8. Managing work environments well. Work environments here doesn’t only refer to the physical environments, such as the work from home setup we each may use, but also the digital environment, which enables us to find the required information at the right time, or which enables us to construct new workflows as we need them. This extends to code and the virtual environments we code in. When we’re building continuous delivery pipelines that provide the required environment for running and testing code, we are enabling such a need. The tools are half the reason that highly effective teams are as effective as they are.
  9. “Gitting Good” – managing asynchronous data science work effectively. Managing an asynchronously updated code base requires your team to adopt and work well, with the right ground rules, on specific development branches of your code repository. A lot of product development firms have nailed the process of managing code on git or other version control systems well, and indeed this applies to many data science teams as well, but this is as much a matter of individual and collective discipline as it is about systems and processes and Wiki pages. Sometimes, we need hard conversations to happen to bring teams on track – and in the context of code discipline, I have seen a fair few situations of this nature.
  10. Taking time to document (and to “RTFD”). Documentation is one of the more tedious tasks for most developers, and it can be a chore for data scientists as well. Good data science teams, however, build their solutions on top of excellent documentation, where key questions are answered and many elements of their problem solving approaches are documented well, where required with links to papers, journals and results as appropriate. In cases where novel algorithms are written these should not only be put together in modular and reusable ways, but also documented well, so that the solution is intelligible to the broader team and to clients. As important as writing the documentation, is reading it. When new team members come in, or people change assignments on a project, it is important to keep the documentation relevant and ongoing.

Concluding Remarks

Naturally, all of these behaviors don’t happen all at once and aren’t developed in a day – excellence takes time, persistence and diligence. There are many teams out there that are doing lots of incredible work, and fully remotely, in the data science space. I hope that some of the above ideas make sense to your team and that if nothing else, this post made you think about how your team’s currently performing and how to amp up your team’s performance!

A View of DevOps from the World of Data Science

Operations management as a discipline has taken many shapes and forms in different industries over the years, but there is perhaps something unique that is discussed in software development operations, commonly referred to as DevOps. Many of these considerations around DevOps also apply to the related and increasingly interesting subset of problems that is MLOps, which is the field of Machine Learning Operations. So, what is unique about DevOps and the discussions of software development operations in this context?

Perceptions of DevOps Today And Contrasts with Traditional Industry Operations

One of the tweets I came across recently by a manager hiring for DevOps roles was this one, that sparked an outpouring of ideas from me. The entire thread is below, with the original tweet for context.

Popular understanding of DevOps seems to revolve around tools. Tools for managing code, workflows, and applications for helping with this or the other thing encountered in the context of software development workflows. Strangely enough, operations management in industries that are more established, such as the manufacturing industry, oil and gas or energy industry, or telecommunications tend to have the following sets of considerations:

  1. People considerations: From the hiring and onboarding of talent for the organization, to the development of these as productive employees, to employee exit. Operational challenges here may be the development of role definitions, establishing the right hierarchy or interactions for smooth operations, and ensuring that the right talent is attracted and retained in the organization.
  2. Process considerations:  All considerations spanning the actual process of value delivery, whereby the resources available to the organization are put to use to efficiently solve day-to-day problems and meeting customer requirements on an ongoing basis. Some elements of innovation and continual improvement would also fall into the ambit of the process management that’s part of Operations.
  3. Technology considerations: All considerations spanning the application of various kinds of technology ranging from the established and mundane, to the innovative and novel – all of these could be considered a part of technology management within Operations in traditional organizations.

Anyone familiar with typical, product-centric or services-oriented software development organizations will observe that the above three considerations are spread out among other supporting functions of these organizations. Perhaps technically centred organizations with very specific engineering and development functions evolve this way, and perhaps there is research to show for this hypothesis. However, the fact remains that what is considered development operations doesn’t normally involve the hiring and development of talent for product/solution engineering or development, or the considerations around the specific technologies used and managed by the software developers. These elements seem to be subsumed by human resources and architects  respectively.

Indeed, the diversification of roles in software development teams is so prolific that delivery managers (of the so-called Scrum teams) are rarely in charge of the development operations process. They’re usually owners for specific solution deliverables. The DevOps function has come to be seen as a combination of software development and tooling roles, with an emphasis on continuous delivery and code management. This isn’t necessarily a bad thing, and there is a need for such capablities  – arguably there is a need for specialists in these areas as well. But here’s the challenge many managers hiring mid-senior professionals for managing DevOps:

Cross Functional DevOps and Lean in Manufacturing Operations

When we have DevOps engineers and managers only interested in setting up pipelines for writing and managing code, rather than thinking holistically about how value is being delivered, and whether it is, we miss crucial opportunities for continuous improvement.

As someone who has worked in both manufacturing product development and software product development teams, I find that there needs to be a greater emphasis in software development organizations on cross-functional thinking, and cross-functional problem solving. While a lot of issues faced by developers and engineers in the context of product or solution development are solved by technical know-how and technical excellence, there are broader organizational considerations that fit into the people, process and technology focus areas, that are important to consider – and without such considerations, wise decisions cannot be taken. A lot of these decisions have to do with managing waste in processes – whether that is wasted effort, time or creativity, or technical debt we build up over time, or redundancy for that matter. The Lean toolbox, which originated from the manufacturing industry, provides us a ready reckoner for this, titled the “eight wastes in processes”: inventory, unused creativity, waiting, excess motion, transportation, overproduction, defects and overprocessing. Short of seeing all development activities through these “waste lenses”, we can use them as general guidelines for keenly observing the interactions between a developer, his tools, other developers, and code. Studying these interactions could yield numerous benefits, and perhaps such serious studies are common in some large enterprise DevOps contexts, but at least in the contexts I’ve seen, there’s rarely discussions of this nature with nuance and deep observation of processes.

In fact, manufacturing organizations see Lean in a fundamentally different way from how software development teams see it.

Manufacturing organizations heavily emphasize process mapping, process observations and process walks. And I shouldn’t paint all manufacturing organizations by the same brush, because indeed, the good and the bad ones in this respect are like chalk and cheese – they’re poles apart in how well they understand and deploy efficient operational processes through Lean thinking. Many may claim to be doing Six Sigma and structured innovation, and in many cases, such claims don’t hold water because they’re using tools to do their thinking.

Which brings me to one of the main problems with DevOps as it is done in the software development world today – the tools have become substitutes for thinking, for many, many teams. A lot of teams don’t evaluate the process of development critically – after all, software development may be a team sport, but in a weird way, software developers can be sensitive to replay and criticism of their development approaches. This is reminiscent of artisans in the days before mass production, and how they developed and practised an art in their day to day trade. It is less similar to what’s happening in large scale car or even bottle manufacturing plants around the world. Perhaps there are good reasons for this too, like the development of complexity and the need for specialization for building complex systems such as software applications, which are built but once, but shipped innumerable times. All this still doesn’t imply, however, that tools can become substitutes for thinking about processes and code – there are many conversations in that ambit that could be valuable, eye-opening elements of any analysis of software development practices.

MLOps: What it Ought to Include

Now I’ll address machine learning operations (MLOps) which is a modern cousin of DevOps, relevant in the context of machine learning models being developed and deployed (generally as some kind of software service). MLOps have come to evolve in much the same we saw DevOps evolving, but there is a set of issues here that go beyond the software-level technicalities, to the statistical and mathematical technicalities of building and deploying machine learning systems.

MLOps workflows and lifecycles appear similar to software development workflows as executed in DevOps contexts. However, there ought to be (and are) crucial differences in how these workflows are different between these two disciplines (of software engineering and machine learning engineering).

Some of the unique technicalities for MLOps include:

  1. Model’s absolute performance, measured by metrics such as RMSE or F1 score
  2. Model deployment performance against SLAs such as latency, load and scalability
  3. Model training and retraining performance, and scalability in that context
  4. Model explainability and interpretability
  5. Security elements – data and otherwise, of the model, which is a highly domain-dependent conversation

In addition to these purely technical elements of MLOps, there are elements of the discipline in my mind, that should include people and processes:

  1. Do we have engineers with the right skills to build and deploy these models?
  2. Have we got statisticians who can evaluate the underlying assumptions of these ML models and their formulation?
  3. Do we have communication processes in the team that ensure timely implementation of specific ML model features?
  4. How do we address model drift and retraining?
  5. If new training data comes from a different region, can it be subject to the same security, operational and other considerations?

There may be more, and some of you reading this, who happen to have deployed and faced production scale ML model development/deployment challenges, may have more to add. MLOps should therefore see significant discussions around these elements, and these and other related discussions should happen early and often, in the context of ML model deployment and maintenance.

The New Post-Covid19 World Order: Remote Work, Data, Cloud, AI, IoT, Governance, Autarky and Relevance

I’m almost certain that those of us reading this blog post have already experienced some of the disruption due to Covid-19 that’s been experienced at a huge scale across the world. The crisis that the world finds itself in as of this writing in April 2020, has brewed for almost six months. Going by recent research, the first cases of Covid19 were identified in Wuhan, China, around 17th November 2019. It has been a long and gruesome six months, marked by global disruption, tens of thousands of dead as on date, and more than a million and a half infected around the world. Most of us woke up each day of these few months to hear of more and more people affected and dead due to Covid19 in countries across the world. Some of us were not as fortunate – having experienced the disease or its effects first hand. In this post, I want to imagine what the world may look like – to use that oft-used expression these days – “when this is all over”.

Covid19: Health and Economic Impact

Not armed with a ready reckoner treatment of any kind because of the virus’ novelty, valuable time was lost in the initial weeks and the spread could not be curbed in Wuhan. The Chinese authorities in Wuhan as well as the WHO have both been blamed for the predicament the world finds itself in today – and perhaps rightly, although other governments of countries with significant numbers of cases are also to blame for their management of what was clearly known to be a highly infectious disease. There are experimental drugs and antivirals being tested, and a large number of people have recovered from such treatments – as of this writing, more than 400,000 people have recovered from this disease. However, the impact of this virus is likely to last a long time. It has been seen as a definitive point in history, marking the beginning of a new kind of social, economic and political world order, because the virus has far reaching consequences.

Annotation 2020-04-12 002730

Source: OurWorldInData.org

The US seems particularly badly hit as of this writing, with European countries such as Italy, Spain and the UK also badly affected. Some countries have had more luck than others in fighting Covid-19, South Korea being one example. In Asia, we’ve seen Iran affected quite badly by the disease, with tens of thousands of cases and thousands of deaths. China’s been reporting minuscule numbers since late February, and while we’re all led to believe that they’ve defeated the virus’ spread, in good faith, the numbers they’ve told the world earlier didn’t add up, with additional research on estimated actual death and case counts, from Covid-19 in China. In South Asia, we’re likely to see a rapid growth in cases, and I hope that in India we will manage to keep the infection and death rates down as far as possible. As of this writing India has more than 8000 confirmed cases, and has seen more than 200 deaths because of the novel coronavirus. In summary, it can be said that this was a bolt from the blue for the world, not only in terms of the impact on the health and medical systems around the world, but also economically.

Staring Down the Barrel of a Global Recession

In the first week of April, millions of people in America reported unemployment, a historic high of 6 million or so in that country. Chinese firms are going back to work in and around Wuhan after the lock down in that country was lifted. I am unsure what the future holds there – if some of the test accuracy rates from Chinese test kits are as low as claimed in some of the reports (30-40%), we are likely to see a relapse of the condition in many of those affected – and without practicing the social distancing and lock downs protocols that seem to be required to curtail the spread of this disease, we may see a resurgence in cases in China as we are seeing elsewhere. This can only be a bad thing for the world’s current economic condition. In fact, as of today, it has been declared that we’re in a global recession.

Politicians, policy makers and companies the world over have been pulled up for not acting fast enough, with even the WHO not being spared – their initial advice on not recommending masks is now widely seen as a problematic piece of advice which led to untold misery in countries like Italy and now in the US, because the advice contradicted the correct practice for curtailing the virus’ spread. Economically speaking, most economists and economic policy makers have indicated that the world economy is already in recession as of 11th April 2020, and that we’ve seen a significant erosion of value in all economies of world with the possible exception of China and India who may recover from this recession better than most. As a services oriented economy with only a satisfactory manufacturing base that’s underwhelming compared to the scale of manufacturing in China, it is hard to imagine India bouncing back strongly from this shock. China controls a lot of the supply chains of global manufacturers in a diverse variety of goods, and therefore have the potential to bounce back better than India on that count. India’s tech-savvy IT businesses and startups may buck the trend and do well, but sectors like agriculture and manufacturing will suffer because of the supply chains everywhere being hit. Even within Indian IT, though, demand is probably going to be hard to come by, and we may see very bad tidings for the Indian economy in general.

In India, where we’re seeing a large number of cases (nearly 7000 cases as of this writing, and almost 250 deaths), the lock down has resulted in a huge disruption, especially affecting the non-salaried class. India’s economy has a large unorganized sector, where artisan-ship, daily wage labor and other such occupations account for a large percentage of the workforce. These jobs are responsible for a vast amount of India’s economic leverage as well as provide a viable income for the masses that don’t possess advanced degrees and skill sets that go beyond the basic skills required for most jobs. With Covid-19 requiring social distancing, many who form part of the unskilled workforce may end up contributing to the supply chains that will run our socially distanced, remote workforce. Without this option, they’re likely to be significantly set back, economically. Already, we see how companies like BigBasket, Swiggy, Dunzo and other delivery-centric and supply-chain-centric firms (in India, and their equivalents elsewhere) are doing really well in this period of crisis and adding significant value to their customer base. We’ve also see how telecommunications firms have seen their value expand as a consequence of their increased demand at this time of crisis. And this is important in the long term for reasons that I will explain below.

Modes and Impact of Remote Work: Technology Sector and Other Sectors

With Covid-19 pushing organizations to follow lock down protocols and social distancing measures, plenty of Indian organizations (like elsewhere) have adopted remote work as a viable alternative to office-based employment. The important thing about this trend, however, isn’t the fact that the organizations have adopted the few changes necessary to enable remote work – it is the fact that the very nature of these organizations will change, thanks to Covid-19, even after this crisis has been relegated to the history books. Why is this the case? For one thing, operations managers and COOs will realize that remote work enables higher productivity and lower costs for knowledge work. They will understand the benefits of having employees manage their time at home, juggling responsibilities at home and work, while completing tasks and meetings required for achieving their goals. Remote working also obviates the need for large offices. The modern glass-paned concrete-jungle city is a consequence of an old school of thought – centralized, synchronized teams, communicating face-to-face and using this face time to build relationships.

Now, in the post-Covid-19 world, companies will have to amend their cultures and working styles to perform all of these functions – from sourcing to the delivery of value, to the collaboration required for sustainable organizations – completely online. This necessitates the use of telecommunications networks first and foremost, and on top of these networks, it necessitates the use of communications and collaboration technologies from audio and video conferencing and the like. As an example, if you’re a software developer, you may start your day with video calls, manage your features and tickets on a tool like JIRA, rely on documentation asynchronously developed by a global, distributed team, and manage your code on git with good engineering and code management practices. If you’re a manager, expect to jump on a number of video conferencing calls, and expect to build relationships remotely. If you’re an executive, you will have to cultivate the ability to write, inspire and influence your stakeholders and team across distances, with very little possibility of direct face-to-face interactions.

Manufacturing, energy sector and other organizations will rely on a combination of process automation for manually intensive tasks, implement a sanitized workplace for those who are required at site by default, and enable remote work options for knowledge workers in those industries so that they may collaborate and add value remotely. In such organizations that range in their scope from industrial equipment production to chemical and oil and gas supply, there is likely to be significant disruption of the standard work practices that were implemented and perfected over the years, because of the unique challenges faced by their employees and customers, in the post-Covid-19 world. Companies across industries will realize and come to value good measurement systems for processes at many layers of the enterprise (technical and process level metrics, and also functional and organizational level metrics), and strive to implement effective and reliable measurement and management systems, because their decisions will be asynchronous and decision making remote, and both processes will be based on such data. Metrics to manage and provide feedback on employee performance and reward or penalize such performance will have to take a different route from what was done in the time of face-to-face communication. Many organizations will have to adopt a process of management by metrics in addition to management by objectives, with old styles of direct and micromanagement going out of fashion.

Technologies such as augmented and virtual reality, which have become popular in recent years for things like product demos, entertainment, games, simulation and so forth, hold a great deal of promise for companies wanting to bring immersive collaborative experiences to their workforce. While a VR/AR meeting seems simplistic as a collaborative addition over the video conference experience, perhaps there are many opportunities for interaction possible in this ambit. Largely, the impersonal world of online conferences and meetings have seen an attention deficit problem, and low engagement. This seems to be true even for some video conference situations, where a lot of the interaction’s elements are voluntary. The subtle non-verbal cues that humans pick up and communicate with in face-to-face conversations play a big role in meetings and trust building, and this impacts credibility and consequently productivity. Naturally, face-to-face conversations set the bar for interpersonal communication far higher than virtual alernatives (text, audio, video and VR/AR), and there is a need for over-communication verbally and gestures-wise when you’re on calls of any nature. This has consequences for team management and dynamics, and can come to define the culture of the organization itself. Case in point: Basecamp and their CEO Jason Fried.

What we can perhaps hope to see by the fusion of data and ML/AI with conferencing, are listening and immersion aids and statistics. In some hypothetical future meeting, such immersion aids could improve the meeting experience. Given the direction that some of these innovations may take, there is an opportunity for new hardware to provide some of these immersive experiences for those at work in different settings – especially since for some of us, the immersion we experience at work is more like that of a musician playing an instrument and less like someone watching or enjoying a movie – there’s a visceral amount of immersion required for some tasks at work to be effective. Direct brain stimulation is the next step in communication beyond the audio-visual domain where we’ve been operating for all of humanity, and there’s work being done in this space. Some of these hardware are, if we are to go by recent advances in AR and VR, full of promise, given that they’re experimenting with experience-creating technologies such as direct brain stimulation (1, 2).

The Impact of Data and AI in the Post-Covid-19 Enterprise

Measurement systems and data will become increasingly more important for asynchronous, global and distributed enterprises. The cloud enables large scale data storage, in remote, managed services models. It also enables enterprises to convert capital expenditure considerations to operational expenditure, thereby providing them the flexibility to manage costs for teams, equipment and project funding separately from IT infrastructure. Serverless and cloud-based IT applications that are very contemporary at the moment will simplify nearly every aspect of the technology-enabled enterprise to sourcing, hiring, engineering, development and delivery, quality and customer experience management, and metrics will drive team performance, goals and agility in projects. For instance, there is no excuse for a modern enterprise (whether it is a startup or a truly large business) to not prefer the cloud for their website. Sure, they could maintain a backup server with their site, but it is a no-brainer to adopt cloud technologies for certain use cases – the risks and costs of starting from scratch don’t make for a good business case for most enterprises.

For cloud scale serverless architectures to be effective, they need economies of scale, among other constraints such as tooling and testing, on the adoption side of things. This is purely by design – cloud based serverless architectures are products rolled out by the big cloud firms, that depend on such scale to keep the costs low. Security and scalability issues currently persist but are far less frequent than those with on-premise infrastructure. One hopes that with the tailwind strengthened by Covid-19 related pressure, many companies seeking to go cloud-native instead of building their own IT infrastructure will use these capabilities going forward.

Companies outside of technology, across the spectrum from manufacturing, retail and telecommunications to oil and gas and energy will likely use the cloud a great deal in the future. Whether Covid-19 or not, many had already begun this journey. As a consultant working with clients on big data, data science and such initiatives, I’ve seen many taking new technologies on to ensure that they are able to stay ahead of the game (by gaining competitive advantage) and cost-effectively so. Manufacturing organizations can do more on the cloud than they thought possible today. Network speeds are fast enough for thin-client CAD applications that have high responsiveness, for example, and cloud-based servers could be used to run analyses such as finite element or CFD computations, that may be required in large scale manufacturing settings. The virtualization and digitization capabilities that the cloud brings in general, therefore, can cut team sizes significantly and manage aspects such as costs and consumption by moving to a pay-per-need model. Such an economic model can greatly benefit manufacturers in developing economies, if the benefits of scale elsewhere are made available to them.

Data, Cloud and Digital Transformation: Pre- and Post-Covid-19

Collecting and managing data from diverse source systems has been one of the many victories of data ingestion tools that have come into prominence and widespread use in the last decade. These ingestion tools, along with scalable data storage and processing systems that use distributed computing, have become the staple of big data and cloud-based AI/ML initiatives for numerous enterprises. I’ve written about these capabilities extensively on my blog earlier, as building blocks of enterprise scale data lakes and AI/ML systems. Such tools will come to see greater relevance and importance in the digitized enterprise transformed by Covid-19 risks.

With the need for large scale analytics and insights to drive efficient decision making in a remote work setting, where the individual is far removed from the business process, there is likely to be a greater demand on suppliers of the data for such insights, and for those who can deliver the insights themselves. This will in turn necessitate machine learning and data science – and overall, this paradigm is not unlike what we have seen earlier. The drivers for the earlier data and ML / AI revolution were competitive advantage, data driven decision making to achieve the upsides in transactions, and the need for low cost, agile, and lean operations. Now, however, Covid-19 related risks have resulted in a completely different set of motivations for digital transformation. For one thing, enterprises with high structural costs of business are now using Covid-19-induced drop in demand as a rationale for restructuring and reinvigorating lean and agility initiatives, by adopting remote work, contract employees, and distributed teams to save costs. In the short term, these trends will result in reduced operating expenses for existing facilities, and in the long term, this will transform into reduced capital expenses and reduced investment in new facilities. Additionally, data and AI adoption will grow for the reasons mentioned above – greater adoption of automation, cognitive/smart automation driven by machine learning, and productivity drivers will enable new kinds of value delivery in the enterprise.

New AI Capabilities and their Adoption

As a practitioner in and an observer of the AI and machine learning space, I find a number of new techniques in the spaces of natural language processing and generative modeling that have become research frontiers for those in machine learning today. Many of these techniques, from transformer models for natural language like BERT (link, explanation), to generative adversarial networks or GANs, have been experimented upon for a wide range of applications ranging from language translation, to face generation. With the rise of remote work and remote teams, there are many upsides to adopting such techniques. The contexts and problem statements around the use of machine learning in the post-Covid-19 world are still being revealed, and many enterprises are discovering such points of value, but the need, in this time of distributed teams for cross-cultural and cross-language communication, digital team building, real-time translation – all while preserving the personal touch – these things are important for effective remote work across distances, regions and time zones. These capabilities, along with virtual avatars for bots and virtual intelligent agents are just some use cases will see enterprise AI adoption (especially of the ML methods for richer data such as text, audio and video) at large scale.

There is another, underlying layer of technologies that will enable collaboration in the post-Covid-19 world – that of the telecommunications network. Large scale data transmission has become nearly ubiquitous now, with fiber optic technology becoming mainstream in the past decade. The coming years, due to Covid-19 and the risks that will follow it, will seed a reliance on the part of businesses on higher speed, near-real-time interactions, that enable complex automation tasks, such as completely remotely executed surgeries. While there is no substitute for a direct, in-person diagnosis and surgery for a lot of patients, for many surgeries, there is a gap between the expertise available and the need of patients around the world, and robotic surgery tools could be the frontline equipment in these battles. The enabler of such technologies is 5G communications technology, which is in turn comprised of a number of enabling capabilities, such as virtualized, software-defined networks. Physical hardware (copper, optic fibre and the like) and the network connectivity we get from this has driven us to the large scale, high speed direct-to-home fiber internet revolution today, but in future, virtualized networks of all kinds that rely on such physical networks such as optic fibre networks will play an important role for the transmission of voice, video and sensor data. The management of these networks and their performance as regards their scalability, security and capacity management could become entirely automated, using machine learning techniques. These are already problems being solved by the big telecommunications technology firms of the world, who are deploying scalable networks defined using software.

Virtualization and container-based environments for running AI and ML applications have become an important capability in 2019 and 2020, and we have seen large scale acceleration of machine learning deployment using container development and orchestration/management frameworks such as Docker and Kubernetes. We’ve also see the development of purpose-built machine learning deployment frameworks such as MLFlow. These capabilities, now being considered a new area of data and AI capabilities termed as Machine Learning Ops, are more likely to be taken up by organizations that are already using machine learning beyond the stage of prototypes and proof of concept activities. Mainstream technology firms and firms in manufacturing, energy and retail sectors may find less direct use for these technologies in the immediate future, unless they’re already building machine learning at scale. Containerized and similarly managed machine learning applications are important in the context of organizational agility to deploy ML capabilities to production and to have fast responses to production-scale ML model performance issues, such as model drift. Further discussion on this topic will be in a future post, since it gets a bit technical from this point on.

Sensors as our Saviours: Measured Surroundings

It goes without saying that Covid-19 has put a lot of focus back on China – the nation where it all started. From examination of the conditions that could have led to the cross-species transfer of the virus from bats to humans via pangolins, to broader examinations of the policy impact and the impact of social and cultural norms of Chinese food habits – there has been a lot written on the subject. It remains though, that this is an exceptional or rare event. Short of calling out the diets and social habits of the Chinese broadly, any root cause analysis that is scientifically minded has to start with the underlying conditions that lead to such transmissions, and that is perhaps a pathologist’s or an epidemiologist’s project.

Beyond simplistic monitoring of the conditions for the formation and transmission of such diseases, there are other direct applications for sensor based systems that can monitor and track environments where humans work – some of these measures over time could improve sanitation processes, especially in high risk zones that have historically been prone to infection.

Those in the IoT space should probably note the extensive need for no-touch systems, which we are all in need of due to this pandemic. For one thing – a lot of objects in our surroundings and a lot of household and public use items we need for daily life require direct physical contact – this repertoire of devices ranges from the ordinary smartphone or tablet screen, all the way to the simple devices which power our homes such as switches, door knobs, taps and faucets. It is clear that there is a new design philosophy that can benefit us all in such times. Providing smart sensor based systems that can open doors automatically, dispense soap automatically, or otherwise sanitize bathrooms and other public spaces, could be a shot in the arm. While these systems aren’t exactly breakthrough innovation for most companies, their widespread adoption hasn’t happened because of the relatively low cost of alternatives, and the high cost of adoption and maintenance. Once this entry barrier is broken either by governmental mandates and policies, or by increased public awareness, large scale IoT solutions like this, could take on additional veneers of sophistication – ranging from gesture recognition to automated sickness detection, automated reporting of sick/needy people in public spaces, for instance, or in sophisticated cases, automated interventions.

Sensors as our Saviours: The Measured Human

Another important theme related to Data and AI in the post Covid-19 world is one stemming directly from the sensor technologies that are mature today, but have not been adopted at large scale for reasons of ethics, cost and other considerations. The instrumented human, or the measured human is a concept that is at once both interesting and fraught with danger, probably ultimately because we each have a deep seated fear of being far simpler than we really are. More accurately, we are afraid of being manipulated by those with data on us. My own contention is that this is not just plausible in the post-Covid-19 world, but that it is a strong possibility. Let me explain.

Social media is an extraneous barometer that provides a periscope into the individual for the powers that be, while the sensors of the future that are embedded in our bodies, could become internal barometers of health. Today, we see people sounding off on social media (me included) on issues that affect us – and these messages are a representation of our thoughts and feelings not only on subjects at hand, but also are indications of our propensities and proclivities towards completely oblique issues from those that we’ve expressed ourselves on. In a sense, we’re putting ourselves out there everyday, and for no good reason. That data has been weaponized before. We have seen the repeated use by politicians, media houses and technology firms of the data we volunteer or otherwise allow them to collect, to manipulate us into buying new products, or clicking on ads (if anyone does indeed click on ads anymore), and even vote for this or the other political entity. In the age of the measured human, where we may see sensors measuring everything from our body temperatures (at different locations on our bodies) and our blood pH, to antibody and platelet counts in our blood, and so forth.

When we have this wealth of information available to an algorithm, leave alone a doctor, we could identify the propensity for specific conditions, and administer preventive medicine. Equally, such data could be misused in negative ways, just as personal data today is used to exclude individuals from opportunities and from credit. For example, data about personal medical metrics could be misused by health insurance providers, especially in cases where applicants may have pre-existing conditions. There are no technological solutions to such sub-problems, however, and the solutions are likely to come from good processes and a reflective, empathetic design process for these systems, rather than one which prizes the short term gains to the insurer or other enterprise in question.

The Short Term: Innovation vis-a-vis Big Government and Coverups

The Covid-19 crisis has revealed two sides and two scales of the global community’s response to the problem. One of these sides is the innovative side, as depicted by Italian doctors who repurposed scuba diving gear to treat patients in the face of equipment shortages for ventilators. The other side of this tragedy is the massive coverup – which we are nevertheless told never happened. One of these – the innovative side – has been more prominent in individual responses and community responses to Covid-19, whereas the other, more pernicious side of the global community’s response has been seen more often in big tech and big government.

It has been easy (and rather cheap) to speculate on the innovations that could solve some problems we face in the Covid-19 world. Here’s one interesting example of a thought/ideation experiment around masks. Masks have become a contentious topic both for the WHO who bungled their advice to the world at large and as a source of the next wave of tech innovations. This is one of the guys I have really come to respect from his Twitter posts on Covid19, Balaji Srinivasan:

Closer to home in Bangalore, India, there are startups coming up with sanitization equipment such as this Corona Oven, that enable a wide range of accessories and objects to be sanitized:

Product innovations like these will solve some of the problems we may face in the post-Covid-19 world. They may help us adjust to the new rhythms of life and work, and enable us to get the bottom layers of Maslow’s hierarchy out of the way, by enabling us to manage the food, shelter, security and safety of ourselves and our families. They also provide the opportunity to add new product features on top of them. Kano’s model of product evolution comes to mind, and is relevant. Just as the smartphone evolved from a pureplay voice communication device to an avenue for media consumption, and became a platform for new value, prosthetic devices such as masks could enable us in new and unforeseen ways – and it wouldn’t have been possible without the specific adversity this crisis brought to industrial design and engineering teams.

From the frontiers of machine learning, numerous innovations in computer vision have been brought to bear on Covid-19 X-ray and other data, to detect and prevent certain conditions that arise during the respiratory illness that Covid-19patients experience. Some of the other techniques rely on proxies to enable prevention, as seen in this research below:

The first response of China has been to cover up the scale of their Covid-19 infections and under-report the number of deaths from Covid-19. A casual glance at the Covid-19 curves for China vis-a-vis other nations that suffer from this crisis makes one wonder whether we’re seeing reliable numbers here. Many have spoken about the impact of Covid19 on governments – the act of strengthening governments by centralizing policy can rarely happen in democracies. One reason for this is the stabilizing power of vocal opposition parties in democracies. With the Covid-19 shock, the government’s instruments will be brought to bear on curbing personal freedoms where that may be a requirement, to prevent cascading infections. We’ve seen steps taken by nearly all nations around the world to curb freedoms, impose lockdowns, and as far as this is done in good spirit and with an intent to return the state to normalcy, we can come out of Covid19 with rights and freedoms intact. In the case of some countries though, this doesn’t apply – China being a key example here, because by definition the ruling party limits the freedoms of people and essentially fields the biggest army in existence for any political party on earth. Generally speaking, the coverup that China managed could not have been managed in any other country in existence. This in itself was a potent vector for transmission of Covid19, especially given China’s important place as the world’s manufacturing powerhouse. Which brings me to the next disruption from Covid19 to the world’s economic system: the world’s supply chain and manufacturing industries.

More generally, popular world leaders have talked about minimizing government footprint for decades in the post-Cold-War era. In recent years, we’ve heard slogans such as “Government has no business being in business” and “Minimum government, maximum governance” in the context of government intervention in industry and value delivery to consumers nation-wide or even across nations. The Covid-19 pandemic and crisis have ignited an interesting dichotomy in government and politics – should governments run institutions such as hospitals and healthcare mandatorily? If the answer to this is not an absolute no, to what extent should they? Perhaps some elements of a response to this question has to be how big the nation is, and how many caregivers are required, and what means the government has to ensure quality of service to patients despite operating at large scale – and these are relevant questions in the age of Covid-19. Generally (and one could say historically) crisis times (such as times of war, famine or epidemics) embolden governments to take strong countermeasures and revoke freedoms while enabling government officials to move faster. With the scale of democracy we have in large countries like the US and India, we probably need big government to enable the leaders to serve the greater good. The other side of the coin of course is the fact that very powerful governments don’t tend to part with that power easily, and excessive power concentration is a slippery slope leading to further mismanagement of countries.

Data and AI can be exceptional tools to enable data-driven governance. In a sense, if we are to look beyond the normal tendency to extend control from the government to the grass roots and lock things down from the top down, we could enable citizens to take informed decisions, by educating them about phenomena that could affect them, the consequences to them and to society at large, and then implore right action from them. Transparency is, in other words, the weapon of strong and sustainable data-driven democracies, because such democracies rely on the facts and information to take decisions, and not based on presumptions of behavior.

The onus for such dissemination of data to enable data-driven governance should fall squarely on governments. Governments often put the onus of interpreting and transmitting vital information on the media – and this model is fraught with problems. From the sensationalist news stories to erroneous reporting to putting important stories behind paywalls and “cookie clutter” screens, the world of internet news reporting is an unmitigated mess that’s accelerating towards becoming a train wreck of a disaster for the consumer. It didn’t have to be this way. Platforms like Twitter and Facebook have been accused of fomenting unrest, of political and other kinds of bias, and despite these reputations, they’re platforms on which important news has to be disseminated and consumed. These platforms are also simplistic and don’t lend themselves well to data-driven journalism. This isn’t a business or data/AI problem so much as a policy problem. If access to the internet is increasingly more important, access to authentic information on it should also be so, and the post-Covid-19 world, especially given the excesses of various world governments and media houses, will likely see a metamorphosis of the status quo.

Additional consequences of the Covid-19 crisis, is to accelerate the adoption of electronic payments the world over, enabled again by telecommunications, and perhaps the growth and acceleration of blockchain technologies for veracity in news and transaction reporting.

Supply Chains and Manufacturing: Globalization to Autarky?

The proverbial elephant in the room for manufacturers around the world in the post-Covid-19 world is the global supply chain, specifically how fragile their businesses have become due to over-reliance on China and the goods and components it produces for the world. From cheap toys and car parts to computer chips and smartphone screens, there are few things China is incapable of producing at large scale today, and this excess concentration of supplier bargaining power (to use a phrase from Michael E. Porter), is purely due to the perils of excessive capitalism. I say this as a bit of a capitalist myself – after all, anyone who has benefited from India’s economic liberalization is a bit of a capitalist. What is more important than just the fact that this is a case of capitalism’s excess, is that the global strategy for sourcing our supply chains across manufacturing industries has followed a groupthink, and a daftly simplistic and unstrategic winner-takes-all effect followed. In other words, it isn’t capitalism itself, but the limited set of strategic sourcing options that the West, which has controlled the world economy for decades, has had.

So, in the post-Covid-19 world, what sourcing options do we have? We risk continuing supply chains to run out of China for reasons of continued concentration of power with their firms, and for reasons of political and economic leverage. China’s one-belt-one-road initiative and the encirclement strategies they’ve used in the context of the South China Sea and elsewhere leave little doubt about that nation’s interest in protecting its strategic assets. On the other hand, in the US and in Europe, you have an ageing population and economies that have become unsustainable for taking on low cost manufacturing jobs. In the Middle East, we see spots of bother thanks to the geopolitical and geostrategic situation there, an overreliance on oil for energy and economics and the lack of skilled engineering and manufacturing talent, not to mention issues such as the impact of religious fundamentalism in societies across the Middle East. India and some of the ASEAN nations are relative bright spots. In addition to these, many eastern nations – excluding Taiwan, Korea and other rich Asian tigers – are probably the best places for maintaining competitive advantage in sourcing. We have already seen some countries incentivizing their companies to move out of China, and to other countries, and we will see this sourcing game continue.

However, even this approach only seeks to prolong what many have been calling the Old World Order of Globalization.  The new era, they claim, will see autarky to a great degree, where in-sourcing or domestic sourcing, and self-reliance will be the order of the day, where boundaries will be drawn again and nations will be closed off from others for years if not decades. Friends and enemies in this new world order will perhaps be long term relationships geostrategically, and the world order we see now, that is unable to solve every man’s ordinary problems (affordable healthcare, for one). Even in this world, one can foresee trade being an alternative for countries without the wherewithal to become autarkic owing to resources. The success of countries in the Middle East, for example, is due largely to their oil exports. In an autarkic world, the transitions made to today’s automotive sector will tend towards electrification, one hopes, which means overreliance on indigenously produced energy in each country, and under-reliance on sources such as oil from the Middle East. However, is this idea of pure autarky a step too far? Perhaps.

Data and AI capabilities are just being explored in the context of global supply chains. From older systems such as bar code scanners and object counters that track objects on conveyors, to modern ones such as computer vision and prescriptive analytics for on-time supply and demand matching in large supply chains, from voice-based ordering systems to no-waiting check out counters companies like Amazon and Walmart have adopted data and machine learning at scale and are putting together compelling examples of how to run much larger supply chains at global scale. Some of these technologies will fare them well in the post-Covid-19 world, although one can imagine a number of products in the post-Covid-19 world being sourced to these e-commerce platforms from places other than China – for whatever reasons. However, I foresee that these large digitized, high tech supply chains will be important even in the post-Covid19 world. American autarky, in other words, seems a distant dream, or more accurately, a lost utopia.

Environmentally Conscious Business in the Post-Covid-19 World

It isn’t an exaggeration to say that the economics of Covid-19 are being stressed more than the root cause of the problem – that of infectious diseases and how they spread. A large part of the reason why SARS, MERS and now Covid-19 have spread, is because of the conditions and policies in China and elsewhere that allowed it. Specifically, Chinese policies on wet markets and their breeding of Chinese wildlife for human consumption has been one of the contentious underlying topics of discussion.

More broadly, it indicates the importance of environmentally conscious business practices and their importance. Far too often, we settle for what is pragmatic and benefits humans, and don’t emphasize the impact to the environment at large from our actions in business. This may seem like a simplistic complaint, but in fact it is a deep and important one. The depth comes from the fact that the enterprise and the businesses we run are but one set of processes in a broad chain of environmental processes that sustain the planet. When we have simplistic policies concerning complex systems, we risk, in the words of Nassim Taleb, naive interventionism (iatrogenics), where we are unsure what true consequences our actions can have, even if we execute those actions “in good faith” or “to the best of our knowledge”. The cycle of value from the animals in question – bats and pangolins – is vast. They are part of an ecological system of balance where upon consuming lower forms of life that they prey upon, these animals enable the sustenance of a balance. When that balance is upset by either close proximity of species that rarely meet in the wild, or by selective breeding of these species, or by other means, there are likely to be shocks to that ecological balance from which we may never fully recover.

Learning and Staying Relevant in the Post-Covid-19 World

For me personally, the last several years have revealed the power of the internet to educate. From short courses on various topics in data science, machine learning and AI to extensive courses such as in a post-graduate diploma – all of these seem to be worked out for large scale skill development online. On the internet today, there are numerous free resources especially for those in technology, computer science, software, data and analytics – these areas of contemporary advances and cutting edge research see a surfeit of information and content which is near-well an embarrassment of riches – an this is all good until we see a gap between supply and demand. Numerous learning opportunities have been opened up specifically post-Covid-19 . Google, EdX and Coursera all announced a number of new courses, some of them free. If you know where to look you can find incredible content on the internet to teach you nearly anything in machine learning from the basics to the latest algorithms and research.

But here’s the thing – in reality, there is a supply vs. demand gap in online learning. Specifically, there is a great deal of supply of content and courses in a few areas of technology, science and engineering, and largely nothing in other areas. There is research hidden behind paywalls in important areas such as epidemiology, which is a core research domain as regards the Covid-19 crisis. This huge disparity is also a problem of curation, of practicality in the dissemination of certain subjects, and of economies of scale.

The internet as a medium is not best suited for teaching certain skills – sitting down in front of a computer is not the best way to learn how to turn a component on a lathe in a machine shop, or how to play a guitar (although you could argue about the latter, as I myself have learnt a lot of guitar just by looking things up online and from practice from self-study). The limitations of this medium in disseminating certain kinds of knowledge is well known and well attested and yet there are attempts to move entire courses, and even masters degrees online. While initially this was seen questionably, in the post-Covid-19 world, we can assume that such online learning will gain momentum – and if my experiences have taught me anything, it is that with the right tools and interactions, you can learn a surprising lot online.

Staying relevant in the post-Covid-19 world is a harder task than just learning in this increasingly socially isolated and digitized world. Learning is just the acquisition of skill, whereas relevance is a consequence of being the right person, in the right environment. The latter is therefore equally contingent on skills and on our own explorations of and (entrepreneurial or other) responses the conditions we’re in – and this consequently influences how we use these skills that we have acquired. For instance, we know that in the post-Covid-19 world, there are likely to be sea changes in which industries are relevant and which ones aren’t. Putting our feet in the right places, and bringing value to these new interactions that we become part of, can make all the difference between whether we’re relevant in the future and creating some of the history, or whether we’re just another casualty of history.

Concluding Remarks

The world post-Covid-19 is a time of change, indicating a complex, new reality. There are economic shocks will will impact us for years, if not decades to come. We’re in a place of incredible opportunity vis-a-vis a position that poses incredible challenges as well. Enterprises as we knew them will change forever, adopting new styles of work and learning, and professionals will awaken to a new age of online learning and a protracted search for relevance and professional meaning in some cases. Smart governments will adopt data and communicate and govern based on facts – even as others will use these opportunities to grow large scale government influence; indeed, questions of governmental oversight on essential services including public health will be debated for years to come. Data and AI adoption in enterprises will accelerate in enterprises and will enable new kinds of collaboration and remote work, required for these months and perhaps years of social distancing and isolation. Enterprises will accelerate their move to the cloud, benefiting from large scale and low cost services for data, web, and other technologies. Emerging technologies such as augmented and virtual reality may become a staple of our boardrooms and classrooms. More and more learners will try and adapt to online learning, and more teachers and professors will be compelled to learn to teach on this medium, even as new technology interventions will improve learning experiences. As many governments around the world will rush to build self-reliance and their respective versions of autarky on many essential manufactured products, the global supply chain will start looking different, and we may see the greater infusion of data and AI technologies in the businesses that control our supply chains and logistics. We may see the growth of blockchain and other trust-centric technologies, for applications in medicine and the news, in addition to finance where it finds its most common use cases. The post-Covid19 world is a clarion call to problem solvers and innovators of all kinds, as much as it is for those in policy and governance, public health and medicine. The world order has been upset, and the new world order that will manifest after this pandemic is behind us, will look to the resourceful and the inventive even as people look towards being part of sustainable, healthy and safe work and living environments in the future.