Machine Decisioning Offers Auditable Alternative to Human Bias in Critical Fields

Original source: Humans of Martech Podcast
This article is an editorial summary and interpretation of that content. The ideas belong to the original authors; the selection and writing are by Streamed.News.

This video from Humans of Martech Podcast covered a lot of ground. 6 segments stood out as worth your time. Everything below links directly to the timestamp in the original video.

Wondering if artificial intelligence can truly make the world a fairer place? This perspective suggests AI's greatest impact might be in overcoming human biases in our most sensitive systems.

Machine Decisioning Offers Auditable Alternative to Human Bias in Critical Fields

Toby Konitzer asserts that machine decisioning extends far beyond marketing applications, holding the potential to significantly reduce systemic biases and suboptimal outcomes in critical areas such as healthcare, hiring, and criminal justice. He highlights that human decision-making, influenced by factors like mood and inherent biases, is often both flawed and unauditable. Machine decisioning, conversely, provides transparent, auditable processes, allowing for precise tracking of how and why a decision was made. This shift offers a profound benefit, moving towards fairer outcomes in contexts like traffic stops and criminal sentencing, where human biases are well-documented. Konitzer sees this as giving Agentic AI a "civilizatory ambition" beyond mere monetization, tying its advanced capabilities to tangible improvements in societal fairness and efficacy by leveraging auditable decision pathways.

"The promise of machine decisioning is in the very least the decisioning process becomes auditable."

▶ Watch this segment — 52:14

Deploying Agentic AI on Correlational Data Poses 'Disastrous' Risks, Expert Warns

Toby Konitzer warns against the dangerous trend of letting Agentic AI operate freely on data warehouses filled with purely correlational data. He explains that without a clear understanding of causal relationships, AI may recommend actions based on spurious correlations, like suggesting dog walker gift certificates to increase food sales because dog owners often spend more on food. Such interventions, lacking causal backing, can backfire dramatically. This "lazy thinking" risks accelerating negative dynamics, where AI amplifies counterproductive actions across millions of examples, potentially sabotaging key performance indicators. Konitzer advocates for a "causal customer context graph" to guide Agentic AI, ensuring it distinguishes between causal effects and mere correlations, thereby preventing disastrous outcomes and fostering responsible AI implementation.

"Agentic can accelerate the things that are good, but it will also accelerate the things that are bad to be really plain spoken. And it doesn't distinguish between the two if you don't."

▶ Watch this segment — 37:02

Human Psychology Hinders Adoption of Superior AI Experimentation, Expert Claims

Toby Konitzer observes a paradox in decisioning science: despite the mathematical superiority and efficiency of dynamic allocation (reinforcement learning) over traditional A/B testing, many sophisticated companies, even those with numerous PhDs in the field, continue to use fixed A/B testing. Dynamic allocation continuously shifts traffic to winning conditions, optimizing for metrics like customer lifetime value (LTV), whereas fixed A/B testing wastes traffic on inferior options during the learning phase. The reason for this adherence to a suboptimal method lies in the psychological aspect of decision science. Konitzer explains that dynamic allocation, being more complex, is harder for internal stakeholders and executives to understand and approve. The need for explainability to convince leadership often leads to the adoption of simpler, albeit less efficient, approaches, even if it results in a suboptimal outcome for the business.

"It's a human bias that leads to a suboptimal outcome, right? But if you're the person enacting this... your job is to actually convince the stakeholder."

▶ Watch this segment — 22:01

Dynamic Allocation Offers Efficient Alternative to Traditional A/B Testing

Toby Konitzer explains dynamic allocation, also known as reinforcement learning or multi-armed bandits, as a more efficient approach to experimentation than traditional A/B testing. While traditional A/B testing randomly splits traffic, typically 50/50, to evaluate different interventions for a fixed period, dynamic allocation continuously monitors performance and dynamically allocates more traffic to the "winning" conditions as data comes in. This method optimizes for specific metrics, such as customer lifetime value (LTV), by minimizing the amount of traffic sent to inferior options, thereby reducing the "cost" of experimentation. Konitzer highlights that while traditional A/B testing is valuable for establishing causal differences, dynamic allocation is superior for optimization, which he argues is the primary goal for most lifecycle marketers. The process involves a trade-off between "exploration" (learning) and "exploitation" (optimizing based on current best options).

"The whole point of dynamic allocation or reinforcement learning is that you could do both things in parallel... you actually pipe more traffic to the condition that is winning, which... is more efficient."

▶ Watch this segment — 16:15

Marketers Must Shift from Prediction to Causal Intervention, Expert Argues

Toby Konitzer identifies a "prediction trap" in marketing, where highly accurate predictive models, such as those for customer lifetime value (LTV) or churn likelihood, only describe what will happen if nothing changes. He argues these models are not directly useful for marketers, whose core job is to implement interventions that actively influence customer behavior. Instead of merely predicting who might churn, marketers need to understand how to prevent churn through causal levers. Konitzer emphasizes that marketing fundamentally operates in a causal world, where the objective is to determine how changing 'X' will affect 'Y'. This shift from observational prediction to causal intervention is gaining traction, with boardrooms increasingly demanding answers to causal ROI questions, signaling a maturation of the marketing field towards more outcomes-based thinking and reinforcement learning.

"The world of marketing is a causal world... if I change X, what is the outcome in Y? How can I maximize Y? Otherwise, there is really no point for marketing."

▶ Watch this segment — 3:11

Uninformed AI Can Sabotage Revenue KPIs in 'Boomerang Effect,' Expert Warns

Toby Konitzer describes the "boomerang effect" in self-learning AI systems, where uninformed Agentic AI, especially without causal priors, initially applies random and potentially counterproductive treatments. During this "cold start problem" phase, the system can inadvertently sabotage key performance indicators, such as customer lifetime value (LTV), for weeks or months before it learns effectively. For instance, an AI might recommend a dog walker gift certificate that customers find intrusive, leading to decreased loyalty and revenue. Even if initial interventions are merely neutral rather than actively harmful, they still waste resources and time without exploring truly beneficial options. Konitzer stresses the need for a mechanism, such as a "causal customer context graph," to provide AI with pre-existing causal knowledge. This semantic layer helps guide the system, preventing it from making initial suggestions that backfire and allowing it to explore a more effective and curated range of interventions from the outset.

"Your initial ideas backfire causally... in the meantime, you actually have a lowering of LTV."

▶ Watch this segment — 31:06

Also mentioned in this video

• The idea that predictive models are not useful for marketers because they only… (0:00)
• Correlative impact on LTV is often mistaken for causal returns, using the… (9:03)
• Marketing is a difficult job requiring an understanding of causal inference,… (13:10)
• Composable decisioning within customer journeys, where models learn in… (48:12)
• Toby Konitzer shares his personal system for staying aligned with what makes… (1:00:33)

Summarised from Humans of Martech Podcast · 1:03:36. All credit belongs to the original creators. Streamed.News summarises publicly available video content.