Hidden Technical Debt in Machine Learning Systems
نویسندگان
چکیده
Machine learning offers a fantastically powerful toolkit for building useful complex prediction systems quickly. This paper argues it is dangerous to think of these quick wins as coming for free. Using the software engineering framework of technical debt, we find it is common to incur massive ongoing maintenance costs in real-world ML systems. We explore several ML-specific risk factors to account for in system design. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues, changes in the external world, and a variety of system-level anti-patterns.
منابع مشابه
Machine Learning: The High-Interest Credit Card of Technical Debt
Machine learning offers a fantastically powerful toolkit for building complex systems quickly. This paper argues that it is dangerous to think of these quick wins as coming for free. Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning. The goal of this paper is highlight several m...
متن کاملDetecting Technical Debt through Issue Trackers
Managing technical debt effectively to prevent it from accumulating too quickly is of great concern to software stakeholders. To pay off technical debt regularly, software developers must be conscious of the existence of technical debt items. The first step is to make technical debt explicit; that is the identification of technical debt. Although there exist many kinds of static source code ana...
متن کاملDebt Collection Industry: Machine Learning Approach
Businesses are increasingly interested in how big data, artificial intelligence, machine learning, and predictive analytics can be used to increase revenue, lower costs, and improve their business processes. In this paper, we describe how we have developed a data-driven machine learning method to optimize the collection process for a debt collection agency. Precisely speaking, we create a frame...
متن کاملA Multiworld Testing Decision Service
Applications and systems are constantly faced with decisions that require picking from a set of actions based on contextual information. Reinforcement-based learning algorithms such as contextual bandits can be very effective in these settings, but applying them in practice is fraught with technical debt, and no general system exists that supports them completely. We address this and create the...
متن کاملDatabase Normalization Debt: A Debt-Aware Approach to Reason about Normalization Decisions in Database Design
Technical debt is a metaphor that describes the long-term effects of shortcuts taken in software development activities to achieve near-term goals. In this study, we explore a new context of technical debt that relates to database normalization design decisions. We posit that ill-normalized databases can have longterm ramifications on data quality and maintainability costs over time, just like ...
متن کامل