This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages) This article may incorporate text from a large language model, which is prohibited in Wikipedia articles. It may include hallucinated information, copyright violations, claims not verified in cited sources, original research, or fictitious references. Any such material should be removed. The reason given is: some weird prose in previous versions and a lot of bold listing (August 2026) (Learn how and when to remove this message) This article may require copy editing for lots of unnecessary caps. You can assist by editing it. (August 2026) (Learn how and when to remove this message) (Learn how and when to remove this message)

Feature Importance (or Variable Importance or Feature Attribution) refers to a set of techniques and mathematical frameworks used in machine learning, statistics, and their applications to quantify the contribution of input variables (features) to a model's output or to the underlying data-generating process. It is a component of Explainable Artificial Intelligence (XAI) and Interpretable Machine Learning (IML) and is used for model improvement, scientific inference, and feature selection.^[1]

Feature importance is a context-dependent score that varies based on the scope of the explanation, the model (or data) of interest, and the ultimate purpose of the analysis.^[1]

The earliest forms of feature importance assessed the strength of the relationships between pairs of variables within animal biology or human psychology using methods such as Francis Galton's correlation coefficient.^{[2][non-primary source needed]} Sewall Wright further developed variable importance, by introducing path analysis in 1921, to understand causal influences in complex systems.^{[3][non-primary source needed]} Path analysis determines correlative influences along direct paths by decomposing correlation and partial correlation coefficients into their path-based components.^{[citation needed]}

With Leo Breiman’s introduction of Random Forests, feature importance expanded to more nonlinear models and explanations. He introduced "permutation importance" sometimes measured by gini index.^[4] Simultaneously in 2001, Lipovetsky and Conklin applied the Shapley value from cooperative game theory to regression, providing a consistent method for variance attribution in the presence of multicollinearity.^[5]

Formal feature importance frameworks use mathematical axioms to define consistent scoring criteria and validate the theoretical properties of different estimation methods.

Shapley values are the unique solution that satisfies four core game-theoretic axioms, which describe how to fairly distribute the "total gain" of a model's prediction among the participating features.^{[6] [7]}

1. Efficiency (Local Accuracy): The sum of the importance scores ${\displaystyle \phi _{j}}$ for all features must equal the difference between the model's prediction for an instance ${\displaystyle f(x)}$ and the expected prediction ${\displaystyle E[f(X)]}$.
2. Symmetry: If two features ${\displaystyle i}$ and ${\displaystyle j}$ contribute exactly the same value to every possible subset of other features, they must receive the same importance score: ${\displaystyle \phi _{i}=\phi _{j}}$.
3. Dummy (Null Player): If a feature contributes nothing to the value function for any subset of features, its importance score must be zero. This is crucial for identifying irrelevant features.
4. Additivity (Linearity): If the value function is the sum of two functions ${\displaystyle v=v_{1}+v_{2}}$, then the importance scores must be the sum of the scores calculated for each function: ${\displaystyle \phi _{j}(v)=\phi _{j}(v_{1})+\phi _{j}(v_{2})}$.

While the Dummy (Null Player) axiom is desirable for obtaining feature importance scores that can be used to optimize a model, this is not the case for explaining the data, since variables may be highly correlated (and hence offer little value to the existing model), yet remain causally linked to the response variable. Alternative axioms have been proposed for feature importance methods that intend to explain the data.^[8]

• Blood Relation (Causal Graph Association): A feature should have non-zero importance if and only if the feature is blood related (associated) with the response in the ground-truth causal graph.
• Marginal Contribution: The importance of a feature must be at least as high as the gain it provides when added to the set of all other features.
• Elimination: Removing other features from the feature set can only decrease (or leave unchanged) the importance of a remaining feature. It cannot increase it.
• Invariance under Redundant Information and Symmetry under Duplication: Adding a redundant feature should not change the importance of preexisting features, and identical features should receive equal importance.

The choice of method is categorized based on the user's objective.

• Model Explanation: The goal is to understand the "logic" of a black-box model to ensure safety, fairness, and reliability. Methods like SHAP, LIME, and Accumulated local effects are standard here.
• Data Explanation (Scientific Inference): The goal is to learn about the real world. Researchers prioritize methods that handle redundancy and correlation in a way that reflects the true underlying relationships (e.g., MCI, UMFI).^[9]
• Model Optimization (Feature Selection): The goal is to improve the model's performance by removing irrelevant or redundant features. Techniques like Recursive Feature Elimination (RFE) use importance scores as a selection criterion.^[10]

Feature importance methods have been classified into four distinct settings based on two axes: Global vs. Local^[1] and Data vs. Model.^{[11] [12] [13]}

	Reference: Model	Reference: Data
Global	Global-Model Importance: Explains how a trained model behaves across the entire dataset. It identifies which features the model generally relies on for its predictions.	Global-Data Importance: Explains the true relationships in the underlying phenomenon. It seeks to identify the intrinsic predictive power of features within the population, regardless of a specific model's choices.
Local	Local-Model Importance: Explains why a specific prediction was made for a single instance. It quantifies the influence of each feature on that a specific model's specific outcome.	Local-Data Importance: Explains the role of a feature for a specific individual in the real world (e.g., why a specific patient developed a disease), focusing on the causal or statistical dependencies for that point.

Methods like SHAP and LIME are local-model. Global-model importance metrics include permutation importance and SAGE.^[13] Global-data methods include MCI and UMFI. ^[11]

Feature importance methods often differ in how they treat the correlation between features. In the presence of correlated features, predictive credit can be distributed in a variety of ways. Methods can therefore be classified based on how they assign credit to correlated features, where one end of the spectrum is given by marginal feature importance, and the other is given by conditional feature importance.^{[14] [15] [16]}

• Conditional Feature Importance: Evaluates a feature by conditioning on the values of all other features. This approach respects the dependence structure of the data and measures the unique information a feature provides that is not already captured by other variables. Methods in this category include conditional permutation importance^[14], Leave-One-Covariate-Out, and partial correlation.

• Marginal Feature Importance: Relies on associations between the response and predictors, regardless of multicollinearity. Purely marginal methods assign the same high importance to all features. Examples include correlation, marginal contribution feature importance, and ultra-marginal feature importance.^[8]

Methods such as SHAP and permutation importance are somewhere in between the two extremes as importance is shared among correlated features.

The technical implementation of the importance measure often dictates its applicability to different model architectures.^[17]

Mechanism	Description	Examples
Gradient-Based	Utilizes the derivatives (gradients) of the model's output with respect to its input features. These are typically model-specific and used for differentiable models like neural networks.	Saliency Maps, Integrated Gradients, Grad-CAM, DeepLIFT.
Non-Gradient Based	Treats the model as a "black box" and relies on perturbations, shuffling, or submodel training. These are typically model-agnostic and applicable to any algorithm.	Permutation Importance, KernelSHAP, LOCO, MCI.

Achen (1982) introduced a classification of linear regression-based feature importance methods: "dispersion importance" (explained variance), "level importance" (impact on the mean), and "theoretical importance" (the change in response for a given change in regressor).^[18]

• Medicine^[19][20]
• Earth Science^[21]
• Computer Science^[22]

• Feature selection
• Explainable Artificial Intelligence
• Shapley value