Practical model interpretation
Know what matters in your model.
Feature Importance is a practical reference for computing, interpreting, and questioning model explanations. Use it to identify the features that matter, understand why they appear important, and avoid claims the evidence cannot support.
A useful question
If this feature stopped carrying information, how much worse would the model get?
That question is often more useful than asking whether a feature has a high-looking score. This site is organized around questions like that: practical, testable, and honest about limitations.
Guides
Start with the guide closest to your question. The library is organized by method, model family, library, and interpretation problem.
library
Feature Importance in Python
A cross-library Python guide covering scikit-learn, XGBoost, LightGBM, and SHAP.
method
Permutation Importance
Measure how much validation performance drops when one feature is shuffled.
model
Linear Regression Feature Importance
Use coefficients, standardized coefficients, and permutation importance for linear regression models.
model
Logistic Regression Feature Importance
Use coefficients, odds ratios, standardized coefficients, and permutation importance for logistic regression.
library
Feature Importance in Scikit-learn
Compute and interpret tree-based and permutation importance with scikit-learn.
concept
SHAP vs Feature Importance
Compare global importance rankings with local contribution explanations.
model
Random Forest Feature Importance
Understand impurity-based importance, bias, variance, and correlated predictors in forests.
library
XGBoost Feature Importance
Use gain, weight, cover, permutation importance, and SHAP with XGBoost models.
concept
Feature Importance vs Correlation
Separate model reliance, statistical association, leakage, and causal interpretation.
concept
Model Interpretability
Use feature importance as one part of a broader model interpretability workflow.
What feature importance means
Feature importance is a family of techniques for estimating which input variables a trained model relies on. It can help you debug a model, explain broad model behavior, select follow-up analyses, and notice when a model is using suspicious signals.
It does not automatically explain causality, fairness, or product impact. A feature can be important to prediction because it is a proxy, a shortcut, a leakage path, or a genuine signal. The method and the dataset determine what the score can support.
Methods worth knowing
Start with the method that matches your model and question. Then check whether the result is stable enough to trust.
Permutation importance
Shuffle one feature at a time and measure how much model performance drops. Useful because it works across many model types.
Tree impurity importance
Read importance from tree-based models such as random forests. Fast, but biased toward some feature types.
Model coefficients
Use coefficients from linear or generalized linear models when inputs are scaled and the model assumptions are acceptable.
SHAP values
Estimate local and global contribution patterns. Powerful, but easier to misuse than the charts suggest.
Common traps
- Calling an importance score causal evidence.
- Ranking features from a model that does not perform well.
- Ignoring correlated features that share or hide importance.
- Comparing raw coefficients across differently scaled inputs.
- Trusting a single chart without validation, uncertainty, or repeats.