Explainable Validation and Feature Reduction in Machine Learning

Although there are many available approaches for designing machine learning models and experiments, not all of these approaches are explainable or interpretable. Combination frequency differencing (CFD) provides information on the relationship between feature values and predicted class in the form of distinguishing combinations. In this paper we investigate how distinguishing combinations in the training and validation datasets may affect the effectiveness of the trained model. We also investigate how distinguishing combinations can be used for a more explainable approach to feature reduction. Notably, results showed that CFD performed comparably to principal component analysis, an established method for reducing features, while providing better explainability. Thus CFD shows promise for providing a unified, interpretable framework for dataset construction, model validation, and feature reduction.