Context-Driven Data Mining through Bias Removal and Data Incompleteness Mitigation
The results of data mining endeavors are majorly driven by data quality. Throughout these deployments, serious show-stopper problems are still unresolved, such as: data collection ambiguities, data imbalance, hidden biases in data, the lack of domain information, and data incompleteness. This paper is based on the premise that context can aid in mitigating these issues. In a traditional data science lifecycle, context is not considered. Context-driven Data Science Lifecycle (C-DSL); the main contribution of this paper, is developed to address these challenges. Two case studies (using data-sets from sports events) are developed to test C-DSL. Results from both case studies are evaluated using common data mining metrics such as: coefficient of determination (R2 value) and confusion matrices. The work presented in this paper aims to re-define the lifecycle and introduce tangible improvements to its outcomes.
- Pub Date:
- October 2019
- Computer Science - Machine Learning;
- Computer Science - Social and Information Networks;
- Statistics - Machine Learning
- 1st Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning (EDML 2019) At SIAM - Society for Industrial and Applied Mathematics