Critical success factors for managing data science projects within agile methodology

Abstract

Data science is an evolving discipline with a major emphasis on developing accessible data analysis techniques. But far less emphasis has been given on the factors which affect the success of data science projects under the Agile umbrella. In the field of software engineering agile approaches were initially developed and are characterized by their iterative software development approach. It is recommended to use process models or methodologies in literature to increase the success rate of data science projects however, organizations, which are perceived to be too restrictive and do not accept the traditional iteratives and transparent nature of data science projects, reluctant to use them. And there are some potential challenges which have been identified in the literature for using Agile methodologies in data science projects. The characteristics of possible critical success factor (CSF)s for Data Science projects have been established from the literature by updating Chow and Cao's list of success factors for agile software development projects in this research. The factors have been identified under five dimensions of organizational, people, process, technical and project. The findings of this study indicate team environment, team capacity, client engagement, project definition processes, agile software engineering techniques and project schedule as the factors that impact the success of data science projects within Agile methodology. Even though these factors were listed as important for managing data science projects within Agile methodology, the significance of these factors may vary according to the nature of the project that the team is involved in. Therefore, the team should always focus on these factors relative to the nature of the project.