REFERENCES
[1] T. M. Mitchell et al., Machine learning. McGraw-hill New York, 2007, vol. 1.
[2] Z.-H. Zhou, Machine learning. Springer Nature, 2021.
[3] I. Lee and Y. J. Shin, “Machine learning for enterprises: Applications, algorithm
selection, and challenges,” Business Horizons, vol. 63, no. 2, pp. 157–170, 2020.
[4] C.-J. Wu, D. Brooks, K. Chen, D. Chen, S. Choudhury, M. Dukhan, K. Hazel-
wood, E. Isaac, Y. Jia, B. Jia et al., “Machine learning at facebook: Understand-
ing inference at the edge,” in 2019 IEEE international symposium on high per-
formance computer architecture (HPCA). IEEE, 2019, pp. 331–344.
[5] C. Olston, N. Fiedel, K. Gorovoy, J. Harmsen, L. Lao, F. Li, V. Rajashekhar,
S. Ramesh, and J. Soyke, “Tensorflow-serving: Flexible, high-performance ml
serving,” arXiv preprint arXiv:1712.06139, 2017.
[6] A. Mishra, Machine learning in the AWS cloud: Add intelligence to applications
with Amazon Sagemaker and Amazon Rekognition. John Wiley & Sons, 2019.
[7] A. Paleyes, R.-G. Urma, and N. D. Lawrence, “Challenges in deploying machine
learning: a survey of case studies,” ACM Computing Surveys, vol. 55, no. 6, pp.
1–29, 2022.
[8] L. Leite, C. Rocha, F. Kon, D. Milojicic, and P. Meirelles, “A survey of devops
concepts and challenges,” ACM Computing Surveys (CSUR), vol. 52, no. 6, pp.
1–35, 2019.
[9] F. Erich, C. Amrit, and M. Daneva, “Report: Devops literature review,” University
of Twente, Tech. Rep, 2014.
[10] C. Ebert, G. Gallardo, J. Hernantes, and N. Serrano, “Devops,” Ieee Software,
vol. 33, no. 3, pp. 94–100, 2016.
[11] S. Alla, S. K. Adari, S. Alla, and S. K. Adari, “What is mlops?” Beginning
MLOps with MLFlow: Deploy Models in AWS SageMaker, Google Cloud, and
Microsoft Azure, pp. 79–124, 2021.
[12] A. Lima, L. Monteiro, and A. P. Furtado, “Mlops: Practices, maturity models,
roles, tools, and challenges-a systematic literature review.” ICEIS (1), pp. 308–
320, 2022.
66
[13] Google. Mlops: Continuous delivery and automation pipelines in ma-
chine learning. [Online]. Available: https://cloud.google.com/architecture/
mlops-continuous-delivery-and-automation-pipelines-in-machine-learning
[14] MLOps. Machine learning operations. [Online]. Available: https://ml-ops.org/
[15] S. Mäkinen, H. Skogström, E. Laaksonen, and T. Mikkonen, “Who needs
mlops: What data scientists seek to accomplish and how can mlops help?” in
2021 IEEE/ACM 1st Workshop on AI Engineering-Software Engineering for AI
(WAIN). IEEE, 2021, pp. 109–112.
[16] Y. Zhou, Y. Yu, and B. Ding, “Towards mlops: A case study of ml pipeline plat-
form,” in 2020 International conference on artificial intelligence and computer
engineering (ICAICE). IEEE, 2020, pp. 494–500.
[17] M. M. John, H. H. Olsson, and J. Bosch, “Towards mlops: A framework and
maturity model,” in 2021 47th Euromicro Conference on Software Engineering
and Advanced Applications (SEAA). IEEE, 2021, pp. 1–8.
[18] N. Hewage and D. Meedeniya, “Machine learning operations: A survey on mlops
tool support,” arXiv preprint arXiv:2202.10169, 2022.
[19] A. Chen, A. Chow, A. Davidson, A. DCunha, A. Ghodsi, S. A. Hong, A. Konwin-
ski, C. Mewald, S. Murching, T. Nykodym et al., “Developments in mlflow: A
system to accelerate the machine learning lifecycle,” in Proceedings of the fourth
international workshop on data management for end-to-end machine learning,
2020, pp. 1–4.
[20] M. Zaharia, A. Chen, A. Davidson, A. Ghodsi, S. A. Hong, A. Konwinski,
S. Murching, T. Nykodym, P. Ogilvie, M. Parkhe et al., “Accelerating the ma-
chine learning lifecycle with mlflow.” IEEE Data Eng. Bull., vol. 41, no. 4, pp.
39–45, 2018.
[21] R. Kuprieiev, D. Petrov, S. Pachhai, P. Redzynski, C. da Costa-Luis, P. Rowlands,
A. Schepanovski, I. Shcheklein, B. Taskaya, J. Orpinel et al., “Dvc: Data ver-
sion control-git for data & models,” Zenodo. Available online at: https://zenodo.
org/record/5654595 (accessed October 30, 2021), 2021.
[22] F. Calefato, F. Lanubile, and L. Quaranta, “A preliminary investigation of mlops
practices in github,” in Proceedings of the 16th ACM/IEEE International Sympo-
sium on Empirical Software Engineering and Measurement, 2022, pp. 283–288.
[23] S. Haines, “Workflow orchestration with apache airflow,” in Modern Data En-
gineering with Apache Spark: A Hands-On Guide for Building Mission-Critical
Streaming Applications. Springer, 2022, pp. 255–295.
67
[24] D. A. Tamburri, “Sustainable mlops: Trends and challenges,” in 2020 22nd inter-
national symposium on symbolic and numeric algorithms for scientific computing
(SYNASC). IEEE, 2020, pp. 17–23.
[25] AWS. Why should you use mlops? [Online]. Available: https://docs.aws.
amazon.com/sagemaker/latest/dg/sagemaker-projects-why.html
[26] S. Selvin, R. Vinayakumar, E. Gopalakrishnan, V. K. Menon, and K. Soman,
“Stock price prediction using lstm, rnn and cnn-sliding window model,” in 2017
international conference on advances in computing, communications and infor-
matics (icacci). IEEE, 2017, pp. 1643–1647.
[27] M. Roondiwala, H. Patel, and S. Varma, “Predicting stock prices using lstm,”
International Journal of Science and Research (IJSR), vol. 6, no. 4, pp. 1754–
1756, 2017.
[28] P. Ruf, M. Madan, C. Reich, and D. Ould-Abdeslam, “Demystifying mlops and
presenting a recipe for the selection of open-source tools,” Applied Sciences,
vol. 11, no. 19, p. 8861, 2021.
[29] S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan,
B. Nushi, and T. Zimmermann, “Software engineering for machine learning: A
case study,” in 2019 IEEE/ACM 41st International Conference on Software En-
gineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 2019, pp.
291–300.
[30] C. Renggli, L. Rimanic, N. M. Gürel, B. Karlaš, W. Wu, and C. Zhang, “A data
quality-driven view of mlops,” arXiv preprint arXiv:2102.07750, 2021.
[31] Y. Demchenko, “From devops to dataops: Cloud based software development
and deployment.”
[32] B. Karlaš, M. Interlandi, C. Renggli, W. Wu, C. Zhang, D. Mukunthu Iyap-
pan Babu, J. Edwards, C. Lauren, A. Xu, and M. Weimer, “Building continu-
ous integration services for machine learning,” in Proceedings of the 26th ACM
SIGKDD International Conference on Knowledge Discovery & Data Mining,
2020, pp. 2407–2415.
[33] I. Karamitsos, S. Albarhami, and C. Apostolopoulos, “Applying devops practices
of continuous automation for machine learning,” Information, vol. 11, no. 7, p.
363, 2020.
[34] D. Wang, Q. V. Liao, Y. Zhang, U. Khurana, H. Samulowitz, S. Park, M. Muller,
and L. Amini, “How much automation does a data scientist want?” arXiv preprint
arXiv:2101.03970, 2021.
68
[35] A. Posoldova, “Machine learning pipelines: From research to production,” IEEE
Potentials, vol. 39, no. 6, pp. 38–42, 2020.
[36] T. Granlund, A. Kopponen, V. Stirbu, L. Myllyaho, and T. Mikkonen, “Mlops
challenges in multi-organization setup: Experiences from two real-world cases,”
in 2021 IEEE/ACM 1st Workshop on AI Engineering-Software Engineering for
AI (WAIN). IEEE, 2021, pp. 82–88.
[37] D. Kreuzberger, N. Kühl, and S. Hirschl, “Machine learning operations (mlops):
Overview, definition, and architecture,” IEEE Access, 2023.
[38] G. Symeonidis, E. Nerantzis, A. Kazakis, and G. A. Papakostas, “Mlops-
definitions, tools and challenges,” in 2022 IEEE 12th Annual Computing and
Communication Workshop and Conference (CCWC). IEEE, 2022, pp. 0453–
0460.
[39] I. L. Markov, P. A. Apostolopoulos, M. Garrard, Y. Huang, T. Gupta, A. Li,
C. Cardoso, G. Han, R. Maghsoudian, N. Zhou et al., “Scalable end-to-end ml
platforms: from automl to self-serve,” arXiv preprint arXiv:2302.14139, 2023.
[40] A. Bodor, M. Hnida, and D. Najima, “Mlops: Overview of current state and
future directions,” in Innovations in Smart Cities Applications Volume 6: The
Proceedings of the 7th International Conference on Smart City Applications.
Springer, 2023, pp. 156–165.
[41] N. Muralidhar, S. Muthiah, P. Butler, M. Jain, Y. Yu, K. Burne, W. Li, D. Jones,
P. Arunachalam, H. S. McCormick et al., “Using antipatterns to avoid mlops
mistakes,” arXiv preprint arXiv:2107.00079, 2021.
[42] Y. Liu, Z. Ling, B. Huo, B. Wang, T. Chen, and E. Mouine, “Building a plat-
form for machine learning operations from open source frameworks,” IFAC-
PapersOnLine, vol. 53, no. 5, pp. 704–709, 2020.
[43] Dvc. [Online]. Available: https://dvc.org/
[44] Dvc repository. [Online]. Available: https://github.com/iterative/dvc
[45] Mlflow. [Online]. Available: https://mlflow.org/
[46] Mlflow repository. [Online]. Available: https://github.com/mlflow/mlflow
[47] Neptune. [Online]. Available: https://mlops.neptune.ai/
[48] Neptune repository. [Online]. Available: https://github.com/neptune-ai
[49] Pachyderm. [Online]. Available: https://www.pachyderm.com/
69
[50] Pachyderm repository. [Online]. Available: https://github.com/pachyderm/
pachyderm
[51] Optuna. [Online]. Available: https://optuna.org/
[52] Optuna repository. [Online]. Available: https://github.com/optuna/optuna
[53] Sigopt. [Online]. Available: https://sigopt.com/
[54] Sigopt repository. [Online]. Available: https://github.com/sigopt/sigopt-server
[55] Kubeflow. [Online]. Available: https://www.kubeflow.org/
[56] Kubeflow repository. [Online]. Available: https://github.com/kubeflow/kubeflow
[57] Polyaxon. [Online]. Available: https://polyaxon.com/
[58] Polyaxon repository. [Online]. Available: https://github.com/polyaxon/polyaxon
[59] Airflow. [Online]. Available: https://airflow.apache.org/
[60] Airflow repository. [Online]. Available: https://github.com/apache/airflow
[61] Bentoml. [Online]. Available: https://www.bentoml.com/
[62] Bentoml repository. [Online]. Available: https://github.com/bentoml/BentoML
[63] Cortex. [Online]. Available: https://www.cortex.io/
[64] Cortex repository. [Online]. Available: https://github.com/cortexlabs/cortex
[65] Seldon. [Online]. Available: https://www.seldon.io/
[66] Seldon repository. [Online]. Available: https://github.com/SeldonIO/seldon-core
[67] Fiddler. [Online]. Available: https://www.telerik.com/fiddler
[68] Fiddler repository. [Online]. Available: https://github.com/telerik/
fiddler-everywhere-docs
[69] Hydrosphere. [Online]. Available: https://hydrosphere.io/
[70] Hydrosphere repository. [Online]. Available: https://github.com/
Hydrospheredata/hydro-serving
[71] Evidently. [Online]. Available: https://www.evidentlyai.com/
[72] Evidently repository. [Online]. Available: https://github.com/evidentlyai/
evidently
70
[73] Github action. [Online]. Available: https://github.com/features/actions
[74] Amazon sagemaker. [Online]. Available: https://aws.amazon.com/sagemaker/
[75] Mlops on vertex ai. [Online]. Available: https://cloud.google.com/vertex-ai/
docs/start/introduction-mlops
[76] Azure machine learning. [Online]. Available: https://azure.microsoft.com/en-us/
products/machine-learning
71