On the maturity of LLMOps services computing: An industrial study

The way in which data-driven services are rendered and operated at industrial scale often assumes the connotation of LLMOps, or large-language model (LLM) operations, specifically, service operations stemming from data-driven, LLM-instrumented business processes. At the same time, a general understanding over the principles of DevOps services computing is still lacking. To gain such a generalisable understanding, In this study we perform mixed-method research using in-depth interviews in the largest service provider—a logistics service provider industry also active around the EU—of the Netherlands combined with an online survey to assess the relevance, challenges, and best practices of LLMOps services computing in industry. Our findings show that, on the one hand the literature has provided so far valuable insights into many practical lessons already applied in action, but, on the other hand, not all such practicalities have equal application to various services computing scenarios in the same way. In conclusion the study shows that the assessment of LLMOps in industry is considerable but its level of maturity—intended as the established way to employ its benefits while minimising the risks connected to its use—is still very low. We conclude recommending a careful evaluation of the principles and goals emerging from our study.