Sciling is taking part in developing an open source project to make life easier with ML models

Sciling is taking part in developing an open source project to make life easier with ML models

In order to see the project’s implications and relevance, let us place ourselves in context to understand a little just how complex it is for any developer to handle the roll-out of a machine learning model using the tools available until now.

Machine learning models

We live in times of information, which inevitably floods us with a huge amount of data circulating non-stop throughout the digital universe. Processing and interpreting all of that information would be almost impossible without the help of artificial intelligence, specifically machine learning (ML) models.

Basically, we could say that every machine learning process is a series of operations designed to run in order to create a model that is nothing more than a mathematical representation of a process in the real world. These models are fed data to find patterns and make predictions much more effectively than a human being.

The use of machine learning in software systems and applications is a revolution, since a machine learning model can solve complex problems far more quickly and efficiently than traditional methods. In fact, entire families of ML models have been created for different uses, and many more are on the way.

For example, one kind of ML model is the linear model, which uses numerical data patterns to predict relationships between variables to make financial projections. Another kind is the graphic model, which can use diagrams to express the probability of whether or not a consumer will choose to buy a product, for example. This is extremely useful for industries that use an online catalogue to sell, since they can predict their target customers’ behaviour.

Putting a machine learning model into production is not easy

Firstly, having a machine learning model ready to comfortably infer or classify data in a Jupyter notebook is not the same as having to roll out that model into production for it to generate true value. While it is true that the DevOps department usually takes care of rolling out a model, it is often still the developer’s responsibility to do so.

Rolling out a machine learning model means training it, putting into the real world and making it available for different kinds of software and applications in order to get the most out of machine learning’s potential. Hence, the model can feed on specific data in order to make predictions or find patterns, while it can also be used to introduce data in other applications.

In other words, in order to get the most out of ML models’ potential, it is essential to roll them out or put them into production available for other applications that may use them. In this sense, the truth is that closing the gap between building models and putting them into production continues to be a real challenge.

That is why it is worth mentioning the results from the report by Algorithmia 2020, “State of Enterprise Machine Learning”, in which a survey was carried out among 750 people, including technology company executives and ML professionals. The following results speak for themselves:

• 50% of those surveyed stated that their company takes between a week and three months to implement a machine learning model.

• 18% of those surveyed said that their company takes from three months to a year to manage to roll out a machine learning model.

• The main challenges as regards implementing a model for “machine learning involve keeping track of the different versions, scaling, reproducibility of the model and aligning all the stakeholders involved”.

KFServing and Modzy: tools to help roll out ML models

Two main tools to roll out production models are KFServing and the Modzy platform. With KFServing, it is possible to create a sroll image of a model using only its data file. This means that after training the model, all you need to do is to define the Kubernetes resourse that KFServing uses to roll it out and develop the entire system of inference, prediction, etc.

Once the ML model has been rolled out with KFServing, the system will carry out the rest of the process, setting up an API to make predictions, consulting the model’s state of health and verifying the state of the model in terms of scalability, etc. Another service of this kind is the one provided by the company Modzy. In this platform, it is only necessary to upload a trained model and the system will take care of automating the entire roll-out.

Standardisation of roll-outs for ML models in production

One of the reasons why it is so difficult to streamline the roll-out process for a machine learning model is that until now there has been no standard for doing so. That means there has been no service to take care of making a model compatible with the different roll-out tools available—until now.

But now the company Modzy has got in touch with Sciling to collaborate and create a service capable of standardising roll-outs, making machine learning models compatible with different roll-out tools.

Hence, it is only necessary to roll-out the service in a Kubernetes cluster and call the API using a Python library to create a Docker image that can now be scrolled down on the Modzy platform and in KFServing.

This project is published as open source, making it possible for ML professionals and developers to contribute to it. All of the documentation about this initiative and its progress are open to the community and can be seen at https://chassis.ml. With all of this, Sciling is once again actively contributing to the world of ML, making a commitment to open source projects to make developers’ lives easier.