We improve customers’ experience in the restaurant sector thanks to the latest in pose estimation technology

Pose

Phil Stevens¹ is a born entrepreneur. He founded his own company before he was even 20 years old, which meant he had to set aside his university studies in Computational Science for years. It was around 1991 when the first web page appeared and Phil glimpsed the Internet’s potential, deciding to create PS Technologies, a company specializing in Internet services that he would sell 20 years later so as to commit himself to other projects.

Since then, he has founded another three Internet companies and invested in 28 technology-based projects. His solid technological knowledge combined with his passion for finding and backing brilliant ideas leads him to continually scan the horizon in search of new projects. So it was that in 2020, when he discovered the latest advances in pose technology², he decided to create a startup that would apply such technology to the sphere of restaurants.

1. Real names protected due to a confidentiality agreement.
2. Pose estimation is a kind of technology based on computer vision that detects and analyzes human posture.

About the client:

Two of the biggest challenges facing the restaurant sector are, on the one hand, providing a consistent service that responds to customers’ expectations and, on the other, mapping their experience in order to improve the service and their satisfaction with it.

Ensuring a positive experience for the customer is directly related to an increase in sales. What’s more, it can even be the difference between the failure or success of a business in this sector. And without a doubt, waiting time is one of those things that can have a positive or negative impact on the impression customers get of a restaurant business.

“I realized that pose technology could be very useful for this sector. Most bars and restaurants have security cameras, so I decided to create a kind of SaaS software to take advantage of the images recorded with them to analyze the service provided,” says Phil. In addition to understanding customers’ waiting times, the application would let the user know the hourly traffic in order to optimize staff schedules, as well as other functions.

Knowing the risks behind such innovative projects, Phil decided to outsource a proof of concept from three different vendors, one of which was Sciling. “I’ve been fully committed to innovation for many years and I am well aware of the implications of working with techniques about which there is not yet much information because they’re so new. I preferred to choose to work in a preliminary stage with several providers rather than risk making a commitment to just one that might not turn out well later. The opportunity cost in that case would have been huge,” says Phil.

Strategic vision:

The outcome of the arduous research work carried out and the work on improvement and adaptation done by Sciling’s team of researchers is an Internet application based on images captured by security cameras that is capable of recognizing individual customers and analyzing what happens during their time on the premises. This is done by modeling their figure based on a skeleton rather than on their outline, which gives much better results.

“The closure of restaurants almost worldwide due to the pandemic has had a negative impact on the project’s timing, but the accuracy obtained by Sciling in this first stage is promising. The difference compared to the other providers is massive,” says Jonathan.

What will the next steps in the project be? To use the information obtained in this first stage to establish ratios that give a measure of the quality of the service, as well as to provide data to help adapt staff levels to the real needs and arrange space in the most efficient way. To sum up, to turn information into useful knowledge for decision-making, which is vital for any company.

And going a step further, the Sciling team is already thinking about new applications for this technology in other sectors. “What is truly interesting about this technology is its many applications, especially at a time like the one we are experiencing. With some adaptations, the models developed for this project could help detect maskless passengers on public transport, for example. Outside of the COVID-19 context, there is also a plethora of cases where it can be used: heat maps in shopping malls, virtual trainers, prevention of occupational risks on building sites and in warehouses, and many more,” explains Antonio Salas, Sciling’s Marketing Director.

Sciling’s researchers explored the technological state-of-the-art in pose estimation for weeks until they came across a kind of software that had won many competitions. After analyzing it in depth, they realized that it could be a good basis for solving the problem they were dealing with.

Taking the time to examine the advances made so far in the technologies they are working with has always worked out well for Sciling, making the rest of the project’s phases move much more swiftly on a more solid basis. “The technical challenge in this case was to adapt and improve a software that had been conceived for very different conditions from the ones we were facing,” says Vicent Alabau, Sciling’s Director of Operations.

In order to adapt and refine the best pose technology at the time, the development team added a new model for classifying activities. They also gathered new samples in order to improve the existing models. Furthermore, they built up a set of test data using videos from a bar that was collaborating with Phil on this project. This enabled them to optimize and adapt the model’s parameters to the specific case they were tackling. Lastly, they included new features in the award-winning software, which they used as a basis in order to improve the accuracy for the problem at hand.

All of this effort paid off, with Sciling’s metrics far exceeding those of the other vendors, which prompted Phil to choose them for the project’s next phases.

Technologies used