Coles has been pioneering the use of AI technology with a visionary approach for nearly a decade. Its AI journey has matured into a suite of AI models that drive day-to-day operations.
This AI-backed operational efficiency helps Coles predict the flow of 20,000 stock-keeping units (SKUs) to 850 stores nationwide with accuracy, harnessing insights from over 2,000 diverse data sets to make 1.6 billion predictions each day.
AI is also helping Coles deliver more personalised shopping experiences. Over four million FlyBuys customers receive bespoke weekly product recommendations curated by an AI model that understands preferences in the context of broader customer patterns.
In partnership with Microsoft, Coles is now shifting from initiating AI solutions to scaling them across its operations.
“Win-win solutions are those where we are helping our team members and our customers at the same time. Our technological investments into operational efficiency have translated into real, tangible benefits for our shoppers,” Coles Group general manager of data and intelligence, Silvio Giorgio said.
Coles has leveraged Microsoft Azure HCI alongside NVIDIA’s A16 GPUs to run large-scale AI workloads. One example is the use of Computer Vision during checkout, particularly in the selection and scanning of fresh produce. The traditionally manual task of scrolling through multiple options to identify and select produce has been simplified. Computer Vision systems can accurately scan items – even through plastic bags – identifying them with a high level of accuracy.
Coles has also identified an opportunity to improve queue monitoring at deli counters as customers are likely to abandon their purchase if service is not available within 20 to 40 seconds. In response, Coles has introduced Computer Vision technology to monitor deli counters, which alerts team members when a customer approaches.
Coles developed an edge computing platform it calls the ‘Intelligent Edge Backbone’ (IEB), using Microsoft Azure Stack HCI, Azure AI and ML – a global-first in retail to be deployed this way and at this scale.
Coles operates over 1,800 sites, each representing a node at the edge of a complex network. It required a platform that could support significant AI workloads of large-format supermarkets. Prior solutions focused on isolated task-specific or site-specific use cases, but Coles’ needed a unified platform that could centralise data and provide clarity and consistency across its entire operation.
The IEB serves as this central plane, enabling the collection and management of data from AI models across Coles’ network. This allows real-time operations such as queue monitoring, assisting customers through fresh produce recognition at scales, and enhancing service at point-of-sale systems. By integrating event detection with responsive notifications to team member devices, checkout systems, or in-store speaker announcements, the IEB provides a seamless flow of information.
“The IEB is effectively our central nervous system, connecting the technological infrastructure we have in stores to a central plane that manages events, triggers alerts and recommends actions. It also collects a wealth of data from which we can extrapolate insights that each store can action in customised ways that work best for them,” Coles Group head of technology innovation, Roslyn Mackay said.
The Coles finance and operations team has been on a digital insights journey for the past two years, looking at how to build a best-in-class commercial governance and support network for 850 stores.
“The whole intention behind the Operations team’s digital insights journey is to be able to give store teams the right information at the right time to make their jobs easier. We also want to understand the impact of and relationship between our operations and the customer experience,” Coles Group general manager of finance, operations and sustainability, Rich Walker said.
The team receives thousands of customer surveys per week, where each customer gives their shopping experience a score. The company also collects thousands of pieces of verbatim feedback every week. While the scores were easy to analyse, the team often found a disparity between the scores and actual customer sentiment.
“We’d get a nine out of 10 score, but then have the customer saying things like, ‘I couldn’t find a spot in the car park’ or ‘my favourite brand of cereal wasn’t stocked’. We realised that we were missing out on the richness of the insights in that verbatim feedback, simply because it’s not humanly possible to analyse the vast number of pieces of written feedback every week,” Walker added.
Coles turned to generative AI to supercharge the semantic analysis of this qualitative data. The team built a model using Azure OpenAI’s LLM capabilities, Microsoft Fabric and PowerBI to ingest, analyse and generate insights from the data. It reads the comments and summarises the top three things each store needs to work on, as well as categorise the comments into semantic themes, including product quality, availability, wait times, prices, team friendliness and more.
The operations teams can then overlay contextual information around time of day, store location and even rostering and compare this against the customer sentiment analysis to get a comprehensive understanding of how operational decisions and factors affect the customer experience.
“AI has given us the extraordinary ability to derive insight at a deeper level than ever before. And the excitement around the technology is electrifying. We are AI-ready and looking forward to what the technology can do for our people, but we are also moving with caution and a great sense of responsibility in how we use data and artificial intelligence,” Giorgio said.