How Do Amazon Go Stores Work Using AI?
Computer Vision, Sensor Fusion, and Deep Learning All Play a Part
No one likes to wait in line at a grocery store. In fact, 43% of American consumers say that long lines affect where they choose to shop. Amazon Go’s checkout-free stores aim to remedy this issue by providing a seamless and convenient shopping experience.
You may have shopped in an Amazon Go store and wondered how it works, or you’ve heard about them and are curious. What you might not know is that Amazon Go stores rely on artificial intelligence (AI) systems in order to function effectively.
Keep reading to learn how Amazon Go stores work, as well as their many benefits.
What Is Amazon Go?
Amazon Go stores offer customers a checkout-free shopping experience using their “Just Walk Out” technology, which allows customers to grab items and go without following the traditional payment process. Instead, payments are charged directly to customers’ Amazon accounts without them having to pay in-store.
While a beta version of the service was first launched in Seattle in 2016, Amazon now has over 40 stores open worldwide in the US and UK, providing users with a seamless shopping experience. It’s estimated that each store generates a revenue of around $1.5 million per year.
Benefits of Amazon Go Stores
Amazon Go is an excellent example of artificial intelligence being used to increase the convenience of customers' daily life. In fact, Amazon Go stores come with several benefits. Here are just a few to consider
Convenience: Not having to go through checkouts means customers can “grab-and-go,” making their shopping experience much faster and avoiding checkout lines.
Seamless Shopping Experience: In Amazon Go stores, customers simply scan in using their Amazon account and are then able to shop without a traditional checkout process. Their purchases are charged directly to their Amazon accounts, along with their receipts, ensuring a seamless shopping experience.
Better Supply Data: Amazon Go stores offer an in-depth insight into supply data, including which products need restocking and which are most popular, meaning more should be available. This is thanks to customer data being instantaneously digitized the moment they pick up products.
More Personalized Advertising: In addition to using customer data to better understand store supply and demand, that same information can be used to personalize advertising. For example, if Amazon recognizes that a customer regularly purchases root beer in their stores, they can then advertise root beer to them when browsing the Amazon app.
Challenges Associated With Checkout-Free Stores
Of course, with a shop that relies almost entirely on digital data, AI-powered camera systems, online networks, and has few employees to monitor the store, there are bound to be some challenges. Here are just a few:
Data Privacy: Amazon Go relies on complex surveillance systems driven by AI and customer Amazon accounts; this means huge amounts of personal data are sent to Amazon. How this data is used is a concern for many consumers, especially in states that have limited data privacy laws.
Network Errors: If there is a network error that means store servers are unable to securely connect to the internet, the entire system fails because data is connected with customers’ online accounts. Therefore, data cannot be transmitted properly.
Social Training: Though many supermarkets now have self-checkout systems, checkout-free stores are a new concept, meaning that people will not feel confident using these systems. As a result, a level of social training is required for the continued success of the Amazon Go stores.
Theft: Monitoring a busy store requires a vast number of cameras to prevent theft, and even then, there is still a chance of shoplifting. For example, in one experiment with AI-driven surveillance cameras, participants were able to confuse cameras by acting weirdly.
Mistakes Happen: There are approximately four mistakes per Amazon Go store per day. This is not a huge amount, but if this technology were to be adopted by much larger stores, it could become noticeable.
Would you shop at a checkout-free store?
How Does Amazon’s ‘Just Walk Out’ System Work?
Amazon Go stores use the company’s “Just Walk Out” system, which uses several pieces of technology and follows a deep learning algorithm. Here’s how the system works, step by step.
Entry and Account Setup: Customers download the Amazon Go app on their smartphones and link it to their Amazon account. They enter the store by scanning a QR code in the app.
Sensors and Cameras: The store is equipped with an array of sensors and cameras that track the customers' movements and actions. These sensors are placed on shelves, on the ceiling, and throughout the store.
Item Recognition: As customers pick up items from the shelves, the sensors and cameras use computer vision, deep learning algorithms, and sensor fusion to identify the items and add them to the customer's virtual shopping cart.
Cart Updates: The virtual shopping cart is continuously updated as customers add or remove items from their physical carts. If an item is put back on the shelf, it is removed from the virtual cart.
Account Sync: The app on the customer's smartphone syncs with the sensors and cameras in real-time to ensure accurate tracking of the items in the cart.
Exit and Payment: When customers are done shopping, they simply leave the store without having to go through a checkout process. The "Just Walk Out" technology automatically charges the customer's Amazon account for the items they took with them.
Receipt and Feedback: Customers receive a digital receipt in the app, and they can provide feedback on their shopping experience.
Technology Used by Amazon Go
In the above outline of how the “Just Walk Out” system works, there were several terms used that are key to the functionality of Amazon Go. So, before we get into how Amazon Go uses AI, let’s break these technologies down.
Computer Vision: Computer vision is an AI field that focuses on teaching computers to understand and interpret visual information from images or videos. By developing algorithms, computers can recognize objects, detect patterns, and extract meaningful insights from visual data. In Amazon Go stores, this technology is used to determine and track which items customers pick up.
Sensor Fusion: Sensor fusion is the process of combining data from multiple sensors to gain a more accurate and comprehensive understanding of the environment. By merging information from different sensors, like surveillance cameras, it creates a robust perception system. This makes verifying which products customers have selected more accurate, as well as allowing Amazon Go store cameras to understand when customers put items back on the shelves.
Deep Learning: Deep learning is a subset of machine learning that utilizes artificial neural networks inspired by the human brain's structure and function. With multiple layers of interconnected nodes, deep learning models can autonomously learn complex patterns and hierarchical representations using vast amounts of data. It excels in tasks such as image recognition, speech processing, and natural language understanding.
Together this technology allows Amazon Go stores to accurately track customers' movements, monitor which items they select, and charge their Amazon accounts accordingly.
How Amazon Go Uses Artificial Intelligence?
Each of the above technologies utilizes artificial intelligence to some degree, but the real use of AI comes in tying the three processes seamlessly together. In fact, combined, these technologies work much like a self-driving car. They identify objects, track them and take appropriate actions.
Amazon Go uses artificial intelligence to process and analyze the data collected by computer vision, sensor fusion, and deep learning. Here’s how the Amazon Go algorithm works in three crucial steps:
Data Collection: Using their computer vision and sensor fusion-driven surveillance system, Amazon Go stores collect customer data, such as timestamps, item selections, and customer paths.
Data Processing: The collected data is sent to a centralized system where deep learning AI algorithms and machine learning techniques are applied. The data is processed and transformed into a format suitable for analysis.
Pattern Recognition: Deep learning AI is used to identify patterns and correlations within the data. This includes identifying customer preferences, popular product combinations, peak shopping hours, and other behavioral insights. By analyzing vast amounts of data, Amazon Go’s AI can uncover hidden patterns that human analysis alone may not easily identify.
Will We See More AI Powered Stores?
Since Amazon Go’s inception in 2016, it’s safe to say that the retail and shopping industry has not been revolutionized, despite their stores being a resounding success. This is likely due to consumer attitudes and trust rather than a failure of technology.
Interestingly, however, AI is finding its way into our stores for its application in advanced surveillance systems. In Japan, for instance, AI-driven surveillance systems are being used to curb the rise of shoplifting.
We have an excellent article on this: How AI Is Being Used to Stop Shoplifters.
So, as companies continue to experiment with AI in stores and attitudes towards the technology shift, it’s probable that we’ll see artificial intelligence more widely used within the shopping industry.
Thanks for reading.
If you enjoyed this article, please subscribe to receive email notifications whenever we post.
AI Business Report is brought to you by Californian development agency Idea Maker.