Primary tab navigation

Augmented reality makes shopping more personal

New mobile application from IBM Research helps both consumers and retailers

While in-store shopping accounts for 92 percent of retail volume1, consumers are expecting the same levels of personalization and customization that they do when they shop online; 58 percent of consumers want to get in-store product information and 19 percent of consumers are already browsing their mobile devices while in-store. Scientists at IBM Research - Haifa are looking to bring all the benefits of online shopping into traditional, brick-and-mortar stores and are creating a new augmented reality mobile shopping application (app) that will give in-store shoppers instant product details and promotions through their mobile devices.

Illustration showing how the augmented reality shopping app will automatically deliver personalized coupons and product details.

The new app from IBM Research will provide shoppers with a personalized shopping experience with immediate product comparisons and special offers as they move throughout the store. It captures images via the built-in video camera on a user's smart phone or tablet and uses advanced image processing technologies to quickly and accurately identify a product or row of items. Once the application recognizes the products, it will display information above the product images and rank them based on a number of criteria, such as price and nutritional value. It will also provide the shopper with any loyalty rewards or incentives that may apply and suggest complementary items based on what the customer has already viewed.

For example, a shopper looking for breakfast cereal could specify they want a brand low in sugar, highly rated by consumers and on sale. As the shopper pans the mobile device's video camera across a shelf of cereal boxes, the augmented reality shopping app will reveal which cereals meet the criteria and also provide a same-day coupon to entice the shopper to make a purchase. While the app will allow shoppers to be more informed about products, it will also help retailers to
shopping experience infographic better connect with their in-store customers. Using the personal information provided by their customers as they move throughout the store, marketers could receive a great deal of insight on the preferences of their shoppers, as well as which areas of the store see the most traffic. Using this information, retailers could better organize their store and adjust business practices accordingly. According to IBM's head of Retail Research, Sima Nadler, the differences between online shopping and in-store shopping will start to merge. "What we're seeing is the blurring of the physical and the virtual."

In the past, shoppers were classified by broad demographics, but soon they will be able to engage in an augmented reality shopping experience that is customized, and lets them take advantage of deals and special offers related to products that they likely need. By providing in-store shoppers with the same kind of personalized information that online shoppers receive, retailers can now harness Big Data in order to cater to each individual on a more personal level and transform marketing into a convenient and welcomed service for consumers.


1 Forrester Research: Understanding Online Shopper Behaviors, US 2011, May 17, 2011

Share this story

Explore this topic

Meet the researchers

  • Amnon Ribak

    Amnon Ribak

    Research Staff Member,
    IBM Research – Haifa

  • Asaf Tzadok

    Asaf Tzadok

    Manager, Image and
    Document Analytics Group,
    IBM Reserach – Haifa

  • Amir Geva

    Amir Geva

    Document Management Engineer,
    IBM Research – Haifa

  • Sima Nadler

    Sima Nadler

    IBM Worldwide Research Lead
    for Retail & Commerce,
    IBM Research – Haifa

  • Pavel Kisilev

    Pavel Kisilev

    Research Scientist:
    Image and Document Analytics,
    IBM Research – Haifa