Words are no longer sufficient in delivering the search results users are looking for, particularly in relation to image search. In this piece, Mike Ranzinger, principal research engineer, Shutterstock, tells RetailTechNews how text and languages pose many challenges in describing visual details and providing the necessary context for optimal results. Machine-learning technology opens a new world of search innovation that can be applied by businesses and industries. More recently, we have seen new examples in the retail industry.

In the past, consumers had no choice but to use textual queries to search a large collection of images. This poses many challenges, due to disparate amounts of information between the query and the images. Sometimes it’s hard to describe exactly what you’re looking for, so users became accustomed to searching through multiple pages of results to find an image that’s similar to what they envisioned. Today, tools such as Reverse Image Search, created by Shutterstock, allow for an entirely new search experience. The tool provides an innovative alternative for customers, as compared to using keywords to search for images in the company’s collection. Users can simply upload a photo of their choosing and the algorithm will provide images that are similar in look and feel to the original using the image pixel data, overcoming the limitations of keyword search.

The retail industry has followed suit with such tools. ASOS has recently launched a visual search tool to aid inspiration and discovery for shoppers. The tool allows shoppers to upload a photo of their desired purchase to the app search bar and then showcases similar purchase options to them. Never before have consumers been able to capture products in passing moments, ranging from clothing, to household goods and accessories, and have artificial intelligence identify similar items. This type of tool can provoke ideas and inspiration, helping retailers both large and small to attract new business by being able to accurately suggest numerous examples that fit the customers’ visual brief. This further reinstates the idea that visual search tools have reformed the way we search for images and shop for items.  

Visual search tools, like those of Shutterstock and ASOS, recognise that it is possible to find inspiration in all places. Today’s consumers can spot a desirable outfit worn by their favourite celebrity in a magazine, photograph said image, and be provided with outfits to match – mirroring their style with ease. The same result would not be found using text to search, further highlighting the benefits of visual search tools.

There is also opportunity for these tools to expand into other retail sectors. This could include supermarkets, for those everyday occasions of inspiration, or when abroad and trying a local delicacy or brand that you have not come across before. A visual search tool would enable the consumer to photograph the product, upload it onto a specific site or app, and be provided with information about where it is stocked, or what similar alternatives they can buy. These advances would have a massive impact on the retail landscape.  

Shutterstock’s Composition Aware Search takes visual search one step further. Built on Shutterstock's next generation visual similarity model, this tool allows users to specify one or more keywords, or to search for copy space, and arrange them spatially on a canvas to reflect the specific layout of the image they are seeking. This patent-pending tool uses a combination of machine vision, natural language processing, and state-of-the-art information retrieval techniques to find strong matches against complex spatially aware search criteria. For example, a user can look for images of wine and cheese, where the wine is on the left and the cheese is on the right. By simply moving the placement in the search, users can see the requested changes reflected in the image results.

This technology enables designers to interact with an image search engine in a new way, helping bridge the gap between storyboards and delivering the visual campaigns they envision. The composed images not only help in preparation, but can also be used as part of the execution of the campaign. Within the retail industry, such a tool could enhance consumer experience once again. Perhaps you know exactly where you want your pocket and tear to be on your next pair of jeans, or you have a design layout in mind for a new T-shirt. The technology behind Composition Aware search could be applied to this use case, allowing your fashion vision to become a reality, and enabling retailers to make the most of consumers who may not have previously found exactly what they were looking for and, in turn, make a purchase based on their own design preference.

Similarly, retailers could also benefit from Shutterstock Showcase, which includes its Chrome extension (Reveal) allowing users to pick any image online and match it with a royalty-free image from their collection. Such a tool would be vital for start-up companies looking for viable images for their marketing campaigns, e.g. as the background for a product advertisement.  

Visual search technology is revolutionising the retail industry, with faster search results, more customer-driven products, and acting as tool for inspiration. This is an ever-growing sector and visual search will be at the forefront.