In this information era, Recommendation System (RS) has become an indispensable part of our everyday life. Whether you are consciously aware of its presence or not, it serves you when you are, for example, listening music on Spotify, watching NetFlix movies, shopping on Amazon, etc.
So what is RS after all?
Imagine the following scenario: you want to get yourself a nice suit for a new job and you walk into a fancy looking fashion shop. A staff of the shop warmly greets you and asks for your needs. Well, a suit of course and you go further to tell the staff your preferred designs, colors and fabric. After a few seconds of thought, the staff leads you to a corner of the shop and show you a variety of suits that match your requests. After going through some of these options, you suddenly get inspired and make a few new requests to the staff. The conversation carries on. In the end, you finally decide to go for a medium gray, slim fit, formal suit. Through the conversation, you also told the staff that you are picking the suit for a new job and you want to leave a sharp impression to your new colleagues. The staff then start to ask if you might also consider accessories that can add extra values to the suit, for example, a blue tie. You can think of a RS simply as a super staff who can serve millions of customers all at once. Apparently, such a staff is not going to be a human-being.
An actual RS is an algorithm which can process static information of products such as textual descriptions and visual images, as well as dynamic information ranging from a particular customers’ preference to the global trends in the market, and can use these information to serve individual customers in a personalized way, just as what you expect from talking with a real knowledgeable staff in a shop. Super powerful as it sounds, it is a highly challenging problem which is currently under intensive research from both industry and academia.
One of the obvious questions that one might have is: how can a computer “understand“ what is a product, or better, how different products are related to each other?
After all, the only thing computer understands is 0’s and 1’s. Back to our story of picking a suit, it is relatively easy for an algorithm to recognize the customer’s choices of design, color and fabric: all one needs is to provide a complete list of possible choices and ask the customers to check the form. Simple solution you may think. It therefore should be reminded that these information (color, style, fabric) must be put in manually for every single product in order for the simple solution to work. This can quickly become a labor-intensive job, let alone the reliability of human inputs. But say we spent hours/days/months/years to create a perfectly organized inventory, the next question comes: how does a computer know that a tie is “related“ to a suit, at least more than a marshmallow is. It is far less obvious that how building an inventory may help solving the problem. In fact, it may even become difficult for a normal person to tell which accessories match the best a given suit style without consulting an expert.
The modern solution to the design of RS involves many of the fundamental ideas from Machine Learning and Artificial Intelligence. For example, recall that the first thing the shop staff did when the customer inquires about suits is to introduce him a handful of suits which match the customer’s taste. These choices may be called alternatives, and they could also be different brands of wine or shoes. One algorithmic way of finding alternatives is to use the so-called clustering techniques. In a nutshell, one finds a vector representative for each product (remember computer doesn’t recognize products, only 0’s and 1’s) in a usually large dimensional vector space, in such a way that similar products get mapped to nearby vectors. The following figure illustrates this idea where products are mapped to a 2-dimensional vector space and different types of products tend to loosely form clusters.
