According to Gartner, deep learning is among the top 10 technology trends that will be strategic for most organizations in 2016. In other words, this is the technology trend with the potential for significant impact on an organization's long-term plans, programs, and initiatives. Gartner highlights the factors denoting the significant impact to include a high disruption potential for business, end-users, or IT, the need for major investment, as well as the risk of being late to adopt. While this trend is evolving quickly, Gartner suggests that organizations must assess how they can apply these technologies to gain a competitive advantage.
To create a better understanding of the potential this technology has, let’s review some successful use cases of deep learning applied in research and business.
Deep learning itself is an extremely popular area in machine learning, artificial intelligence, and data mining that creates multiple new opportunities in data-related research areas, such as signal processing, pattern recognition as well as natural language processing.
Typically, when working with languages, we are used to relying on dictionaries to get a definition of the particular word, as well as its grammatical features or the context of its use. Those dictionaries basically contain the encoding of human reasoning about words: the semantics, the synonymy, as well as the other grammatical characteristics. All this information has been gathered and arranged by people over the years.
With the capabilities of data science and artificial intelligence, the way words are processed and interpreted can be changed dramatically. It turns out we can define the meaning of the word based on its position in the text without the need to use a dictionary.
According to Richard Socher, we can learn how to represent words with numbers, getting all the required information directly from the text with no need to leverage any human knowledge at all. The resulting vectors of these numbers are representing the meaning of the word, while similar words are represented with similar vectors.
Moreover, these vectors represent the relationships between words as shown in the picture below.
These representations were later used to improve the accuracy of text classification, matching sentences, automatic categorization, question answering and other text-mining tasks.
One of the most recent successful appliances of deep learning for image recognition came from Large Scale Visual Recognition Challenge, when Alex Krizhevsky applied convolutional neural networks to organize images from ImageNet, a dataset containing 1.2 million pictures, into 1,000 different classes. In 2012, Krizhevsky’s network, AlexNet, achieved a top-5 test error rate of 15.3%, outperforming traditional computer vision solutions with more than 10% accuracy.
The experience of Alex Krizhevsky changed the landscape of the data science and artificial intelligence field from the perspective of the research and business application. In 2012, AlexNet was the only deep learning model at ILSVRC (ImageNet Large Scale Visual Recognition Competition). Two years later, in 2014, there were no conventional computer vision solutions among the winners. And today, deep learning is, de-facto, a standard solution for computer vision tasks.
Being early adopters by nature, we are continuously researching the capabilities of data-driven analytics and machine learning methods. One of our recent experiments featured deep learning neural networks applied for automatic colorization of old photos and videos. We have used residual autoencoder network suggested by Ryan Dahl and extended it to colorize a full-sized image.
The model for automatic colorization has been trained on the ImageNet dataset using TensorFlow library with GPU.
In the images below, you can see the “before” and “after” pictures and the videos we have coloured. The photos display the city of Lviv of the beginning of the 20th century and an episode from the 1939 movie Włóczęgi (The Vagabonds).
After its prominent success in the academic area, deep learning has been finally introduced into the business world. The industry giants, including Microsoft and Facebook, started to successfully use the advantages of this approach. The technology began to spread among verticals, bringing in advanced capabilities for speech and image recognition, stock trade, medical diagnostics and other.
Businesses are well aware that in the modern world data is the key to well-informed, sound decisions and reliable predictions. Companies have been continuously looking for better ways of leveraging the data they collect in order to make better decisions. Deep learning today not only offers them better accuracy of data processing - when compared to conventional models - but also solutions to the problems that were previously unsolvable. Here are some recent cases of how deep learning provided a significant business advantage.
In 2015, MOZ revealed their deep learning model for customer churn prediction. The approach was based on the recurrent neural networks, a powerful model to work with sequential data. As customer behaviour is sequential in nature, this approach fits in perfectly.
The actions a customer is about to perform are caused by a vast variety of factors from the past. In relation to the customer, these factors could be external ( changing economic circumstances, rules and regulations, market demand), internal ( change in income, urgent needs, changes in tastes) as well as product related (previous experience with the product, the functionality required, the quality of the support). In combination, all this information provides much more insight compared to any single source.
The MOZ model uses an event aggregation approach to represent the customer lifecycle for a particular user along with the demographic and other factors in a single vector. This vector represents the user’s past behavior. Using this data, the recurrent neural network predicts the future behavior of this customer at every stage of the buyer's journey, based on the person’s activity in the past. The information that MOZ was interested in was associated with the possibility for the customer to shift from the free trial mode towards the purchase of the product or its withdrawal.
The model was tested on two types of churn: voluntary and involuntary. And in both cases, there was an obvious improvement in the prediction accuracy compared to conventional approaches.
Another approach was suggested by Microsoft Research in their paper A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems. The aim of the approach was to provide better recommendations to customers looking for Windows Apps, taking information from the search queries and browsing history as a base. A wide variety of factors was taken into account, including the news the person was searching for or the movies and TV shows that he or she was interested in.
The new, deep learning architecture that Microsoft has proposed was called “multi-view deep neural network”. This model analyzes user behavior in different domains (namely search queries, news read and movies/TV shows liked) and transforms this information into a single vision. This vision can be used later for making personalized suggestions about news, movies - literally anything.
The approach has outperformed the conventional recommendation techniques on Microsoft’s internal dataset as well as on several public benchmarks. The results are quite impressive from the technical standpoint. The system was able to link automatically the behavior of a user in one type of environment (visiting a particular website) with the actions performed in an entirely different environment (searching for news).
For example, the fact that the user has visited some websites for young parents triggers the recommendation on baby clothes. While the subject seems quite straightforward, this example actually shows how learning jumps domains – an inference which is apparently clear to a human is not so obvious to a machine.
Even though deep learning still has a long way to go, the potential opportunities and benefits it can bring look quite promising. For now, no one can predict for sure how this technology will evolve, but it is very likely to quickly spread among industries, shaping business opportunities through the upcoming decades.
The breadth of knowledge and understanding that ELEKS has within its walls allows us to leverage that expertise to make superior deliverables for our customers. When you work with ELEKS, you are working with the top 1% of the aptitude and engineering excellence of the whole country.
Right from the start, we really liked ELEKS’ commitment and engagement. They came to us with their best people to try to understand our context, our business idea, and developed the first prototype with us. They were very professional and very customer oriented. I think, without ELEKS it probably would not have been possible to have such a successful product in such a short period of time.
ELEKS has been involved in the development of a number of our consumer-facing websites and mobile applications that allow our customers to easily track their shipments, get the information they need as well as stay in touch with us. We’ve appreciated the level of ELEKS’ expertise, responsiveness and attention to details.