What is deep learning?
Deep learning is a subfield of machine learning inspired by the structure and function of the brain called Artificial Neuronal Network.
At the beginning you have a field of AI called machine learning. This is a discipline that studies how algorithms can learn by studying examples. The deep learning is a specific method to do machine learning.
A rough image would be the following:
Take a neural network. This is a typical 4 steps machine learning algorithm.
- Take n variables in input
- Process the algorithm according to the given values
- Output results
- Store these results
Now, plug networks after networks like many layers. Each algorithm outputs becoming inputs to the following to process more complex steps.
Welcome to deep learning!
Our french readers might watch this video as a good introduction by Science étonnante.
UX Design
Designing for the individual has always been a sweet goal for the perfect experience. After all, this is all about “user experience” and not “users experience”. However, most of our process are about targeted groups.
The inevitable marriage with machine learning is a great opportunity for the user. When the data exists, it is most of the time specific to a person. The location area, the social graph, the history, all personal behaviors and tastes. Algorithms measure and store what each specific user is doing for future reference to predictively improve the next user interaction.
UX design is a business discipline in most cases. We are in the middle of business goals and human needs. Measurement tools that give us the ability to see what the user is doing help us to make more informed choices.
Designers have enough of a job to create experiences for groups. We can’t fully focus on individuals. With the help of big data and deep learning, designers can start to think about deeper and more personal immersions of individuals. But how?
Today, designers have to deal with complex and sometimes huge datasets. We design for the experience in a highly conditional states and for the flow. To get around this problem, we tend to reduce the number of variables to create clusters of datas which we can work with at a human level.
Now let’s take a machine trained to deal with a set of variables for each user. Designers would save time and energy on finding clusters and testing concepts. We would work at a higher detailed level for the individual experience. The machine help us to create an experience totally unique and individual.
In another world, AI increases the workload but designers can use it to help us make sense of it at the same time.
Last week, we talked about chatbots. A pure one to one experience. These are systems trained to extract commands and suggest data based on the use data that looks like natural language to us. This is another field to improve the user experience. We need systems that interact like humans but think like machines. Deep learning is a way for machines to learn how to “behave like humans”.
How about the designer as a human being? Will we see the end of our discipline replaced by data scientists?
Well, UX design is already a matter of science in many cases. On the other hand, you can’t objectively measure an entire experience. No design is perfect after all. We design for humans that are subjects to evolution. Therefore, there will always be something to improve. “Data informed” design make all the difference in a product but in the end, some decisions come right from the guts or from a point of view. I think design and machine learning are in a good relationship in best cases.
Development
In order to play with AI and deep learning, we have to design neural networks. Then complex ones for deep learning. Or, we can use frameworks!
A new JavaScript library runs Google’s TensorFlow from the browser with GPU acceleration. TensorFire is developed by a team of MIT students. It runs TensorFlow-style machine learning models.
This project is another step into making machine learning accessible to a wide public using web technologies like WebGL.
With this framework we can deploy a trained model directly into the browser and serve predictions locally from it. The downside is that everything is processed on client side which requires client resources of course but think about it as a network. How much would we save if each individual processes his own little datas and sends the result only to a server?
Source : The neuronal network zoo by The Asimov Institute
Social Media Management / Marketing
Interaction. That’s what we, marketers, want and need. Interacting with people is winning people’s interests or curiosity… at least people’s attention.
Deep learning is used to unlock insight from unstructured data, such as image and video analytics, speech analytics, facial recognition, and text analytics.
We actually use deep learning without even knowing it! Does Siri help you on a daily basis? Do you actually use the facial recognition in order to classify your photos? Have you ever use an instant translator via the camera of your phone?
AI uses deep learning for customer analytics, giving brands and marketers the ability to get the right message to the right customer at the right time. Beyond segmentation, deep learning is able to extract the topics discussed and shared by a given target. It deduces themes and formats of content to be created. This also applies to the elements of the advertisements (images, colors, expressions to be used). The data collected following the behavior of a group of Internet users that make it possible to predict what content should be proposed to it.
Have you realised that your facebook news feed is pretty accurate those days? The more you use it, the more the content become appropriate to what you want to read or would like the most.
This does actually use deep learning through your habits on the social media. It learns from your acts, it saves the pattern and then reajusts the top news in order to make you react, by satisfying you. It is a virtuous circle. WONDERFUL! It does know you better than anyone else… (Please, don’t panic because it is evolution)
Source : Evolution of marketing trends and practices since 1990s by Toopixel
Deep Learning has changed the way Search Engine Optimization (SEO) works too. It doesn’t use only tags or metadata anymore, visual contents take full advantage of it these days.
Bonus: Interview through the most searched questions on Google 😉
Photo
Deep photo style transfer
We already know the app Prisma for transforming photos and videos into visuals using painters styles.
Photo by Sweet Koala modified with Prisma
Well, four researchers from Adobe and the Cornell University have teamed up to develop an algorithm that can apply the style of one photo to another by adding a deep learning layer to an existing method of style transfer. The process goes further than Prisma by using another photo as reference.
To get this result, the team dove into a deep learning approach. The program analyses colors, quality, light and more to apply these characteristics to the original photo by preserving its details.
The bets are open to know when this feature will appear in Lightroom!
EyeEm
AI is now able to understand aesthetics with deep learning. EyeEm is a community of more than 15 million photographers. This enormous source of datas allowed the company to feed their deep-learning-based automatic aesthetic curation to improve the search experience.
For those who are curious about the method, I invite you to read the paper of Appy Shaji, the Head of R&D at EyeEm.
Google, our friend Google. One of the biggest AI player today. Well, its AI is currently teaching itself photography. Google researchers have published a paper about “a deep-learning photographer capable of creating professional work” to integrate it into a creative process.
Like EyeEm, they found a pretty great source of thousands of high-ranked photographies but the comparison stops here because EyeEm learned aesthetics. Here, 500px.com has been crawled by the neural network to understand popular cropping and lighting effects.
After the learning process, they unleashed the machine on Google Street View. The program took snapshots of several locations with the cropping and light characteristics previously learned.
The aesthetic steps has been a “manual” work done by six professional photographers. None of them knew that these photos sets have been generated by a machine.
I wonder if Google heard about Street View Wayfarer who traveled around the world through Google Street View in 2016 and published his snapshots on Instagram.
A developer’s idea for the world of photography
While writing this article I can’t stop thinking about an association with these 3 groups to build a real time AI based composition analysis by Google + AI based aesthetic analysis by EyeEm + AI photo style transfer by Adobe & the Cornell University.
The program could theoretically detect a good light, a good framing and a good aesthetic to shot. Then finally improve this shoot with an external photo effect in real time. A new way to look at our world through a machine-based photographer point of view.
Video
It would be easy to picture what does photo style transfert and apply it to a video or a movie. After all, a video is only a succession of still frames. However it also bring a new parameter, Time.
Using AI to create an image will always make a unique version of the result by design. Prisma or any other deep-learning based program is no exception to that. You will always find a difference even if you repeat the process with the same source.
It won’t be a big deal on a single image. Now, applying this process to 30 frames per second will highlight these differences to the human eye. You will get things popping everywhere and a terrible flickering.
Consistency through time is key.
M.Ruder, A.Dosovitskiy and T.Brox proposed a method using constraints or weights on the parts of the pictures that were occluded, changed drastically or were inexistant. That technique allows a consistency of style through the video. Even when an actor is moving.
While being very effective, this method requires a lot of computing power and it can take hours to generate only a bunch of 1080p frames.
Even if it is not very effective yet, DeepArt.io implemented this method to allow you to get your video with a new style, and it looks really promising:
Future applications:
If like us at Sweet Koala you have been creating artwork digitally, you know how hard it can be to get a certain style right for each pieces.
Getting a simple animation using simple shapes and flat shades and converting this into an artistically rich video would be both amazing and a bit scary.
from art piece to a doodle
At first we could be frighten by this technical prowess, seeing that if a computer can do this, it would be able to replace designers, directors and artists but it could also be a very useful tool.
Using neural network style transfert would allow artists and directors to test and iterate faster and potentially produce a new style of art.
Digital Nomad
Hilton Hotels experiment a robot concierge that employs AI to answer questions. The partnership with IBM’s Watson program and Wayblazer on “Connie” is designed to inform guests on local tourist attractions, dining recommendations and other hotel details.
This cognitive technology helps the customer experience on simple tasks while giving the staff more time to answer more complex requests.
However, keep an eye on the jacket of your concierge the next time you’re looking for a high quality service. The best ones are members of the international union of hotels’ concierges known as Les Clefs d’Or. They wear these “gold keys” in the best hotels of the planet.
You will need a private concierge if you want to benefit from their high class services without being a hotel client. We, as digital nomads, have been lucky enough to meet one. That’s why we naturally recommend to visit the discrete conciergerie CCAZUR founded by a former french Clefs d’Or.