Deep learning may be one of the most overhyped of modern technologies, but there is a good chance that it will one day become the secret sauce in many different business processes". This presentation is a brief history of neural nets and deep learning trying to explain why and how did we get there. We will go through new developments in deep learning covering basic motivations, ideas, models and optimization, identifying challenges and opportunities. It will focus on issues related with large scale learning that is: high dimensional features, large variety of visual classes, and large number of examples. Some recent advances in deep automatic architecture design and adversarial learning (GAN) will be also discussed.