Description
Book SynopsisUnderstand how neural networks work and learn how to implement them using TensorFlow 2.0 and Keras. This new edition focuses on the fundamental concepts and at the same time on practical aspects of implementing neural networks and deep learning for your research projects.
This book is designed so that you can focus on the parts you are interested in. You will explore topics as regularization, optimizers, optimization, metric analysis, and hyper-parameter tuning. In addition, you will learn the fundamentals ideas behind autoencoders and generative adversarial networks.
All the code presented in the book will be available in the form of Jupyter notebooks which would allow you to try out all examples and extend them in interesting ways. A companion online book is available with the complete code for all examples discussed in the book and additional material more related to TensorFlow and Keras. All the code will be available in Jupyter notebook format and can be opened
Table of Contents
Chapter 1 : Optimization and neural networks
Subtopics: How to read the book Introduction to the book
Chapter 2: Hands-on with One Single NeuronSubtopics: Overview of optimization A definition of learning Constrained vs. unconstrained optimization Absolute and local minima Optimization algorithms with focus on Gradient Descent Variations of Gradient Descent (mini-batch and stochastic) How to choose the right mini-batch size
Chapter 3: Feed Forward Neural NetworksSubtopics: A short introduction to matrix algebra Activation functions (identity, sigmoid, tanh, swish, etc.) Implementation of one neuron in Keras Linear regression with one neuron Logistic regression with one neuron
Chapter 4: RegularizationSubtopics: Matrix formalism Softmax activation function Overfitting and bias-variance discussion How to implement a fully conneted network with Keras Multi-class classification with the Zalando dataset in Keras Gradient descent variation in practice with a real dataset Weight initialization How to compare the complexity of neural networks How to estimate memory used by neural networks in Keras
Chapter 5: Advanced OptimizersSubtopics: An introduction to regularization l_p norm l_2 regularization Weight decay when using regularization Dropout Early Stopping
Chapter 6Chapter Title: Hyper-Parameter tuningSubtopics: Exponentially weighted averages Momentum RMSProp Adam Comparison of optimizers
Chapter 7Chapter Title: Convolutional Neural NetworksSubtopics: Introduction to Hyper-parameter tuning Black box optimization Grid Search Random Search Coarse to fine optimization Sampling on logarithmic scale Bayesian optimisation
Chapter 8Chapter Title: Brief Introduction to Recurrent Neural NetworksSubtopics: Theory of convolution Pooling and padding Building blocks of a CNN Implementation of a CNN with Keras Introduction to recurrent neural networks Implementation of a RNN with Keras
Chapter 9: AutoencodersSubtopics: Feed Forward Autoencoders Loss function in autoencoders Reconstruction error Application of autoencoders: dimensionality reduction Application of autoencoders: Classification with latent features Curse of dimensionality Denoising autoencoders Autoencoders with CNN
Chapter 10: Metric AnalysisSubtopics: Human level performance and Bayes error Bias Metric analysis diagram Training set overfitting How to split your dataset Unbalanced dataset: what can happen K-fold cross validation Manual metric analysis: an example
Chapter 11 Chapter Title: General Adversarial Networks (GANs)Subtopics: Introduction to GANs The building blocks of GANs An example of implementation of GANs in Keras
APPENDIX 1: Introduction to KerasSubtopics: Sequential model Keras Layers Functional APIs Specifying loss functions Putting all together and training a model Callback functions Save and load models
APPENDIX 2: Customizing KerasSubtopics: Custom callback functions Custom training loops Custom loss functions
APPENDIX 3: Symbols and Abbreviations
