Gradient descent is an optimization algorithm that follows the negative gradient of an objective function in order to locate the minimum of the function. A problem with gradient descent is t... Read more

Weight initialization is an important design choice when developing deep learning neural network models. Historically, weight initialization involved using small random numbers, although ove... Read more

Last Updated on February 1, 2021 There is a lot of confusion for beginners around what algorithm is used to train deep learning neural network models. It is common to hear neural networks le... Read more

Optimization refers to finding the set of inputs to an objective function that results in the maximum or minimum output from the objective function. It is common to describe optimization pro... Read more

Developing a neural network predictive model for a new dataset can be challenging. One approach is to first inspect the dataset and develop ideas for what models might work, then explore the... Read more

The Nelder-Mead optimization algorithm is a widely used approach for non-differentiable objective functions. As such, it is generally referred to as a pattern search algorithm and is used as... Read more

Recommender systems may be the most common type of predictive model that the average person may encounter. They provide the basis for recommendations on services such as Amazon, Spotify, and... Read more

Regression refers to predictive modeling problems that involve predicting a numeric value. It is different from classification that involves predicting a class label. Unlike classification,... Read more

Activation functions are a critical part of the design of a neural network. The choice of activation function in the hidden layer will control how well the network model learns the training... Read more

Function optimization involves finding the input that results in the optimal value from an objective function. Optimization algorithms navigate the search space of input variables in order t... Read more