During logistic regression, in order to compute the optimal parameters in the model, we have to use an iterative numerical optimization approach (Newton method or Gradient descent method, instead of a simple analytical approach). Numerical optimization is a crucial mathematical concept in machine learning and function fitting, and it is deeply integrated in model training, regularization, support vector machine, neural network, and so on. In the next few posts, I will summarize key concepts and approaches in numerical optimization, and its application in machine learning.

#
Posts in the *Connect the Dots* category:

# Goal setting for function fitting (classification)

In the previous post, I have discussed loss function in regression. In this post, I will elaborate how we develop loss function in classification when the output is discrete, rather than continuous.

# Do you write production code as a data scientist?

In the past month, I posted this question to my friends, peers, online tech forum, and got responses from more than 30 data scientists in various industries and different academic background and career path. The responses show a wide spectrum of data scientists’ involvement in production, and reveal some shared concerns about career development among data scientists.

# Goal setting for function fitting (regression)

Whenever we see the word “optimization”, the first question to ask is “what is to be optimized?” Defining an optimization goal that is meaningful and approachable is the starting point in function fitting. In this post, I will discuss goal setting for function fitting in regression.

# Restrict function search space by assumptions

In the previous post, I discussed the function fitting view of supervised learning. It is theoretically impossible to find the best fitting function from an infinite search space. In this post, I will discuss how we can restrict the search space in function fitting with assumptions.

# Set up the supervised learning problem as function fitting

In this very first post of the Connect the Dots series, I set up the supervised learning problem from a function fitting perspective and discuss the objective of function fitting.

# Connect the Dots

Entering Year 2019, I plan to start a post series discussing what I have learned in statistics, machine learning, big data, computer science, and neuroscience (always!). I name this series “Connect the Dots”, as in the puzzle game “connect the dots“.