In the previous post, I talked about using MapReduce and Spark for distributed model training. In this post, I will talk about parameter server and how it is used in distributed model training.
In the previous post, I introduced challenges in machine learning systems with big data and complex models. In this post, I will discuss distributed systems in the era of big data.
Let’s start with linear regression. Using established libraries such as scikit-learn, it is almost trivial to train a linear regression model. We can easily run the model training with a few hundred Megabytes of data on our laptop with a build-in CPU.
Now let’s think big.
In business management, product lifecycle is broken into 4 stages with the distinct pattern of sales over time: introduction, growth, mature, and decline. In the diagram below, I adapt the classic product lifecycle curve to show the engineering load over time in machine learning (ML): from model development to maintenance. Managing and coordinating different stages in ML lifecycle presents pressing challenges for ML practitioners.
The more I work on building end-to-end machine learning (ML) pipelines, the more I realize the importance of system design and infrastructure. ML shares many concerns with traditional software development, and poses new challenges to system design.
It’s been 3 months since I started my new position as a Machine Learning Engineer (MLE) at Spotify. What I like most about this position is that I get to participate in building an end-to-end pipeline, including ideation and experiment, data engineering, machine learning modeling, model serving, online A/B test, monitoring, and many more.
第一次在职跳槽总算完成了！在面试准备的过程中，我学到了不少工作之外的知识和技能，这里分享一下我的跳槽经验。下周入职新公司Spotify的Machine Learning Engineer职位。
This summer, I volunteered to be a mentor for a data science intern. It reminds me of my own internship 2 years ago, when I learned so much from my mentor (see my previous posts). Being a mentor not only allows me to view a summer internship from the other side of the table, but also presents new challenges and learning opportunities for myself. In this post, I will share some tips for first-time mentors from my experience as a first-time mentor.