I watched Andrej Karpathy’s 3rd video in the YouTube playlist: Neural Networks: Zero to Hero, and built a MLP (multi layer perceptron) from scratch to embed characters in a feature vector space and predict the next character based on the previous 3 characters. The original paper was published in 2003 by Bengio et al. titled “Neural Probabilistic Language Model”, 10 years before the classic Word2Vec paper was published in 2013. The core idea is similar: representing discrete entities in a continuous feature vector space. My code can be found on Github.
2. Build a bigram language model from scratch
I watched Andrej Karpathy’s 2nd video in the YouTube playlist: Neural Networks: Zero to Hero, and built a bigram language model from scratch in Python. Gradient descent of neural network training produces shockingly(?) similar results as the statistical analysis of bigram. My code can be found on Github.
1. Build backpropagation from scratch
This is the first time I implemented an automatic differentiation and the chain rule in Python, and built a backpropagation gradient descent algorithm from scratch. Thank you Andrej Karpathy so much for your amazing YouTube playlist: Neural Networks: Zero to Hero. My code can be found on Github.
Act like a leader
I spent the past few days binge reading the book “Act Like a Leader, Think Like a Leader” by Herminia Ibarra. I rarely read books on leadership, partly because I identify myself first as an IC (individual contributor) and a tech expert, not a leader, a role I consider as more talking than working, and partly because I always think it is a waste of time to think about leadership when I’m not even a leader. I’d rather spend more time pushing my code to production and delivering a solution. This book completely changes my naive perception of how leadership develops and motivates me to act now.
【职场分享】如何培养主人翁意识How to take ownership
【职场分享】如何让自己有话可说 / How to be a good verbal communicator at work
【职场分享】程序员如何出活 / How to deliver fast as an IC
从entry level升到senior的路径通常很清晰，从技术上讲，就是能deliver能出活。从一个小ticket开始做，到独立完成一个feature相关的若干ticket，再到独立完成若干个feature的时候，基本就是senior的水平了。其他方面的衡量标准（主要是soft skills）以后单独聊。