3. Build multi layer perceptron for feature vector from scratch

I watched Andrej Karpathy’s 3rd video in the YouTube playlist: Neural Networks: Zero to Hero, and built a MLP (multi layer perceptron) from scratch to embed characters in a feature vector space and predict the next character based on the previous 3 characters. The original paper was published in 2003 by Bengio et al. titled “Neural Probabilistic Language Model”, 10 years before the classic Word2Vec paper was published in 2013. The core idea is similar: representing discrete entities in a continuous feature vector space. My code can be found on Github.

2. Build a bigram language model from scratch

I watched Andrej Karpathy’s 2nd video in the YouTube playlist: Neural Networks: Zero to Hero, and built a bigram language model from scratch in Python. Gradient descent of neural network training produces shockingly(?) similar results as the statistical analysis of bigram. My code can be found on Github.

1. Build backpropagation from scratch

This is the first time I implemented an automatic differentiation and the chain rule in Python, and built a backpropagation gradient descent algorithm from scratch. Thank you Andrej Karpathy so much for your amazing YouTube playlist: Neural Networks: Zero to Hero. My code can be found on Github.

Act like a leader

 

I spent the past few days binge reading the book “Act Like a Leader, Think Like a Leader” by Herminia Ibarra. I rarely read books on leadership, partly because I identify myself first as an IC (individual contributor) and a tech expert, not a leader, a role I consider as more talking than working, and partly because I always think it is a waste of time to think about leadership when I’m not even a leader. I’d rather spend more time pushing my code to production and delivering a solution. This book completely changes my naive perception of how leadership develops and motivates me to act now.

(more…)

【职场分享】如何让自己有话可说 / How to be a good verbal communicator at work

工作之后,我渐渐发现自己一个人干活的时间越来越少,而跟人说话的时间越来越多。跟人说话,包括组里的工作会议,比如daily standup、spring planning、pairing、demo,跟老板和组里同事的日常1on1,跟越级老板的1on1,跟其他相关stakeholder的开会扯皮,茶水间的闲聊,等等。一开始我很不适应,因为我更习惯asynchronous的方式,有足够的时间思考说什么,琢磨一句话要怎么表达。我以前觉得,有什么话不能在slack上说,不能发一封邮件,不能在google doc上留言呢。但现在的我:synchronous真香,面对面才是王道。

(more…)

【职场分享】程序员如何出活 / How to deliver fast as an IC

从entry level升到senior的路径通常很清晰,从技术上讲,就是能deliver能出活。从一个小ticket开始做,到独立完成一个feature相关的若干ticket,再到独立完成若干个feature的时候,基本就是senior的水平了。其他方面的衡量标准(主要是soft skills)以后单独聊。

(more…)

【职场分享】新人如何打基础 / How to lay the foundation for a new job

在科技公司工作四年半,我从一个新人成长为一个资深工程师,一路上有很多贵人相助,我自己也带了不少新人。有一些工作的感想,会在接下来的几篇博客里跟大家分享一下。下面是第一篇,新人如何打基础。

(more…)