In the previous chapter, I introduced three key steps for building a career in AI: Learning foundational technical skills, working in projects, and finding a job, all of which is supported by being part of a community. In the chapter, I’d like to dive more deeply into the first step: learning foundational skills.
More research papers have been published on AI than anyone can read in a lifetime. So, when learning, it’s critical to prioritize topic selection. I believe the most important topics for a technical career in machine learning are:
Foundational machine learning skills: For example, it’s important to understand models such as linear regression, logistic regression, neural networks, decision tree, clustering, and anomaly detection. Beyond specific models, it’s even more important to understand the core concepts behind how and why machine learning works, such as bias/variance, cost functions, regularization, optimization algorithms, and error analysis.
Deep learning: This has become such a large fraction of machine learning that it’s hard to excel in the field without some understanding of it! It’s valuable to know the basics of neural networks, practical skills for making them work ( such as hyperparameter turning), convolutional networks, sequence models, and transformers.
Math relevant to machine learning: Key areas include linear algebra (vectors, matrices, and various manipulations of them) as well as probability and statistics (including discrete and continuous probability, standard probability distributions, basic rules such as independence and Bayes’ rule, and hypothesis testing). In addition exploratory data analysis (EDA) — using visualizations and other methods to systematically explore a dataset — is an underrated skill. I’ve found EDA particularly useful in data-centric AI development, where analyzing errors and gaining insights can really help drive progress! Finally, a basic intuitive understanding of calculus will also help. The math needed to do machine learning well has been changing. For instance, although some tasks require calculus, improved automatic differentiation software makes it possible to invent and implement new neural network architectures without doing any calculus. This was almost impossible a decade ago.
Software development: While you can get a job and make huge contributions with only machine learning modeling skills, your job opportunities will increase if you can also write good software to implement complex AI systems. There skills include programming fundamentals, data structures ( especially those that relate to machine learning, such as data frames), algorithms ( including those related to databases and data manipulation), software design, familiarity with Python, and familiarity with key libraries such as TensorFlow or PyTorch, and scikit-learn.
This is a lot to learn!
Even after you master everything on this list, I hope you’ll keep learning and continue to deepen your technical knowledge. I’ve know many machine learning engineers who benefitted from deeper skills in an application area such as natural language processing or computer vision, or in a technology area such as probabilistic graphical models or building scalable software systems.
How do you gain there skills? There’s a lot of good content on the internet, and in theory, reading dozens of web pages could work. But when the goal is deep understanding, reading disjointed web pages is inefficient because they tend to repeat each other, use inconsistent terminology (which slows you down), vary in quality, and leave gaps. That’s why a good course — in which a body of material has been organized into a coherent and logical form — is often the most time-efficient way to master a meaningful body of knowledge. When you’ve absorbed the knowledge available in courses, you can switch over to research pagers and other resources.
Finally, no one can cram everything they need to know over a weekend or even a month. Everyone I know who’s great at machine learning is a lifelong learner. Given how quickly our field is changing, there’s little choice but to keep learning if you want to keep up.
How can you maintain a steady pace of learning for years? If you can cultivate the habit of learning a little bit every week, you can make significant progress with what feels like less effort.
The Best Way to Build a New Habit
One of my favorite books is BJ Fogg’s, Tiny Habits: The Small Changes That Change Everything. Fogg explains that the best way to build a new habit is to start small and succeed, rather than start too big and fail. For example, rather than trying to exercise for 30 minutes a day, he recommends aspiring to do just one push-up, and doing it consistently.
This approach may be helpful to those of you who want to spend more time studying. If you start by holding yourself accountable for watching, say, 10 seconds of an educational video every day — and you do so consistently — the habit of studying daily will grow naturally. Even if you learn nothing in that 10 seconds, you’re establishing the habit of studying a little every day. On some days, maybe you’ll end up studying for an hour or longer.