Becoming creators of machine learning
We have heard calls for us, the African Machine Learning (ML) community, to move beyond being consumers of the latest ML techniques (such as 1 and 2). Whether we are solving African problems or core theoretical challenges, we should be part of creating and building this technology. We shouldn’t be regulated to the sidelines, only being allowed to be involved in data annotation or content moderation. I believe most of us support this call, but how? How do we become the architects of our ML destiny? I think a step towards this is becoming ML experts – creators of the technology we use. I will briefly discuss some ideas of the first steps of how I think we can get there.
Find your long-term motivation
Before dedicating time to something, it is important to think about your motivation – your “why”. This could be your reason to be part of the ML community or seek to be an ML expert. You could be intrinsically motivated by a passion for a topic. ML often has these hype cycles, where one topic becomes popular for a few years. Although it is not necessarily negative to follow a trending field, as this means it is important and likely to lead to employment, it is still important to explore different fields to figure out what you are interested in. Who knows what will be the next big trend in ML?
Another angle to finding motivation is to think about important problems and opportunities around you. Where do your interests align with what your community needs or what they could need in the future? In this case, the motivation is helping solve the problem, rather than the specific topic. In 2021, I played a small part in a project trying to predict locust outbreaks in Africa. These projects have a high potential impact, but they can also have some challenges. These challenges could include the problem being more difficult than you initially thought and it could require many stakeholders to have a meaningful impact.
Irrespective of the kind of motivation, it should be strong enough to push you through the years and the challenges it takes to become an expert. At this time, it could be useful to join multiple ML communities – such as Masakhane, Sisonke Biotik, or Ro’ya-CV4Africa – and find your motivation through interacting with others.
Foundations
Once you have your motivation and a general idea of your interests, it is important to try and get a good understanding of the fundamentals. In ML, this is usually mathematics and computer science, but depending on your topic of interest, it would be important to identify the relevant foundations. I would recommend finding a project or paper you find interesting, then looking at which aspects you struggle to understand and focusing on learning these things. There are many excellent online resources out there, such as MIT OpenCourseWare, Deep Learning Specialisation on Coursera (financial aid is available) and past Indaba lectures, just to name a few.
Effective Learning Techniques
Absorbing information for long periods of time is challenging. Two techniques that could be useful are spaced repetition and active learning. Active learning is an activity that engages the student in the learning process (Prince, 2004), through activities such as assessed questions, group discussions or applying techniques to new domains. These activities force students to think about what they are learning.
In spaced repetition, instead of learning/”cramming” knowledge in a single session, you can spread out the learning of a topic into multiple sessions at increasing intervals over time. This spacing out over time helps retain learned concepts and has even been shown to be useful for remembering mathematical principles (Pashler, 2007). Anki is a useful app for spaced repetition.
Let’s say you want to learn about how neural networks learn, sure you can watch a short YouTube video about this or read a blog post. For most things, this level of understanding might be adequate, but if you are learning something foundational and important to you, you need to be engaged in the learning and sometimes do the calculations. In the 2022 Indaba, I was working on the “Gradient descent” section in the “Intro to ML using JAX” practical. Calculating the gradients for backpropagation by hand (Appendix 2.4) engaged me in the learning process and helped reinforce the intuition of how neural networks learn. For long-term retention of the main calculus rules used in the proof, using Anki flashcards would have been useful.
Some people might prefer working through textbooks or online courses on foundational topics. This is also a great way of learning, but I would recommend using content focused on these topics relevant to ML. For example, if you are interested in the mathematics of ML, I would recommend reading and doing the exercises in math for ML books (e.g. Mathematics for Machine Learning or Probabilistic Machine Learning: An Introduction), rather than general courses on mathematics. This would be a more streamlined way to learn important concepts with a limited amount of time.
Find a project
Communities aren’t just useful for helping with motivation, but also for learning through engaging or teaching. Once you are a part of a community – join discussions, do talks, present papers and project ideas. The process of teaching or discussing with others will help clarify your thinking. Through this time, you will likely find interesting projects or people to work with, and this might be an ideal time to specialise in a subfield of ML and start a project. This will further reinforce the foundations you have learned and you can learn through collaborating with others.
In applied research, the project you work on might be inspired by challenges you have seen in your community or globally. In fundamental research, finding a problem can be tricky. One approach is problem-driven research, where you find limitations with the current methods, i.e. what can’t current state-of-the-art algorithms do/do efficiently. Sometimes you can find these limitations from reading papers, and other times this might require implementing some of these methods and trying on domains and seeing where they fall apart. There is no single defined way to make progress on ML problems, but having solid foundations, good ways to scientifically measure progress in your problem and fast idea and implementation iteration, are some of the possible ways to make progress on these sorts of problems. You can read here for more on problem-driven research.
Let’s mould our ML destiny
The learning process can be long and tough, one that never really ends. There will likely be many challenges along the way, such as time constraints (try allocating an amount of time you are comfortable with), lack of resources (try Google Colab) or lack of mentorship (try the Indaba Mentorship Programme). Don’t let confidence or self-belief hold you back! It is important to be kind to yourself and realise no one is an expert from day one. Also, remember to have fun!
Through many iterations of the above steps, we can become creators of ML techniques and methods relevant to Africa and the world. Let’s turn from open hands, waiting to receive ML, to hands grabbing and moulding our ML destiny.
– Kale-ab Tessera, 2022-2023 Chair of the Practicals Committee, 2022 Member of the Programme Committee and long-time contributor to Baobab. March 2024.
