{"id":1601,"date":"2019-08-23T16:12:34","date_gmt":"2019-08-23T16:12:34","guid":{"rendered":"https:\/\/machinelearningindaba.com\/2019\/?page_id=1601"},"modified":"2019-08-23T16:22:35","modified_gmt":"2019-08-23T16:22:35","slug":"practical-coding-sessions-at-indaba-2019","status":"publish","type":"page","link":"https:\/\/deeplearningindaba.com\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/","title":{"rendered":"Practical Coding Sessions at Indaba 2019"},"content":{"rendered":"<p>This year we\u2019re expanding our practical sessions into two parallel tracks covering Fundamentals in one and more Specialized topics in the other.<br \/>\n\u200b<br \/>\n<b>NOTE<\/b>:<\/p>\n<ul>\n<li>If you\u2019re new to machine learning and deep learning, we suggest that you follow the Fundamental track.<\/li>\n<li>If you have some experience training deep learning models already, you may be interested in the more advanced topics covered in our Specialized track.<\/li>\n<\/ul>\n<h4>Fundamental Track<\/h4>\n<p>\u200b1a. <b>Machine Learning Fundamentals<\/b>: We introduce the idea of classification (sorting things into categories) using a machine-learning model. We explore the relationship between a classifier&#8217;s parameters and its decision boundary (a line that separates predictions of categories) and also introduce the idea of a loss function. Finally, we briefly introduce Tensorflow. <a href=\"https:\/\/colab.research.google.com\/drive\/1Eexka2tDbbW39T1LeQsOe0w66WSorNAs#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">[Colab Notebook]<\/a><br \/>\n2a. <b>Deep Feedforward Networks<\/b>: We implement and train a feed-forward neural network (or &#8220;multi-layer perceptron&#8221;) on a dataset called &#8220;Fashion MNIST&#8221;, consisting of small greyscale images of items of clothing. We consider the practical issues around generalisation to out-of-sample data and introduce some important techniques for addressing this. <a href=\"https:\/\/colab.research.google.com\/drive\/19YwLLREPmE0ZFoneEKQpSk-XRIIF9pPP#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">[Colab Notebook]<\/a><br \/>\n3a. <b>Convolutional Networks<\/b>: We cover the basics of convolutional neural networks (&#8220;ConvNets&#8221;). ConvNets were invented in the late 1980s, and have had tremendous success especially with computer vision (dealing with image data), although they have also been used very successfully in speech processing pipelines, and more recently, for machine translation. <a href=\"https:\/\/colab.research.google.com\/drive\/1GhO1DN8J1lmgIgV1zuKKMWd6m0SCUC5m#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">[Colab Notebook]<\/a><br \/>\n4a. <b>Recurrent Neural Networks<\/b>: Recurrent neural networks (RNNs) were designed to be able to handle sequential data (eg text or speech), and in this practical we will take a closer look at RNNs and then build a model that can generate English sentences in the style of Shakespeare! <a href=\"https:\/\/colab.research.google.com\/drive\/18r3RipZvESHPE-XM1Y8w9iIsJ8JXPK8s#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">[Colab Notebook]<\/a><\/p>\n<h4>Specialized Track<\/h4>\n<p>\u200b<br \/>\n<b>\u200bImportant Note<\/b>: Please note that most of the Specialized sessions require pre-work (before the Indaba!) in order for you to get the most out of them. The pre-work for each of these sessions is to read through them and go through the background knowledge, making sure that you are comfortable with most of it, to make sure that you will be able to enjoy and get the most out of attending the session.<br \/>\n1b. <b>Build your own TensorFlow<\/b>: This practical covers the basic idea behind Automatic Differentiation, a powerful software technique that allows us to quickly and easily compute gradients for all kinds of numerical programs. We will build a small Python framework that allows us to train our own simple neural networks, like Tensorflow does, but using only Numpy. <b>NOTE<\/b>: This practical is particularly long, so coming adequately prepared is very important.<\/p>\n<ul>\n<li>Pre-work: Please read through all the background sections in the practical.<\/li>\n<li>Background knowledge requirements:<\/li>\n<ul>\n<li>Linear algebra (multiplying matrices, row vectors, column vectors, summation notation)<\/li>\n<li>Calculus (derivatives and partial derivatives, Jacobian matrix)<\/li>\n<li>Deep Learning (have used a framework like TensorFlow or pytorch before)<\/li>\n<\/ul>\n<\/ul>\n<p><a href=\"https:\/\/colab.research.google.com\/drive\/14GeXkFd5pQKKNIJ7BMswP0ihlYg5nIbS#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">[Colab Notebook]<\/a><br \/>\n2b. <b>Optimization for Deep Learning<\/b>: We take a deep dive into optimization, an essential part of deep learning, and machine learning in general. We&#8217;ll take a look at the tools that allow as to turn a random collection of weights into a state-of-the-art model for any number of applications. More specifically, we&#8217;ll implement a few standard optimisation algorithms for finding the minimum of Rosenbrock&#8217;s banana function and then we&#8217;ll try them out on FashionMNIST.<br \/>\nBackground knowledge requirements:<br \/>\nEveryone with an undergraduate background in calculus should be able to enjoy this practical.<br \/>\n<a href=\"https:\/\/colab.research.google.com\/drive\/1Dn6nqhbhFnwdNtTHtHMCe-KRXKOTyp_G#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">[Colab Notebook]<\/a><br \/>\n3b. <b>Deep Generative Models<\/b>: We will investigate two kinds of deep generative models, namely Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). We will first train a GAN to generate images of clothing, and then apply VAEs to the same problem.<br \/>\nBackground knowledge requirements:<br \/>\nSupervised learning<br \/>\nConvNets (deep learning for image data)<br \/>\nProbability theory (the normal distribution, conditional and marginal probabilities, expected values, Bayes\u2019 rule)<br \/>\n<a href=\"https:\/\/colab.research.google.com\/drive\/1QYej56ctstejvRx6uK9qTTlF44NXzqH1#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">[Colab Notebook]<\/a><br \/>\n4b. <b>Reinforcement Learning<\/b>: We explore the reinforcement learning problem using the OpenAI Gym environment. We will then build agents that control the environments in three different ways: An agent that takes random actions, a neural net agent trained with random search, and lastly a neural net agent trained using a policy gradient algorithm.<br \/>\nBackground knowledge requirements:<br \/>\nMLPs (neural networks basics)<br \/>\nSupervised learning<br \/>\nProbability theory (expectations)<br \/>\n<a href=\"https:\/\/colab.research.google.com\/drive\/1K2UHOet0o0HKY31JaY2CSEh3te7LdPhS#forceEdit=true&#038;offline=true&#038;sandboxMode=true\" rel=\"noopener noreferrer\" target=\"_blank\">\u200b\u200b[Colab Notebook]<\/a><\/p>\n<h4>Getting Help<\/h4>\n<p>There will be several tutors around during the session to help with the sessions to help you move through the practicals.<br \/>\nHelping each other is part of the Indaba spirit. You can do this using the <a href=\"https:\/\/deeplearningi-sgh4338.slack.com\/messages\/CLU8X0WF5\" rel=\"noopener noreferrer\" target=\"_blank\">#practicals<\/a> channel on Slack.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This year we\u2019re expanding our practical sessions into two parallel tracks covering Fundamentals in one and more Specialized topics in the other. \u200b NOTE: If you\u2019re new to machine learning and deep learning, we suggest that you follow the Fundamental track. If you have some experience training deep learning models already, you may be interested [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1536,"parent":1267,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":"","_links_to":"","_links_to_target":""},"class_list":["post-1601","page","type-page","status-publish","has-post-thumbnail","hentry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Practical Coding Sessions at Indaba 2019 - Deep Learning Indaba 2019<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Practical Coding Sessions at Indaba 2019 - Deep Learning Indaba 2019\" \/>\n<meta property=\"og:description\" content=\"This year we\u2019re expanding our practical sessions into two parallel tracks covering Fundamentals in one and more Specialized topics in the other. \u200b NOTE: If you\u2019re new to machine learning and deep learning, we suggest that you follow the Fundamental track. If you have some experience training deep learning models already, you may be interested [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/\" \/>\n<meta property=\"og:site_name\" content=\"Deep Learning Indaba 2019\" \/>\n<meta property=\"article:modified_time\" content=\"2019-08-23T16:22:35+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/34.24.164.239\/2019\/wp-content\/uploads\/sites\/4\/2019\/08\/accommodation.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"4000\" \/>\n\t<meta property=\"og:image:height\" content=\"2387\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/\",\"url\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/\",\"name\":\"Practical Coding Sessions at Indaba 2019 - Deep Learning Indaba 2019\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/deeplearningindaba.com\\\/2019\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2019\\\/08\\\/accommodation.jpg\",\"datePublished\":\"2019-08-23T16:12:34+00:00\",\"dateModified\":\"2019-08-23T16:22:35+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/#primaryimage\",\"url\":\"https:\\\/\\\/deeplearningindaba.com\\\/2019\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2019\\\/08\\\/accommodation.jpg\",\"contentUrl\":\"https:\\\/\\\/deeplearningindaba.com\\\/2019\\\/wp-content\\\/uploads\\\/sites\\\/4\\\/2019\\\/08\\\/accommodation.jpg\",\"width\":4000,\"height\":2387},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/indaba\\\/practical-coding-sessions-at-indaba-2019\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Indaba\",\"item\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Practical Coding Sessions at Indaba 2019\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/#website\",\"url\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/\",\"name\":\"Deep Learning Indaba 2019\",\"description\":\"Just another Deep Learning Indaba site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/34.24.164.239\\\/2019\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Practical Coding Sessions at Indaba 2019 - Deep Learning Indaba 2019","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/","og_locale":"en_US","og_type":"article","og_title":"Practical Coding Sessions at Indaba 2019 - Deep Learning Indaba 2019","og_description":"This year we\u2019re expanding our practical sessions into two parallel tracks covering Fundamentals in one and more Specialized topics in the other. \u200b NOTE: If you\u2019re new to machine learning and deep learning, we suggest that you follow the Fundamental track. If you have some experience training deep learning models already, you may be interested [&hellip;]","og_url":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/","og_site_name":"Deep Learning Indaba 2019","article_modified_time":"2019-08-23T16:22:35+00:00","og_image":[{"width":4000,"height":2387,"url":"https:\/\/34.24.164.239\/2019\/wp-content\/uploads\/sites\/4\/2019\/08\/accommodation.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/","url":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/","name":"Practical Coding Sessions at Indaba 2019 - Deep Learning Indaba 2019","isPartOf":{"@id":"https:\/\/34.24.164.239\/2019\/#website"},"primaryImageOfPage":{"@id":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/#primaryimage"},"image":{"@id":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/#primaryimage"},"thumbnailUrl":"https:\/\/deeplearningindaba.com\/2019\/wp-content\/uploads\/sites\/4\/2019\/08\/accommodation.jpg","datePublished":"2019-08-23T16:12:34+00:00","dateModified":"2019-08-23T16:22:35+00:00","breadcrumb":{"@id":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/#primaryimage","url":"https:\/\/deeplearningindaba.com\/2019\/wp-content\/uploads\/sites\/4\/2019\/08\/accommodation.jpg","contentUrl":"https:\/\/deeplearningindaba.com\/2019\/wp-content\/uploads\/sites\/4\/2019\/08\/accommodation.jpg","width":4000,"height":2387},{"@type":"BreadcrumbList","@id":"https:\/\/34.24.164.239\/2019\/indaba\/practical-coding-sessions-at-indaba-2019\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/34.24.164.239\/2019\/"},{"@type":"ListItem","position":2,"name":"Indaba","item":"https:\/\/34.24.164.239\/2019\/"},{"@type":"ListItem","position":3,"name":"Practical Coding Sessions at Indaba 2019"}]},{"@type":"WebSite","@id":"https:\/\/34.24.164.239\/2019\/#website","url":"https:\/\/34.24.164.239\/2019\/","name":"Deep Learning Indaba 2019","description":"Just another Deep Learning Indaba site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/34.24.164.239\/2019\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/pages\/1601","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/comments?post=1601"}],"version-history":[{"count":9,"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/pages\/1601\/revisions"}],"predecessor-version":[{"id":1610,"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/pages\/1601\/revisions\/1610"}],"up":[{"embeddable":true,"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/pages\/1267"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/media\/1536"}],"wp:attachment":[{"href":"https:\/\/deeplearningindaba.com\/2019\/wp-json\/wp\/v2\/media?parent=1601"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}