<div style='margin-bottom: 20px;'> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">Why is it surprising that the brain can recognize low-resolution digits?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Because the specific values of each pixel vary greatly between images.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What function is applied to each component of the resulting vector in the neural network model described?</h2> <p style="font-weight: normal; font-size: 1.2rem;">The sigmoid function is applied to each specific component of the resulting vector.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does the activation number inside a neuron represent?</h2> <p style="font-weight: normal; font-size: 1.2rem;">The grayscale value of the corresponding pixel.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the purpose of the hidden layers in a neural network?</h2> <p style="font-weight: normal; font-size: 1.2rem;">To handle the process of recognizing digits.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">Why is a good grasp of linear algebra important in machine learning?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Much of machine learning comes down to having a good grasp of linear algebra.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the resolution of the image used to recognize the digit '3'?</h2> <p style="font-weight: normal; font-size: 1.2rem;">28x28 pixels</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does taking the weighted sum of the activations in the first layer represent in the context of matrix vector multiplication?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It corresponds to one of the terms in the matrix vector product.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How should the activations from one layer be organized for processing?</h2> <p style="font-weight: normal; font-size: 1.2rem;">The activations from one layer should be organized into a column as a vector.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What becomes difficult when trying to program a computer to recognize digits from a 28x28 pixel grid?</h2> <p style="font-weight: normal; font-size: 1.2rem;">The task goes from trivial to dauntingly difficult.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What range do activations in this network aim to fall within?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Between 0 and 1.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the purpose of having negative weights associated with the surrounding pixels?</h2> <p style="font-weight: normal; font-size: 1.2rem;">To ensure the sum is largest when the middle pixels are bright but the surrounding pixels are darker.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does the visual cortex do when recognizing digits?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It resolves images as representing the same idea while recognizing other images as distinct ideas.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the goal of introducing neural networks in the context provided?</h2> <p style="font-weight: normal; font-size: 1.2rem;">To show what a neural network is and help visualize its function, assuming no background knowledge.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How many neurons correspond to the 28x28 pixels of the input image?</h2> <p style="font-weight: normal; font-size: 1.2rem;">784 neurons</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">Which function is commonly used to squish the real number line into the range between 0 and 1?</h2> <p style="font-weight: normal; font-size: 1.2rem;">The sigmoid function.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What analogy is used to describe how activations in one layer determine the activations of the next layer?</h2> <p style="font-weight: normal; font-size: 1.2rem;">The analogy of biological networks of neurons where some groups of neurons firing cause others to fire.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How are the weights between layers in a neural network organized?</h2> <p style="font-weight: normal; font-size: 1.2rem;">The weights are organized as a matrix, where each row corresponds to the connections between one layer and a particular neuron in the next layer.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What classic example is used to introduce neural networks?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Recognizing handwritten digits.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does the activation of a neuron measure?</h2> <p style="font-weight: normal; font-size: 1.2rem;">How positive the relevant weighted sum is.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How many neurons are in the last layer of the network described, and what do they represent?</h2> <p style="font-weight: normal; font-size: 1.2rem;">10 neurons, each representing one of the digits.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is added to the matrix vector product to include the bias in the neural network computation?</h2> <p style="font-weight: normal; font-size: 1.2rem;">A vector containing all the biases is added to the previous matrix vector product.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">Why do many libraries optimize matrix multiplication in the context of neural networks?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Because it makes the relevant code both a lot simpler and a lot faster.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the purpose of adding a bias to the weighted sum?</h2> <p style="font-weight: normal; font-size: 1.2rem;">To determine how high the weighted sum needs to be before the neuron becomes meaningfully active.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How does representing weight matrices and vectors as symbols benefit the communication of activations transition?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It allows for the full transition of activations from one layer to the next to be communicated in an extremely tight and neat little expression.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the best hope for the function of the middle layers in recognizing digits?</h2> <p style="font-weight: normal; font-size: 1.2rem;">That each neuron in the second to last layer corresponds with one of the subcomponents of digits.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the total number of weights and biases in the network?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Almost exactly 13,000.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How might the network recognize a digit like '9'?</h2> <p style="font-weight: normal; font-size: 1.2rem;">By lighting up neurons associated with specific little edges that form a loop up top and a long vertical line.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How can understanding weights and biases help when the network doesn't perform as expected?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It provides a starting place for experimenting with changes to the structure to improve.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does each neuron in a neural network functionally represent?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Each neuron represents a function that takes in the outputs of all the neurons in the previous layer and outputs a number between 0 and 1.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does it mean when the network is described as having been 'trained' to recognize digits?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It means the network can process an image and output the digit it represents based on the pattern of activations.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How many hidden layers and neurons per layer are chosen for the network, and why?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Two hidden layers with 16 neurons each, chosen for motivational and visual clarity reasons.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What makes the neural network function described as 'absurdly complicated'?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It involves 13,000 parameters in the forms of weights and biases, and iterates many matrix vector products and the sigmoid function.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">Why might manually setting weights and biases be considered both fun and horrifying?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Because it involves purposefully tweaking the numbers to achieve desired outcomes, which is a meticulous and daunting task.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What role do weights play in determining how one layer influences the next in a neural network?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Weights determine the influence of activations from one layer on the activations of the next layer.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How many numbers does the neural network described take as input and output?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It takes in 784 numbers as an input and outputs 10 numbers.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does the ReLU function stand for?</h2> <p style="font-weight: normal; font-size: 1.2rem;">ReLU stands for rectified linear unit.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How many weights are there in the hidden layer of 16 neurons, considering each neuron is connected to all 784 pixel neurons from the first layer?</h2> <p style="font-weight: normal; font-size: 1.2rem;">12,544 (784 times 16).</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">Why have modern networks moved away from using the sigmoid function?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Because ReLU is much easier to train compared to sigmoid.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the broader application of detecting edges and patterns beyond image recognition?</h2> <p style="font-weight: normal; font-size: 1.2rem;">It could be useful for other intelligent tasks like parsing speech.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What is the challenge in designing how activations in one layer influence the next?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Determining a mechanism that can combine pixels into edges, edges into patterns, or patterns into digits.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">What does the process of learning in this context refer to?</h2> <p style="font-weight: normal; font-size: 1.2rem;">Getting the computer to find a valid setting for all the weights and biases to solve the problem at hand.</p> </div> <div style="margin-bottom: 10px; background-color: #f2f2f2; border-radius: 1rem; padding: 10px 20px;"> <h2 style="font-weight: bold; margin-bottom: 3px; font-size: 1.5rem;">How can weights be visualized in the context of influencing neuron activations?</h2> <p style="font-weight: normal; font-size: 1.2rem;">As a grid with green pixels indicating positive weights and red pixels indicating negative weights.</p> </div> </div>