Hyperparameters’ Impact Takeaways!
Batch Size
🤌 Impact: Increase to have more stable and smoother update to the Neural Network = Learning Time increase as a result, car actions take more time to generate because it would need higher experience buffer (memories). But if too small, then there will be very little relation between data points for the machine to learn.
Learning Rate
🤌 Impact: Helps to decrease learning time by increasing how likely the behavior sticks, but can be counter productive on bad reward functions. Keep in mind, smaller the LR, longer for the model to converge (Converge: Reach the 100% lap completion rate and no longer gain positive habits or ML Behaviors).
Discount Factor
🤌 Impact: Impacts how far ahead the model looks when deciding the actions to take to get around a track. We usually lower the discount factor, let’s say to 0.9, 0.95, 0.97 from the original to increase the model’s ability to go faster. High complexity tracks with sharp turns should benefit from a higher than or equal to 0.97 discount factor.
🤌 Analogy: Small kid, “I want your goal is to walk a step forward, that’s 1 cookie.” = low DF. Let it learn how to walk first before you teach how to make that kid run.
Epochs
Definition: The number of passes through the training data to update the neural network weights during gradient descent.
🤌 Analogy: Imagine reading a book. Reading once = 1 Epoch. Reading one more time = 2 Epochs, and you gain new insights that you missed last time. It just adds time, may not converge if it takes too much time.
Entropy
Definition: The added uncertainty helps the AWS DeepRacer vehicle explore the action space more broadly.
More Knowledge Blogged Here:
https://lnkd.in/gZurhE4A
#DeepLearning #ReinforcementLearning #MachineLearning #Concepts
