Made a simple neural network in C implementing Stochastic Gradient Descent algorithm
Repo: https://github.com/aadityansha06/DL-C/blob/main/stochastic-gradient-descent.c
So I mostly code in C, yesterday I thought of trying doing DL though I never did any course or something on DL, I have basic understanding of what neuron is, what forward propagation means etc... just these basic terms.
Thus I tried to create very small neural network or a single layer neuron which get trained on single dataset to find the optimal weight for it. Like if yre input is 2 and expected output is 4. Thus optimal weight i got was around 4.000002 in 300 epochs
After doing this I tried to do it for multiple dataset like {2,4} {3,9} {4,16} but I wasn't able to do. Thus I asked Ai what's the issue it said that I have implemented Stochastic Gradient Descent algorithm which work for single dataset meanwhile for multiple dataset, I needed to find the gradient for each of the dataset and the it's mean value or something and then optimizing it (today studying about gradient maths tbh on yt )..
The whole reason I'm sharing this because I never knew something like Stochastic Gradient Descent algorithm exist and unknowingly I implemented it rather getting into tutorial hell or something and learnt a lot more other stuff..
https://redd.it/1rynx7r
@r_cpp
Post #24855
18