[ML] 2. Introduction to neural networks
Training an algorithm involes four ingredients:
- Data
- Model
- Objective function: We put data input a Model and get output out of it. The value we call it as 'lost'. We want to minimize the 'lost' value.
- Optimization algorithm: For example the linear model, we will try to optimize y = wx + b, 'w' & 'b' so that it will minimize the 'lost' value.
Repeat the process...
Three types of machine learning:
Supervised: Give feedback
- Classification: outputs are categories: cats or dogs
- Regression: output would be numbers.
Unsupervised: No feedback, find parttens
Reinforcement: Train the algorithm to works in a enviorment based on the rewords it receives. (Just like training your dog)
Linear Model:
f(x) = x * w + b
x: input
w: coefficient / weight
b: intercept / bias
Linear Model: Multi inputs:
x, w are both vectors:
x: 1 * 2
w: 2 * 1
f(x): 1 * 1
Notice that the lienar model doesn't chage, it is still:
f(x) = x * w + b
Lienar Model: multi inputs and multi outputs:
For 'W', the first index is always the same as X; the second index is always the same as ouput Y.
If there is K inputs and M outputs, the number of Weigths would be K * M
The number of bias is equal to the number of ouputs: M.
N * M = (N * K) * (K * M) + 1 * M
Each model is determined by its weights and biases.
Objection function:
Is the measure used to evaluate how well the model's output match the desired correct values.
- Loss function: the lower the loss function, the higher the level of accuracy (Supervized learning)
- Reward function: the hight of the reward function, the higher the level of accuracy (Reubfircement learning)
Loss functions for Supervised learning:
- Regression: L2-NORM
- Classification: CROSS-ENTROPY
Expect cross-entropy should be lower.
Optimization algorithm: Dradient descent
Until one point, the following value never update anymore.
The picture looks like this:
Generally, we want the learning rate to be:
High enough, so we can reach the closest minimum in a rational amount of time
Low enough, so we don't oscillate around the minimum
N-parameter gradient descent
相关文章
- Reporting Service 服务启动时报错The service did not respond to the start or control request in a timely fashion
- [React + Functional Programming ADT] Connect State ADT Based Redux Actions to a React Application
- [TS] Parse a string to an integer
- [Grid Layout] Use the minmax function to specify dynamically-sized tracks
- [Cycle.js] From toy DOM Driver to real DOM Driver
- [Web] Use Web Speech API to make the browser speak out loud using SpeechSynthesis
- [React Recoil] Use a Recoil atom to share state between two React components
- [NPM] Use npx to run commands with different Node.js versions
- [React] Pass a function to setState in React
- SAP CRM IBASE头部字段valid from和valid to的填充逻辑
- How to bind multiple properties with formatter on one control
- 成功解决To fix this you could try to: 1. loosen the range of package versions you‘ve specified
- CentOS-8,执行yum报错:Failed to download metadata for repo ‘AppStream‘,更换yum数据源