# 9 regression problem

## 9.1 fundamentals of machine learning

- Course review

- Fundamentals of Python language
- Numpy / Matplotlib / pandas / pilot application
- Tensorflow 2.0 low level API

- About to learn

- Machine learning, artificial neural network, deep learning, convolutional neural network
- Tensorflow 2.0 implementation of typical model

### 9.1.1 machine learning

- Machine Learning: the process of learning models from data through learning algorithms.
- process

- Establish model y=wx+b
- Learning model determination w, b
- Forecasting house prices using model calculation algorithm

- Learning algorithm: an algorithm for generating models from data
- data set / sample set: the set of data used for learning
- Sample: for each record in the dataset, the sample is composed of attributes and calibrations
- attribute: also known as feature: reflects the performance and nature of the sample
- label: it is the result of prediction or classification

#### 9.1.1.1 Supervised Learning

- Supervised Learning: the learning of this marked data set is called Supervised Learning. Its process is to learn the data and summarize the relationship between attributes and labels, that is, the model.
- Model / hypothesis / learner: estimation function
- ground truth: the learned model should approximate the real law
- Supervised learning can be divided into:

- regression: predicting continuous values
- Classification: predicting discrete values

#### 9.1.1.2 Unsupervised Learning

- Unsupervised Learning: when the sample data is not marked, mining the relationships contained in the data
- Clustering: aggregation of samples with high similarity. Birds of a feather flock together and people flock together. I don't care what this category is
- Distance: describes the similarity between eigenvalues

#### 9.1.1.3 semi supervised learning

- Combining supervised learning with unsupervised learning
- A large number of unlabeled data and a small amount of labeled data are used to learn together

### 9.1.2 development and application of machine learning

- Symbolic learning is the mainstream, theoretical research and model research in early machine learning
- Statistical machine learning was developed in the 1980s and 1990s and applied research
- Machine learning can extract valuable information from data, highlight the laws behind data, and realize large-scale data recognition, classification and prediction

## 9.2 Simple linear regression

- y=wx+b

- Model variable: x
- Model parameters: w is weights and b is bias
- Estimate: y'i = wxi+b
- Fitting error / residual: yi-y'i = yi - (wxi+b)
- The best fit line should minimize the cumulative residual value of all points

### 9.2.1 loss function

#### 9.2.1.1 select loss function

- How?

- Residual sum minimum

Loss/cast function: the degree of inconsistency between the predicted value and the real value of the model

- Absolute sum of residuals

- Minimum sum of squares of residuals

This loss function is called the Square Loss function, Euclidean distance - Mean square error

In practical applications, it is often used as a loss function

- The method of minimizing the mean square error is called Least Square Method

#### 9.2.1.2 two properties of loss function

- Nonnegativity: ensure that sample errors do not offset each other
- Consistency: the value of the loss function is consistent with the change of the error. Monotonically bounded, converging to 0

#### 9.2.1.3 solution stage

- Extreme value problem: the partial derivative of the extreme point is zero

- The solution may be different if the solution process is different

In fact, it is equivalent. The latter is generally used, which is more commonly used

- The solution obtained by strictly pushing to the calculation is called Analytical solution. The Analytical solution is a closed form function. Given any independent variable, the accurate dependent variable can be obtained through strict formula. Therefore, the Analytical solution is also called closed form solution

## 9.3 example: realizing univariate linear regression by analytical method

### 9.3.1 example: implementation of univariate linear regression by analytical method (1)

#### 9.3.1.1 realize a commercial housing value evaluation system

- step

- Loading sample data: x, y
- Learning model: calculating w, b

- Forecast house price

y' = wx+b

- The following is implemented by Python, Numpy and TensorFlow

##### 9.3.1.1.1 Python implementation only

###### 9.3.1.1.1.1 loading sample data

# 1 load sample data x = [137.97,104.50,100.00,124.32,79.20,99.00,124.00,114.00,106.69,138.05,53.75,46.91,68.00,63.02,81.26,86.21] y = [145.00,110.00,93.00,116.00,65.32,104.00,118.00,91.00,62.00,133.00,51.00,45.00,78.50,69.65,75.69,95.30]

###### 9.3.1.1.1.2 learning model: calculation w, b

# 2 learning model: calculation w, b meanX = sum(x)/len(x) meanY = sum(y)/len(y) sumXY = 0.0 sumY = 0.0 for i in range(len(x)): sumXY += (x[i]-meanX)*(y[i]-meanY) sumY += (x[i]-meanX)*(x[i]-meanX) w = sumXY/sumY b = meanY - w*meanX print("w=",w) print("b=",b) print(type(w),type(b))

The output result is:

w= 0.8945605120044221 b= 5.410840339418002 <class 'float'> <class 'float'>

###### 9.3.1.1.1.3 forecast house price

# Forecast house price x_test = [128.15,45.00,141.43,106.27,99.00,53.84,85.36,70.00] for i in range(len(x_test)): print(x_test[i],"\t",w*x_test[i]+b)

The output result is:

128.15 120.0487699527847 45.0 45.66606337961699 141.43 131.92853355220342 106.27 100.47578595012793 99.0 93.97233102785579 53.84 53.57397830573609 85.36 81.77052564411547 70.0 68.03007617972756

###### 9.3.1.1.1.4 all code records:

# 1 load sample data x = [137.97,104.50,100.00,124.32,79.20,99.00,124.00,114.00,106.69,138.05,53.75,46.91,68.00,63.02,81.26,86.21] y = [145.00,110.00,93.00,116.00,65.32,104.00,118.00,91.00,62.00,133.00,51.00,45.00,78.50,69.65,75.69,95.30] # 2 learning model: calculation w, b meanX = sum(x)/len(x) meanY = sum(y)/len(y) sumXY = 0.0 sumY = 0.0 for i in range(len(x)): sumXY += (x[i]-meanX)*(y[i]-meanY) sumY += (x[i]-meanX)*(x[i]-meanX) w = sumXY/sumY b = meanY - w*meanX print("w=",w) print("b=",b) print(type(w),type(b)) # Forecast house price x_test = [128.15,45.00,141.43,106.27,99.00,53.84,85.36,70.00] print("the measure of area\t Estimated house price") for i in range(len(x_test)): print(x_test[i],"\t",round(w*x_test[i]+b,2))

The output result is:

w= 0.8945605120044221 b= 5.410840339418002 <class 'float'> <class 'float'> the measure of area Estimated house price 128.15 120.05 45.0 45.67 141.43 131.93 106.27 100.48 99.0 93.97 53.84 53.57 85.36 81.77 70.0 68.03

##### 9.3.1.1.2 Numpy implementation

###### 9.3.1.1.2.1 loading sample data

import numpy as np x = np.array([137.97,104.50,100.00,124.32,79.20,99.00,124.00,114.00,106.69,138.05,53.75,46.91,68.00,63.02,81.26,86.21]) y = np.array([145.00,110.00,93.00,116.00,65.32,104.00,118.00,91.00,62.00,133.00,51.00,45.00,78.50,69.65,75.69,95.30])

###### 9.3.1.1.2.2 learning model: calculation w, b

meanX = np.mean(x) meanY = np.mean(y) sumXY = np.sum((x-meanX)*(y-meanY)) sumY = np.sum((x-meanX)*(x-meanX)) w = sumXY/sumY b = meanY - w*meanX print("w=",w) print("b=",b) print(type(w),type(b))

The output result is:

w= 0.894560512004422 b= 5.410840339418002 <class 'numpy.float64'> <class 'numpy.float64'>

###### 9.3.1.1.2.3 forecast house price

x_test = np.array([128.15,45.00,141.43,106.27,99.00,53.84,85.36,70.00]) y_pred = w*x_test + b print("the measure of area\t Estimated house price") for i in range(len(x_test)): print(x_test[i],"\t",np.round(y_pred[i],2))

The output result is:

the measure of area Estimated house price 128.15 120.05 45.0 45.67 141.43 131.93 106.27 100.48 99.0 93.97 53.84 53.57 85.36 81.77 70.0 68.03

###### 9.3.1.1.2.4 all code records

import numpy as np x = np.array([137.97,104.50,100.00,124.32,79.20,99.00,124.00,114.00,106.69,138.05,53.75,46.91,68.00,63.02,81.26,86.21]) y = np.array([145.00,110.00,93.00,116.00,65.32,104.00,118.00,91.00,62.00,133.00,51.00,45.00,78.50,69.65,75.69,95.30]) meanX = np.mean(x) meanY = np.mean(y) sumXY = np.sum((x-meanX)*(y-meanY)) sumY = np.sum((x-meanX)*(x-meanX)) w = sumXY/sumY b = meanY - w*meanX print("w=",w) print("b=",b) print(type(w),type(b)) x_test = np.array([128.15,45.00,141.43,106.27,99.00,53.84,85.36,70.00]) y_pred = w*x_test + b print("the measure of area\t Estimated house price") for i in range(len(x_test)): print(x_test[i],"\t",np.round(y_pred[i],2))

The output result is:

w= 0.894560512004422 b= 5.410840339418002 <class 'numpy.float64'> <class 'numpy.float64'> the measure of area Estimated house price 128.15 120.05 45.0 45.67 141.43 131.93 106.27 100.48 99.0 93.97 53.84 53.57 85.36 81.77 70.0 68.03