Linear Regression

*Linear : a mathematical or statistical relationship that is proportional or additive in nature.

In a linear relationship, two variables are related in such a way that when one variable changes, the other variable changes proportionally.

*Regression : a statistical method for modeling the relationship between a dependent variable and one or more independent variables.

The goal of regression analysis is to find a function or equation that can predict the value of the dependent variable based on the values of the independent variables.

 

So how do we make variables that are dependent and independent linear?

 

Plot the data: First, plot the data points for the dependent variable and the independent variable on a graph. This will allow you to visualize the relationship between the variables and identify any patterns or trends.

Check for linearity: Look for a linear pattern in the data by examining the scatter plot. If the data points appear to fall along a straight line, then the relationship between the variables may be linear.

Transform the data: If the relationship between the variables is not linear, you can try transforming the data to make it linear. Common transformations include taking the logarithm or square root of one or both variables.

Fit a linear model: Once you have identified a linear relationship between the variables, you can fit a linear model to the data. In the case of simple linear regression, this involves fitting a straight line to the data that minimizes the sum of squared errors between the observed values and the predicted values.

Evaluate the model: Finally, evaluate the fit of the linear model by examining the residuals (the differences between the observed values and the predicted values). If the residuals are randomly distributed around zero, then the linear model is a good fit for the data.

Design Matrix (input) || Target Vector (label) || Output Vector (prediction)

the goal is to get the output vector as close as we can to the target vector.

 

the representation of a linear model that has only weight / weight+bias

single training model that has only weight

weight + bias -> theta^T*x + theta(0) (theta(0) is bias)

 

Summary of a linear model