Auto Regressive(AR) Model

This model uses observations from some no of lagged observation i.e it takes observations from previous time steps as input to a regression equation to predict the value at the next time step.Or in other word, It predicts future behaviour based on past behaviour.

It’s used when there is some correlation between values in a time series and the values that precede and succeed them.It is an example of stochastic process, which have degrees of uncertainty or randomness in it. The randomness means that you might be able to predict future trends pretty well with past data, but you’re never going to get 100 percent accuracy.

Source: https://www.statisticshowto.datasciencecentral.com/autoregressive-model/

The model is given as AR(p) where p is the order of autoregressive part and gives the info about how many lagged observation have to be taken. This model can be summarised as a linear regression model whose Equation is given as y_t<= δ + φ₁y_t-1 + φ₂y_t-2 + … + φ_py_t-1 + A_t

Where:

• y_t-1, y_t-1, y_t-2….y_t-p are the lagged values.
• A_t is white noise(randomness) * δ is given as :

where μ is the process mean.

AR(0): No dependence between the terms. Only the error/noise term contributes to the output of the process.

AR(1): Only the previous term in the process and the noise term contribute to the output.

AR(2): The previous two terms and the noise term contribute to the output.

Moving Average (MA) Model

A moving average term in a time series model is a past error (multiplied by a coefficient). Instead of using past values of the series in a regression, It uses the series mean and previous errors to make predictions.

where ε_t is white noise.
Here the series current deviation from mean, depends on previous deviations.

The model is given as MA(q) where q is the Order of the moving-average process and gives the  number of lagged forecast errors in the prediction equation.

MA(q) models are very similar to AR(p) models. The difference is that the MA(q) model is a linear combination of past white noise error terms as opposed to a linear combo of past observations like the AR(p) model. The motivation for the MA model is that we can observe “shocks” in the error process directly by fitting a model to the error terms. In an AR(p) model these shocks are observed indirectly by using the ACF on the series of past observations.

Source: http://www.blackarbs.com/blog/time-series-analysis-in-python-linear-models-to-garch/11/1/2016

Autocorrelation and Partial Autocorrelation

ACF

The coefficient of correlation between two values in a time series is called the autocorrelation function (ACF) For example the ACF for a time series y_t is given by:

Corr(y_t,y_t−k)

where k is lag.

ACF function is a way to measure the linear relationship between an observation at time t and the observations at previous times.

Partial Autocorrelation Function (PACF)

PACF is a conditional correlation.It is the correlation between two variables taking into account how thee two variable are related to some other set of variables.

For example, If we have to find out the correlation between the variables y and x₃ then we need to determine how both y and x₃ are related to x₁ and x₂. PACF of that can be determined as :

Taking the case of time series, the PACF between x_t and x_t-h will be determined taking the account x_t and x_t-h are related to x_t-h-1, … x_t-1 .

The 1st order partial autocorrelation will be defined to equal the 1st order autocorrelation. The 2nd order PACF is:

similarly 3rd order PACF is:

The PACF is most useful for identifying the order of an autoregressive model.

---

NOTE

Variance and covariance are mathematical terms frequently used in statistics, and despite the similar-sounding names they actually have quite different meanings.

A covariance refers to the measure of how two random variables will change together and is used to calculate the correlation between variables.

The variance refers to the spread of the data set—how far apart the numbers are in relation to the mean, for instance. Variance is particularly useful when calculating the probability of future events or performance.

 In a finance context, covariance is the term used to describe how two stocks will move together. A positive covariance indicates both tend to move upward or downward in value at the same time, while an inverse, or negative, covariance means they will move counter to each other; when one rises, the other falls.

---

Source: https://www.investopedia.com/ask/answers/041515/what-difference-between-variance-and-covariance.asp

You have to look both ACF and PACF values vs lag graph. If in ACF vs lag graph if you see large ACF values and a non-random pattern then it is most likely to have serially correlated values. If in PACF vs lag graph though pattern usually appear random but large PACF values at a given lag indicate this lag value will be a possible choice for our AR model.

Time series can have AR or MA signatures:

• An AR signature corresponds to a PACF plot displaying a sharp cut-off and a more slowly decaying ACF;
• An MA signature corresponds to an ACF plot displaying a sharp cut-off and a PACF plot that decays more slowly.

	ACF	PACF
AR	Geometric	Significant till p lags
MA	Significant till p lags	Geometric
ARMA	Geometric	Geometric

credit: https://www.youtube.com/watch?v=-vSzKfqcTDg