Do you remember those puzzles of the following form:
What is the next number in the series: 1, 4, 9, 13, ??
What you usually look for in this kind of puzzle is a rule that
gives the next number based on the previous one or two numbers.
In this case 9 is computed from 4 and 1 in the same way that 13
is computed from 9 and 4. Try to figure this one out.
View the answer
in a pop up window.
One wit has said that when confronted with this kind of problem,
he always answers "19."
How can this be?
It is always possible to interpolate a polynomial through the given
numbers, plus the number 19. The polynomial that goes through the
series 1, 4, 9, 13, 19 is
y = 9  18*x + 12.75*x2  3*x3 + 0.25*x4; where x = 1, 2, …
Plugging 1 through 4 into this equation generates the given series.
If you plug 5 into this equation, you get 19.

This answer violates the assumption that you're looking for a rule
that the next value is a linear combination of the previous two
values. Put another way, the usual assumption is that you are looking
for an autoregressive model of degree 2, denoted an AR(2) model.
What if we relaxed the assumption a different way?
Let's allow products of previous values in the series. In other
words, we will allow multivariate polynomial autoregression (PAR)
models. Try this series: 1, 2, 3, 11, ??
View the answer
in a pop up window.
You can check that there is no possible AR(2) model that can describe
this series, and in fact no type of linear model. (Although if you
allow an ordinary polynomial, you can still answer 19!) This particular
type of polynomial model includes an interaction term. The next
one is a bit more difficult. It includes a quadratic term.
Can you guess the next number in this series?
1, 2, 4, 10, 30, ??
View the answer
in a pop up window.
You can see that if you started considering more variables, including
more lags, and allowed higherdegree exponents, the number of possible
terms in the model increases greatly, even if there are few terms
actually in the model. You would need a program like TaylorFit to
discover the model that describes the data.
