• Multilinear Regression (MLR)
• ARIMA (Linear time-series models, or Box-Jenkins models)
• Artificial Neural Networks (ANN)

These are described elsewhere, but compared below. Other available methods include:

• TableCurve, Jandel Scientific
• Abductive fitting, or Group means of data handling (GMDH)
• Response Surface Analysis (RSA)

Response Surface Analysis uses multivariate polynomial models, but usually restricts them to maximum exponent of 2, and doesn't do significance testing on the terms or stepwise model identification. Thus RSA is essentially a subset of MPR.

A summary of advantages and disadvantages of MPR compared with these methods follows.

ADVANTAGES of MPR vs. COMPETITION

Advantages of MPR over both ARIMA and Neural Networks:

• MPR gives absolute convergence in a finite number of steps.

Advantages of MPR over ARIMA models:

• Provides better fits to both fitting data and independent testing data;
• Includes nonlinear behaviors;
• Includes interactions among independent variables;
• Reduces bias (which may be caused by nonlinearity and interactions);
• Algorithm automatically selects (locally) optimal model.

Advantages of MPR over Artificial Neural Network models:

• Parsimonious: gives better fit with fewer coefficients;
• Reduced overfitting or "memorizing" data;
• Based on multiple regression: therefore easier for more people to use and understand;
• Selects only statistically significant coefficients (weights);
• Less computationally intensive;
• Does not require a priori selection of model structure;
• Automatically determines which independent variables and/or lags are significant;
• Gives a simple explicit equation which can be easily communicated, or used in programs or for further analysis;
• Allows use of numerous standard statistical tests developed for linear regression methods;
• Allows calculation of confidence intervals on forecasts.
• Models are easy to disseminate.

Advantages of MPR over TABLECURVE

• MPR is not limited to three dimensions;
• Shape of response surface is not restricted to a priori choices.

Advantages of MPR over ABDUCTIVE or GMDH models. These models also generate multivariate polynomials. However, they include numerous extra polynomial terms.

• MPR is more parsimonious and compact;
• Less prone to overfitting and explosive behavior.

DISADVANTAGES of MPR vs. COMPETITION

Disadvantages of MPR versus both MLR, ARIMA and ANN models:

• MPR may exhibit "explosive" behavior outside data ranges. However, this can be controlled either by checking the ranges when using the model or by use of an inverse sigmoidal transformation. Also, ARIMA and ANN models, while they might not "explode", are nevertheless unreliable outside the range of the data.
• Empirical results indicate that claims (e.g. by Casdagli) that polynomial models are more prone to explode upon iteration do not, in fact, occur (see chaos paper).

Disadvantages versus MLR and ARIMA models:

• More computationally intensive;
• Does not lend itself easily to tests of stationarity or stability (on the other hand, this is likely to be constrained by the behavior of the data. That is, if the model is accurate, it should only show instability where the actual system also is unstable).
• Produces "nonlinear bias." However, this is an inevitable feature of any nonlinear model, and is less serious than ignoring nonlinear effects. Furthermore, techniques are available for compensating for them.

Disadvantages versus ANN models:

• MPR is not as suitable as ANNs for very high-dimensional tasks such as graphical pattern recognition, unless the number of inputs can be limited to less than several dozen.

Disadvantages of MPR versus TABLECURVE

• TABLECURVE may be faster, since the list of candidate models is fixed and limited a priori.

Disadvantages of MPR versus ABDUCTIVE or GMDH models

• These models are probably faster than MPR, since they effectively restrict the candidate terms that may be examined.
  

Comparison of MPR with MECHANISTIC or FUNDAMENTAL Models

A mechanistic or fundamental model is one that is derived from fundamental principles. For example, we could use the laws of physics to derive a mechanistic model that predicts the stress and strain on a building in an earthquake.

In many cases, a system is too complicated to predict from fundamental principles. Almost anything to do with human behavior, such as economic systems, will fall into this category. But many physical systems can also be beyond mechanistic modeling. Some aspects of weather prediction would be an example. In such cases, we use empirical modeling. Literally, this means modeling based on experience. In practice, an empirical mathematical model is an equation whose coefficients are adjusted to match a given set of data.

In almost all cases, when it is possible to create a mechanistic model, then that is to be preferred over developing an empirical model. There are a few exceptions. Sometimes the empirical model will be much simpler than the mechanistic model, and will serve the required application better.

For example, in one application a researcher was developing a control system for growing plants in space for NASA (Fleisher, 2002). The controller required a model to predict plant growth. A mechanistic model was available, but was too complex to incorporate into the controller. In this case, the researcher "modeled the model." That is, he generated data using the mechanistic model, then used MPR to produce a simpler empirical model that would reproduce the behavior of the mechanistic model.

Sometimes a situation would seem to be simple enough to be suitable for mechanistic modeling, but really isn't. For example, engineers have a simple theory-based model to predict flow over a dam based on the level of the reservoir behind the dam. However, this equation doesn't work if the water level below the dam is also above the height of the top edge of the dam (although not as high as upstream of the dam). In this case, it is possible to do experiments to determine the effect of both the upstream and downstream water level on the flow rate, and to develop an empirical correlation to predict it.