Call Us: US - +1 845 478 5244 | UK - +44 20 7193 7850 | AUS - +61 2 8005 4826

Rank variables based on their degree of contribution to the model.

First, the regression might be used to identify the strength of the effect that the independent variable(s) have on a dependent variable.  Typical questions are what is the strength of relationship between dose and effect, sales and marketing spending, or age and income.

Second, it can be used to forecast effects or impact of changes.  That is, the regression analysis helps us to understand how much the dependent variable changes with a change in one or more independent variables.  A typical question is, “how much additional sales income do I get for each additional $1000 spent on marketing?”

Third, regression analysis predicts trends and future values.  The regression analysis can be used to get point estimates.  A typical question is, “what will the price of gold be in 6 months?”

There are several types of linear regression analyses available to researchers.

  • Simple linear regression
    1 dependent variable (interval or ratio), 1 independent variable (interval or ratio or dichotomous)
  • Logistic regression
    1 dependent variable (dichotomous), 2+ independent variable(s) (interval or ratio or dichotomous)
  • Ordinal regression
    1 dependent variable (ordinal), 1+ independent variable(s) (nominal or dichotomous)
  • Multinominal regression
    1 dependent variable (nominal), 1+ independent variable(s) (interval or ratio or dichotomous)

When selecting the model for the analysis, an important consideration is model fitting.  Adding independent variables to a linear regression model will always increase the explained variance of the model (typically expressed as R²).  However, overfitting can occur by adding too many variables to the model, which reduces model generalizability.  Occam’s razor describes the problem extremely well – a simple model is usually preferable to a more complex model.  Statistically, if a model includes a large number of variables, some of the variables will be statistically significant due to chance alone.