Regression - Generalized Linear Model
Model the relationships between a dependent variable and one or more independent variables
The Generalized Linear Model feature models the relationships between a dependent variable and one or more independent variables. There are seven types of regression analysis to choose from. The linear regression model is the default.
Regression Types
Linear
- The Linear Regression models the linear relationship between a dependent variable and one or more independent variables. The linear regression option is most commonly used when the dependent variable is continuous. See Regression - Linear Regression.
Binary Logit
- The Binary Logit is a form of regression analysis that models a binary dependent variable (e.g. yes/no, pass/fail, win/lose). It is also known as a Logistic regression, and Binomial regression. See Regression - Binary Logit.
Ordered Logit
- The Ordered Logit is a form of regression analysis that models a discrete and ordinal dependent variable with more than two outcomes (Net promoter Score, Customer Satisfaction rating, etc.). It is also known as an Ordinal Logistic Regression and the cumulative link model. See Regression - Ordered Logit.
Multinomial Logit
- The Multinomial Logit is a form of regression analysis that models a discrete and nominal dependent variable with more than two outcomes (Yes/No/Maybe, Red/Green/Blue, Brand A/Brand B/Brand C, etc.). It is also known as a multinomial logistic regression and multinomial logistic discriminant analysis. See Regression - Multinomial Logit.
Poisson
- The Poisson Regression is used to model count data with the assumption that the dependent variable has a Poisson distribution, where the mean is equal to the variance. If there is a high level of variance (overdispersion), the Quasi-Poisson or NBD may be a better option. See Regression - Poisson Regression.
Quasi-Poisson
- The Quasi-Poisson Regression is a generalization of the Poisson regression and is used when modeling an overdispersed count variable. The Quasi-Poisson model assumes that the variance is a linear function of the mean. See Regression - Quasi-Poisson Regression.
NBD
- The Negative Binomial Distribution (NBD) Regression is a generalization of the Poisson regression and is used when modeling an overdispersed count variable. The NBD model assumes that the variance is a quadratic function of the mean. See Regression - NBD Regression.
Create a Generalized Linear Model in Displayr
- 1. Go to Insert > Regression > Generalized Linear Model
- 2. Under Inputs > Outcome, select your dependent variable
- 3. Under Inputs > Predictor(s), select your independent variables
- 4. Under Inputs > Regression, select the model you want to use
Object Inspector Options
Outcome The variable to be predicted by the predictor variables.
Predictors The variable(s) to predict the outcome.
Type:
- Linear See Regression - Linear Regression.
- Binary Logit See Regression - Binary Logit.
- Ordered Logit See Regression - Ordered Logit.
- Multinomial Logit See Regression - Multinomial Logit.
- Poisson See Regression - Poisson Regression.
- Quasi-Poisson See Regression - Quasi-Poisson Regression.
- NBD See Regression - NBD Regression.
Robust standard errors Computes standard errors that are robust to violations of the assumption of constant variance (i.e., heteroscedasticity). See Robust Standard Errors. This is only available when Type is Linear.
Missing data See Missing Data Options.
- Summary The default; as shown in the example above.
- Detail Typical R output, some additional information compared to Summary, but without the pretty formatting.
- ANOVA Analysis of variance table containing the results of Chi-squared likelihood ratio tests for each predictor.
- Relative Importance Analysis The results of a relative importance analysis. See here and the references for more information. This option is not available for Multinomial Logit. Note that categorical predictors are not converted to be numeric, unlike in Driver (Importance) Analysis - Relative Importance Analysis.
- Effects Plot Plots the relationship between each of the Predictors and the Outcome. Not available for Multinomial Logit.
Correction The multiple comparisons correction applied when computing the p-values of the post-hoc comparisons.
Variable names Displays Variable Names in the output instead of labels.
Absolute importance scores Whether the absolute value of Relative Importance Analysis scores should be displayed.
Auxiliary variables Variables to be used when imputing missing values (in addition to all the other variables in the model).
Weight. Where a weight has been set for the R Output, it will automatically applied when the model is estimated. By default, the weight is assumed to be a sampling weight, and the standard errors are estimated using Taylor series linearization (by contrast, in the Legacy Regression, weight calibration is used). See Weights, Effective Sample Size and Design Effects.
Filter The data is automatically filtered using any filters prior to estimating the model.
Crosstab Interaction Optional variable to test for interaction with other variables in the model. See Linear Regression for more details.
Automated outlier removal percentage A numeric value between 0 and 50 (including 0 but not 50) to specify the percentage of the data that is removed from analysis. If a zero-value is selected for this input control then no outlier removal is performed and a standard regression output for the entire (possibly filtered) dataset is applied. If a non-zero value is selected for this option then the regression model is fitted twice. The first regression model uses the entire dataset (after filters have been applied) and identifies the observations that generate the largest residuals. The user specified percent of cases in the data that have the largest residuals are then removed. The regression model is refitted on this reduced dataset and output returned. The specific residual used in a generalized linear model (GLM) depends on the type of GLM. Typically a studentized deviance residual in an unweighted GLM and the Pearson residual in a weighted GLM, although sometimes surrogate residuals are used. See the Wiki pages for each model type above for more details.
Random seed Seed used to initialize the (pseudo)random number generator for the model fitting algorithm. Different seeds may lead to slightly different answers, but should normally not make a large difference.
Additional options are available by editing the code.
DIAGNOSTICS
Plot - Cook's Distance Creates a line/rug plot showing Cook's Distance for each observation.
Plot - Cook's Distance vs Leverage Creates a scatterplot showing Cook's distance vs leverage for each observation.
Plot - Influence Index Creates index plots of studentized residuals, hat values, and Cook's distance.
Multicollinearity Table (VIF) Creates a table containing variance inflation factors (VIF) to diagnose multicollinearity.
Plot - Normal Q-Q Creates a normal Quantile-Quantile (QQ) plot to reveal departures of the residuals from normality.
Prediction-Accuracy Table Creates a table showing the observed and predicted values, as a heatmap.
Test Residual Heteroscedasticity Conducts a heteroscedasticity test on the residuals.
Test Residual Normality (Shapiro-Wilk) Conducts a Shapiro-Wilk test of normality on the (deviance) residuals.
Plot - Residuals vs Fitted Creates a scatterplot of residuals versus fitted values.
Plot - Residuals vs Leverage Creates a plot of residuals versus leverage values.
Plot - Scale-Location Creates a plot of the square root of the absolute standardized residuals by fitted values.
Test Residual Serial Correlation (Durbin-Watson) Conducts a Durbin-Watson test of serial correlation (auto-correlation) on the residuals.
SAVE VARIABLE(S)
Fitted Values Creates a new variable containing fitted values for each case in the data.
Predicted Values Creates a new variable containing predicted values for each case in the data.
Residuals Creates a new variable containing residual values for each case in the data.
Additional Properties
When using this feature you can obtain additional information that is stored by the R code which produces the output.
- To do so, select Create > R Output.
- In the R CODE, paste: item = YourReferenceName
- Replace YourReferenceName with the reference name of your item. Find this in the Report tree or by selecting the item and then going to Properties > General > Name from the object inspector on the right.
- Below the first line of code, you can paste in snippets from below or type in str(item) to see a list of available information.
For a more in depth discussion on extracting information from objects in R, checkout our blog post here.
Properties which may be of interest are:
- Summary outputs from the regression model:
- item$summary$coefficients # summary regression outputs
Acknowledgements
Estimated using:
- R (R Core Team 2016).
- survey (Lumley 2014,2014), and MASS packages.
- car (Fox and Weisberg 2011)
- MASS (Venables and Ripley 2002)
See How to Read a Standard R Table for acknowledgements regarding the outputs.
References
R Core Team (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
John Fox and Sanford Weisberg (2011). An {R} Companion to Applied Regression, Second Edition. Thousand Oaks CA: Sage. URL: http://socserv.socsci.mcmaster.ca/jfox/Books/Companion
T. Lumley (2014) "survey: analysis of complex survey samples". R package version 3.30.
T. Lumley (2004) Analysis of complex survey samples. Journal of Statistical Software 9(1): 1-19.
Venables, W. N. & Ripley, B. D. (2002) Modern Applied Statistics with S. Fourth Edition. Springer, New York. ISBN 0-387-95457-0
Code
var controls = [];
// ALGORITHM
var algorithm = form.comboBox({label: "Algorithm",
alternatives: ["CART", "Deep Learning", "Gradient Boosting", "Linear Discriminant Analysis",
"Random Forest", "Regression", "Support Vector Machine"],
name: "formAlgorithm", default_value: "Regression",
prompt: "Machine learning or regression algorithm for fitting the model"});
controls.push(algorithm);
algorithm = algorithm.getValue();
var regressionType = "";
if (algorithm == "Regression")
{
regressionTypeControl = form.comboBox({label: "Regression type",
alternatives: ["Linear", "Binary Logit", "Ordered Logit", "Multinomial Logit", "Poisson",
"Quasi-Poisson", "NBD"],
name: "formRegressionType", default_value: "Linear",
prompt: "Select type according to outcome variable type"});
regressionType = regressionTypeControl.getValue();
controls.push(regressionTypeControl);
}
// DEFAULT CONTROLS
missing_data_options = ["Error if missing data", "Exclude cases with missing data", "Imputation (replace missing values with estimates)"];
// AMEND DEFAULT CONTROLS PER ALGORITHM
if (algorithm == "Support Vector Machine")
output_options = ["Accuracy", "Prediction-Accuracy Table", "Detail"];
if (algorithm == "Gradient Boosting")
output_options = ["Accuracy", "Importance", "Prediction-Accuracy Table", "Detail"];
if (algorithm == "Random Forest")
output_options = ["Importance", "Prediction-Accuracy Table", "Detail"];
if (algorithm == "Deep Learning")
output_options = ["Accuracy", "Prediction-Accuracy Table", "Cross Validation", "Network Layers"];
if (algorithm == "Linear Discriminant Analysis")
output_options = ["Means", "Detail", "Prediction-Accuracy Table", "Scatterplot", "Moonplot"];
if (algorithm == "CART") {
output_options = ["Sankey", "Tree", "Text", "Prediction-Accuracy Table", "Cross Validation"];
missing_data_options = ["Error if missing data", "Exclude cases with missing data",
"Use partial data", "Imputation (replace missing values with estimates)"]
}
if (algorithm == "Regression") {
if (regressionType == "Multinomial Logit")
output_options = ["Summary", "Detail", "ANOVA"];
else if (regressionType == "Linear")
output_options = ["Summary", "Detail", "ANOVA", "Relative Importance Analysis", "Shapley Regression", "Jaccard Coefficient", "Correlation", "Effects Plot"];
else
output_options = ["Summary", "Detail", "ANOVA", "Relative Importance Analysis", "Effects Plot"];
}
// COMMON CONTROLS FOR ALL ALGORITHMS
var outputControl = form.comboBox({label: "Output", prompt: "The type of output used to show the results",
alternatives: output_options, name: "formOutput",
default_value: output_options[0]});
controls.push(outputControl);
var output = outputControl.getValue();
if (algorithm == "Regression") {
if (regressionType == "Linear") {
if (output == "Jaccard Coefficient" || output == "Correlation")
missing_data_options = ["Error if missing data", "Exclude cases with missing data", "Use partial data (pairwise correlations)"];
else
missing_data_options = ["Error if missing data", "Exclude cases with missing data", "Dummy variable adjustment", "Use partial data (pairwise correlations)", "Multiple imputation"];
}
else
missing_data_options = ["Error if missing data", "Exclude cases with missing data", "Dummy variable adjustment", "Multiple imputation"];
}
var missingControl = form.comboBox({label: "Missing data",
alternatives: missing_data_options, name: "formMissing", default_value: "Exclude cases with missing data",
prompt: "Options for handling cases with missing data"});
var missing = missingControl.getValue();
controls.push(missingControl);
controls.push(form.checkBox({label: "Variable names", name: "formNames", default_value: false, prompt: "Display names instead of labels"}));
// CONTROLS FOR SPECIFIC ALGORITHMS
if (algorithm == "Support Vector Machine")
controls.push(form.textBox({label: "Cost", name: "formCost", default_value: 1, type: "number",
prompt: "High cost produces a complex model with risk of overfitting, low cost produces a simpler mode with risk of underfitting"}));
if (algorithm == "Gradient Boosting") {
controls.push(form.comboBox({label: "Booster",
alternatives: ["gbtree", "gblinear"], name: "formBooster", default_value: "gbtree",
prompt: "Boost tree or linear underlying models"}));
controls.push(form.checkBox({label: "Grid search", name: "formSearch", default_value: false,
prompt: "Search for optimal hyperparameters"}));
}
if (algorithm == "Random Forest")
if (output == "Importance")
controls.push(form.checkBox({label: "Sort by importance", name: "formImportance", default_value: true}));
if (algorithm == "Deep Learning") {
controls.push(form.numericUpDown({name:"formEpochs", label:"Maximum epochs", default_value: 10, minimum: 1, maximum: Number.MAX_SAFE_INTEGER,
prompt: "Number of rounds of training"}));
controls.push(form.textBox({name: "formHiddenLayers", label: "Hidden layers", prompt: "Comma delimited list of the number of nodes in each hidden layer", required: true}));
controls.push(form.checkBox({label: "Normalize predictors", name: "formNormalize", default_value: true,
prompt: "Normalize to zero mean and unit variance"}));
}
if (algorithm == "Linear Discriminant Analysis") {
if (output == "Scatterplot")
{
controls.push(form.colorPicker({label: "Outcome color", name: "formOutColor", default_value:"#5B9BD5"}));
controls.push(form.colorPicker({label: "Predictors color", name: "formPredColor", default_value:"#ED7D31"}));
}
controls.push(form.comboBox({label: "Prior", alternatives: ["Equal", "Observed",], name: "formPrior", default_value: "Observed",
prompt: "Probabilities of group membership"}));
}
if (algorithm == "CART") {
controls.push(form.comboBox({label: "Pruning", alternatives: ["Minimum error", "Smallest tree", "None"],
name: "formPruning", default_value: "Minimum error",
prompt: "Remove nodes after tree has been built"}));
controls.push(form.checkBox({label: "Early stopping", name: "formStopping", default_value: false,
prompt: "Stop building tree when fit does not improve"}));
controls.push(form.comboBox({label: "Predictor category labels", alternatives: ["Full labels", "Abbreviated labels", "Letters"],
name: "formPredictorCategoryLabels", default_value: "Abbreviated labels",
prompt: "Labelling of predictor categories in the tree"}));
controls.push(form.comboBox({label: "Outcome category labels", alternatives: ["Full labels", "Abbreviated labels", "Letters"],
name: "formOutcomeCategoryLabels", default_value: "Full labels",
prompt: "Labelling of outcome categories in the tree"}));
controls.push(form.checkBox({label: "Allow long-running calculations", name: "formLongRunningCalculations", default_value: false,
prompt: "Allow predictors with more than 30 categories"}));
}
var stacked_check = false;
if (algorithm == "Regression") {
if (missing == "Multiple imputation")
controls.push(form.dropBox({label: "Auxiliary variables",
types:["Variable: Numeric, Date, Money, Categorical, OrderedCategorical"],
name: "formAuxiliaryVariables", required: false, multi:true,
prompt: "Additional variables to use when imputing missing values"}));
controls.push(form.comboBox({label: "Correction", alternatives: ["None", "False Discovery Rate", "Bonferroni"], name: "formCorrection",
default_value: "None", prompt: "Multiple comparisons correction applied when computing p-values of post-hoc comparisons"}));
var is_RIA_or_shapley = output == "Relative Importance Analysis" || output == "Shapley Regression";
var is_Jaccard_or_Correlation = output == "Jaccard Coefficient" || output == "Correlation";
if (regressionType == "Linear" && missing != "Use partial data (pairwise correlations)" && missing != "Multiple imputation")
controls.push(form.checkBox({label: "Robust standard errors", name: "formRobustSE", default_value: false,
prompt: "Standard errors are robust to violations of assumption of constant variance"}));
if (is_RIA_or_shapley)
controls.push(form.checkBox({label: "Absolute importance scores", name: "formAbsoluteImportance", default_value: false,
prompt: "Show absolute instead of signed importances"}));
if (regressionType != "Multinomial Logit" && (is_RIA_or_shapley || is_Jaccard_or_Correlation || output == "Summary"))
controls.push(form.dropBox({label: "Crosstab interaction", name: "formInteraction", types:["Variable: Numeric, Date, Money, Categorical, OrderedCategorical"],
required: false, prompt: "Categorical variable to test for interaction with other variables"}));
if (regressionType !== "Multinomial Logit")
controls.push(form.numericUpDown({name : "formOutlierProportion", label:"Automated outlier removal percentage", default_value: 0,
minimum:0, maximum:49.9, increment:0.1,
prompt: "Data points removed and model refitted based on the residual values in the model using the full dataset"}));
stacked_check_box = form.checkBox({label: "Stack data", name: "formStackedData", default_value: false,
prompt: "Allow input into the Outcome control to be a single multi variable and Predictors to be a single grid variable"})
stacked_check = stacked_check_box.getValue();
controls.push(stacked_check_box);
}
controls.push(form.numericUpDown({name:"formSeed", label:"Random seed", default_value: 12321, minimum: 1, maximum: Number.MAX_SAFE_INTEGER,
prompt: "Initializes randomization for imputation and certain algorithms"}));
var outcome = form.dropBox({label: "Outcome",
types: [ stacked_check ? "VariableSet: BinaryMulti, NominalMulti, OrdinalMulti, NumericMulti" : "Variable: Numeric, Date, Money, Categorical, OrderedCategorical"],
multi: false,
name: "formOutcomeVariable",
prompt: "Independent target variable to be predicted"});
var predictors = form.dropBox({label: "Predictor(s)",
types:[ stacked_check ? "VariableSet: BinaryGrid, NumericGrid" : "Variable: Numeric, Date, Money, Categorical, OrderedCategorical"],
name: "formPredictorVariables", multi: stacked_check ? false : true,
prompt: "Dependent input variables"});
controls.unshift(predictors);
controls.unshift(outcome);
form.setInputControls(controls);
form.setHeading((regressionType == "" ? "" : (regressionType + " ")) + algorithm);
library(flipMultivariates)
model <- MachineLearning(formula = if (isTRUE(get0("formStackedData"))) as.formula(NULL) else QFormula(formOutcomeVariable ~ formPredictorVariables),
algorithm = formAlgorithm,
weights = QPopulationWeight, subset = QFilter,
missing = formMissing,
output = if (formOutput == "Shapley Regression") "Shapley regression" else formOutput,
show.labels = !formNames,
seed = get0("formSeed"),
cost = get0("formCost"),
booster = get0("formBooster"),
grid.search = get0("formSearch"),
sort.by.importance = get0("formImportance"),
hidden.nodes = get0("formHiddenLayers"),
max.epochs = get0("formEpochs"),
normalize = get0("formNormalize"),
outcome.color = get0("formOutColor"),
predictors.color = get0("formPredColor"),
prior = get0("formPrior"),
prune = get0("formPruning"),
early.stopping = get0("formStopping"),
predictor.level.treatment = get0("formPredictorCategoryLabels"),
outcome.level.treatment = get0("formOutcomeCategoryLabels"),
long.running.calculations = get0("formLongRunningCalculations"),
type = get0("formRegressionType"),
auxiliary.data = get0("formAuxiliaryVariables"),
correction = get0("formCorrection"),
robust.se = get0("formRobustSE", ifnotfound = FALSE),
importance.absolute = get0("formAbsoluteImportance"),
interaction = get0("formInteraction"),
outlier.prop.to.remove = if (get0("formRegressionType", ifnotfound = "") != "Multinomial Logit") get0("formOutlierProportion")/100 else NULL,
stacked.data.check = get0("formStackedData"),
unstacked.data = if (isTRUE(get0("formStackedData"))) list(Y = get0("formOutcomeVariable"), X = get0("formPredictorVariables")) else NULL)
Further reading: Data Analysis Software