You can use the following methods to extract p-values for the coefficients in a linear regression model fit using the statsmodels module in Python:
#extract p-values for all predictor variables for x in range (0, 3): print(model.pvalues[x]) #extract p-value for specific predictor variable name model.pvalues.loc['predictor1'] #extract p-value for specific predictor variable position model.pvalues[0]
The following examples show how to use each method in practice.
Example: Extract P-Values from Linear Regression in Statsmodels
Suppose we have the following pandas DataFrame that contains information about hours studied, prep exams taken, and final score received by students in a certain class:
import pandas as pd #create DataFrame df = pd.DataFrame({'hours': [1, 2, 2, 4, 2, 1, 5, 4, 2, 4, 4, 3, 6], 'exams': [1, 3, 3, 5, 2, 2, 1, 1, 0, 3, 4, 3, 2], 'score': [76, 78, 85, 88, 72, 69, 94, 94, 88, 92, 90, 75, 96]}) #view head of DataFrame df.head() hours exams score 0 1 1 76 1 2 3 78 2 2 3 85 3 4 5 88 4 2 2 72
We can use the OLS() function from the statsmodels module to fit a multiple linear regression model, using “hours” and “exams” as the predictor variables and “score” as the response variable:
import statsmodels.api as sm #define predictor and response variables y = df['score'] x = df[['hours', 'exams']] #add constant to predictor variables x = sm.add_constant(x) #fit linear regression model model = sm.OLS(y, x).fit() #view model summary print(model.summary()) OLS Regression Results ============================================================================== Dep. Variable: score R-squared: 0.718 Model: OLS Adj. R-squared: 0.661 Method: Least Squares F-statistic: 12.70 Date: Fri, 05 Aug 2022 Prob (F-statistic): 0.00180 Time: 09:24:38 Log-Likelihood: -38.618 No. Observations: 13 AIC: 83.24 Df Residuals: 10 BIC: 84.93 Df Model: 2 Covariance Type: nonrobust ============================================================================== coef std err t P>|t| [0.025 0.975] ------------------------------------------------------------------------------ const 71.4048 4.001 17.847 0.000 62.490 80.319 hours 5.1275 1.018 5.038 0.001 2.860 7.395 exams -1.2121 1.147 -1.057 0.315 -3.768 1.344 ============================================================================== Omnibus: 1.103 Durbin-Watson: 1.248 Prob(Omnibus): 0.576 Jarque-Bera (JB): 0.803 Skew: -0.289 Prob(JB): 0.669 Kurtosis: 1.928 Cond. No. 11.7 ==============================================================================
By default, the summary() function displays the p-values of each predictor variable up to three decimal places:
- P-value for intercept: 0.000
- P-value for hours: 0.001
- P-value for exams: 0.315
However, we can extract the full p-values for each predictor variable in the model by using the following syntax:
#extract p-values for all predictor variables for x in range (0, 3): print(model.pvalues[x]) 6.514115622692573e-09 0.0005077783375870773 0.3154807854805659
This allows us to see the p-values to more decimal places:
- P-value for intercept: 0.00000000651411562269257
- P-value for hours: 0.0005077783375870773
- P-value for exams: 0.3154807854805659
Note: We used 3 in our range() function because there were three total coefficients in our regression model.
We can also use the following syntax to extract the p-value for the ‘hours’ variable specifically:
#extract p-value for 'hours' only model.pvalues.loc['hours'] 0.0005077783375870773
Or we could use the following syntax to extract the p-value for the coefficient of a variable in a specific position of the regression model:
#extract p-value for coefficient in index position 0 model.pvalues[0] 6.514115622692573e-09
Additional Resources
The following tutorials explain how to perform other common tasks in Python:
How to Perform Logistic Regression in Python
How to Calculate AIC of Regression Models in Python
How to Calculate Adjusted R-Squared in Python