Running a Chi-Square Test of Independence

For the purpose of running Chi Square Test of Independence
on the Gapminder dataset I am working on, as indicated in the assignment
instruction, I will have to categorize some of my Gapminder quantitative
variables into categorical variables.

In this sense, I have added another quantitative variable from the Gapminder data to the dataset csv file I will be working with; the new quantitative variable is exports.

The new exports variable:

exports: Exports of goods and services represent the value of all
goods and other market services provided to the rest of the world. They include
the value of merchandise, freight, insurance, transport, travel, royalties,
license fees, and other services, such as communication, construction,
financial, information, business, personal, and government services. They
exclude compensation of employees and investment income (formerly called factor services) and transfer payments.

 

It is this exports variable I will categorise to 2 level categorical response variable in order to be able to run Chi Square Test of Independence on my work.

I have subsequently, updated My Personal Codebook to include this new variable.

See updated codebook here:

https://docs.google.com/document/d/177YfOjdk4oekFu20OLt4fgmu-n7cgRULAJ_Kd9KdYaM/edit?usp=sharing

 

Secondly, the purpose of the Chi Square Test of Independence,
I have slightly changed the research question.

The new research question is:

Is there an association or relation between Inflation and Exports of Ghana?

 

Hypothesis Testing

The Null and Alternate Hypotheses:

From the above research question, the Null Hypothesis (H_o) is that there is no association / relation between Inflation and Exports of the people of Ghana.

Whereas the Alternate Hypothesis (H_a) states that there is an association /
relation between Inflation and Exports of the people of Ghana.

 

Sample:

Sample is the data from the Gapminder dataset with focus on Ghana

 

Assessing the evidence:

This is done by Running Chi Square Test of Independence on the hypotheses.

I do this by using running the test using the Python program.

 

MY PYTHON PROGRAM CODE:

# -*- coding: utf-8 -*-

"""

Created on Mon Jan  4 00:59:30 2016



@author: Bernard

"""

import pandas

import scipy.stats

import seaborn

import matplotlib.pyplot as plt





#load the gapminder_ghana_updated dataset csv into the program

data = pandas.read_csv('gapminder_ghana_updated.csv', low_memory = False)





#Converting data to numeric

data['incomeperperson'] = pandas.to_numeric(data['incomeperperson'],
errors='coerce')

data['lifeexpectancy'] = pandas.to_numeric(data['lifeexpectancy'],
errors='coerce')

data['literacyrate'] = pandas.to_numeric(data['literacyrate'],
errors='coerce')

data['Inflation'] = pandas.to_numeric(data['Inflation'], errors='coerce')

data['exports'] = pandas.to_numeric(data['exports'], errors='coerce')





#create a variable for inflationCategory

data["inflationCategory"] = data["Inflation"]





#categorical groupings for inflation. This is to get one categorical
variable for the

#Chi Square test

data["inflationCategory"] = pandas.cut(data.inflationCategory,
[-4, 32, 64, 96, 128])





#create a variable for exportsCategory

data["exportsCategory"] = data["exports"]





#categorical groupings for exports. this is to get a two level
categorical varible

# for the CHI SQUARE Test

data["exportsCategory"] =
pandas.qcut(data["exports"], 2)







#including only data relevant for our testing by droping irrelavant data

dataSub = data[["inflationCategory",
"exportsCategory"]].dropna()





#describe inflation category

print("describe inflation Category")

desc1 = dataSub["inflationCategory"].describe()

print(desc1)



print("")

print("")



#inflationCategory count

print("inflation category")

c1 = dataSub["inflationCategory"].value_counts(sort=False,
dropna=True)

print(c1)



print("")

print("")



#describe exportsCategory

print("describe Exports Category")

desc1 = dataSub["exportsCategory"].describe()

print(desc1)





print("")

print("")





#Exports category count

print("Exports category")

c1 = dataSub["exportsCategory"].value_counts(sort=False,
dropna=True)

print(c1)



print("")

print("")





# contingency table of observed counts

count1=pandas.crosstab(dataSub['exportsCategory'],
dataSub['inflationCategory'])

print (count1)





print("")

print("")





# column percentages

colmnSum=count1.sum(axis=0)

colPercent=count1/colmnSum

print(colPercent)



print("")

print("")



# chi-square

print ('chi-square value, p value, expected counts')

chiSq1= scipy.stats.chi2_contingency(count1)

print (chiSq1)



print("")

print("")





#Change format of inflationCategory from numberic to categorical

dataSub["inflationCategory"] =
dataSub["inflationCategory"].astype("category")



# make exportsCategory numberic

dataSub["exportsCategory"] =
pandas.to_numeric(dataSub['exportsCategory'], errors='coerce')







# graph percent with export level within each inflation group

seaborn.factorplot(x="inflationCategory",
y="exportsCategory", data=dataSub, kind="bar", ci=None)

plt.xlabel("Inflation group level ")

plt.ylabel("Proportion Export level")





#compare [-4, 32] and [32, 64]

recode1 = {[-4, 32]: [-4, 32], [32, 64]: [32, 64]}

dataSub['COMP-4v32']= dataSub['inflationCategory'].map(recode1)







# contingency table of observed counts

count2=pandas.crosstab(dataSub['inflationCategory'],
dataSub['COMP-4v32'])

print (count2)







# column percentages

colmnSum2=count2.sum(axis=0)

columnPerc2=count2/colmnSum2

print(columnPerc2)





print ('chi-square value, p value, expected counts')

chis2= scipy.stats.chi2_contingency(count2)

print (chis2)





#compare [-4, 32] and [64, 96]

recode2 = {[-4, 32]: [-4, 32], [64, 96]: [64, 96]}

dataSub['COMP-4v64']= dataSub['inflationCategory'].map(recode2)



# contingency table of observed counts

count2=pandas.crosstab(dataSub['inflationCategory'],
dataSub['COMP-4v64'])

print (count2)



# column percentages

colmnSum2=count2.sum(axis=0)

columnPerc2=count2/colmnSum2

print(columnPerc2)



print ('chi-square value, p value, expected counts')

chis2= scipy.stats.chi2_contingency(count2)

print (chis2)







#compare [-4, 32] and [96, 128]

recode3 = {[-4, 32]: [-4, 32], [96, 128]: [96, 128]}

dataSub['COMP-4v96']= dataSub['inflationCategory'].map(recode3)



# contingency table of observed counts

count3=pandas.crosstab(dataSub['inflationCategory'],
dataSub['COMP-4v96'])

print (count3)



# column percentages

colmnSum3=count3.sum(axis=0)

columnPerc3=count3/colmnSum3

print(columnPerc3)



print ('chi-square value, p value, expected counts')

chis3= scipy.stats.chi2_contingency(count3)

print (chis3)





#compare [32, 64] and [64, 96]

recode4 = {[32, 64]: [32, 64], [64, 96]: [64, 96]}

dataSub['COMP32v64']= dataSub['inflationCategory'].map(recode4)



# contingency table of observed counts

count4=pandas.crosstab(dataSub['inflationCategory'],
dataSub['COMP32v64'])

print (count4)



# column percentages

colmnSum4=count4.sum(axis=0)

columnPerc4=count4/colmnSum4

print(columnPerc4)



print ('chi-square value, p value, expected counts')

chis4= scipy.stats.chi2_contingency(count4)

print (chis4)





#compare [32, 64] and [96, 128]

recode5 = {[32, 64]: [32, 64], [96, 128]: [96, 128]}

dataSub['COMP32v96']= dataSub['inflationCategory'].map(recode5)



# contingency table of observed counts

count5=pandas.crosstab(dataSub['inflationCategory'],
dataSub['COMP32v96'])

print (count5)



# column percentages

colmnSum5=count5.sum(axis=0)

columnPerc5=count5/colmnSum5

print(columnPerc5)



print ('chi-square value, p value, expected counts')

chis5= scipy.stats.chi2_contingency(count5)

print (chis5)







#compare [64, 96] and [96, 128]

recode6 = {[64, 96]: [64, 96], [96, 128]: [96, 128]}

dataSub['COMP64v96']= dataSub['inflationCategory'].map(recode6)



# contingency table of observed counts

count6=pandas.crosstab(dataSub['inflationCategory'],
dataSub['COMP64v96'])

print (count6)



# column percentages

colmnSum6=count6.sum(axis=0)

columnPerc6=count6/colmnSum6

print(columnPerc6)



print ('chi-square value, p value, expected counts')

chis6= scipy.stats.chi2_contingency(count6)

print (chis6)

 

 

 

CODE OUTPUT:

<<<<<<<<<<<<< CODE OUTPUT BEGIN >>>>>>>>>>>>>>>>>>>

describe inflation Category

count           51

unique           4

top       (-4, 32]

freq            37

Name: inflationCategory, dtype:
object

 

 

inflation category

(-4, 32]     37

(32, 64]      9

(64, 96]      4

(96, 128]     1

dtype: int64

 

 

describe Exports Category

count                  51

unique                  2

top       [3.338, 20.254]

freq                   26

Name: exportsCategory, dtype:
object

 

 

Exports category

[3.338, 20.254]     26

(20.254, 48.802]    25

dtype: int64

 

 

inflationCategory  (-4, 32]
(32, 64]  (64, 96]  (96, 128]

exportsCategory

[3.338, 20.254]          16         6         3          1

(20.254, 48.802]         21         3         1          0

 

 

inflationCategory  (-4, 32]
(32, 64]  (64, 96]  (96, 128]

exportsCategory

[3.338, 20.254]    0.432432
0.666667      0.75          1

(20.254, 48.802]   0.567568
0.333333      0.25          0

 

 

chi-square value, p value,
expected counts

(3.6574740124740135,
0.30090520911852703, 3, array([[ 18.8627451 ,
4.58823529,   2.03921569,   0.50980392],

[ 18.1372549 ,   4.41176471,
1.96078431,   0.49019608]]))

 

 

please find graph for analysing further the values

https://drive.google.com/file/d/0B2KfPRxy4ootVzR3R3RoNW9iOEk/view?usp=sharing

<<<<<<<<<<<<<CODE OUTPUT ENDED>>>>>>>>>>>>>>>>>>>

 

 

DRAWING CONCLUSION (SUMMARY):

Model Interpretation for Chi-Square Tests:

When examining the relation /association between Inflation (categorical explanatory variable) and Exports (categorical response variable) of Ghana, we can see that the p-value from the Chi-Square Test results is p= 0.3009 with an associated chi-square value, X² = 3.657

 

From this p-value and its associated chi-square value the test is significant and therefore I will accept the Null Hypothesis (H_o) that there is no association / relation between Inflation and Exports of the people of Ghana

 

Model Interpretation for post hoc Chi-Square Test results:

However, since there are different categories of inflation rate, to reduce a Type Error between each of the various exports levels with respect to the inflation

I ran post hoc Chi-Square Test for each of the values of the explanatory variable – inflation. Hence two by two Chi-Square Tests were run for each of the values in my explanatory categorical variable.
There were a total of six combinations of these individual tests.

With the help of the Bonferroni Adjustment test method most of these compared variables were not very far or different from each other. Hence I will accept the Null Hypothesis (H_o) that there is no association / relation between
Inflation and Exports of the people of Ghana.