Running a Classification Tree – Decision Tree – Data Analysis and Intrepretation

Bydatapandasadmin March 23, 2016October 23, 2016

Overview

My research work deals with Ghana, a country from the Gapminder dataset as has already been discussed from the beginning and progression through this course.

Picture
Of My Final Decision Tree Outputted From My Python Program

The variables in my observation dataset
are all quantitative.

For the purposes of this assignment, I have binned my quantitative target variable, Life Expectancy (lifeexpectancy) into a 2-level binary categorical target variable. I have named this categorical target variable, lifeExpectancyCat. It has been coded as 0 = low life expectancy and 1 = high life expectancy

I have also binned all the other predictor variables categorical variables for the purpose of this assignment.

Thus these are the respective categorical end result variables of my quantitative predictor variables:

Quantitative Binary Categorical variable

incomeperperson – incomeLevelGrp

inflation – inflationCatGrp

exports – exportsCatGrp

However, I have only used 2 of these variables, namely, incomeLevelGrp and exportsCatGrp predictor variables in classifying my tree.

Decision tree analysis was performed to test nonlinear relationships among my 2 explanatory variables and my binary, categorical response variable. All possible
separations (categorical) are tested.

From my decision, it first starts with a split on X[1] which is exportsCatGrp my
second explanatory variable. If the value for exportsCatGrp is less than 0.5, that is low exports as my exportsCatGrp has values of 0 = low exports and 1 = high exports, then the observation moves to the left side of the split and includes 28 of the 30 observations in the training sample.

From this node another split is made on incomeLevelGrp which is X[0] so that I have periods with low exports and high incomes and also periods with low exports and low income levels.

From the resulting leaf that has periods with low exports and high incomes, which shown to the right of the split from X[0], 7 of the people of Ghana have High Life Expectancy and 7 also have Low Life Expectancy.

From the resulting leaf that has periods with low exports and low incomes, 8 of the people of Ghana have High Life Expectancy and 6 have Low Life Expectancy.

However, if the value for exportsCatGrp, X[1] is greater than 0.5 that is high exports, then the observation moves to the right hand side of the split and
includes 2 of the 30 observations in the training sample.

From this, a decision tree leaf is created and from this leaf, 0 person(s), that is no one, has High Life Expectancy rate and 2 people have Low Life Expectancy.

Hence from the decision tree classification, periods with low exports and low incomes are more likely to have High Life Expectancy than periods of high exports and high income levels. (which is rather surprising!)

Requesting the shape of my predictor training sample it can be seen that it has 30 observation or rows which represents 60% of our original sample and 2 explanatory variables as indicated above.

The test sample has 21 observation or rows which is 40% of
the original sample and 2 explanatory variables or columns

A Confusion Matrix was used to estimate the prediction accuracy of my model. From the results of the Confusion Matrix, the models accurately classified 15 of the total 21 observations included in my data set for the classification and misclassified 6 of the 21 observations included in the classification process. This means the model correctly classified 71% of the observations as having High Life
Expectancy or Low Life Expectancy and misclassified 29% of the observations in
my data set.

The Confusion Matrix result can be seen below:

This can be interpreted as the model having low prediction error as it correctly classified high percentage of the observations and misclassified low percentage of the observations in my dataset

This can further be seen by running a Test Accuracy score on
my model and the result was 0.7142857142857143 (71%) as can also be seen below:

################################

PYTHON CODE

#################################

################################

CODE OUTPUT

#################################

Want more information like this?

Coursera | Data Science

Running An Analysis of Variance (ANOVA) – Data Analysis Tools
Bydatapandasadmin January 23, 2016October 23, 2016

Chosen Dataset I will be working with Data from the Gapminder dataset. This happens to be the same dataset I worked with under the Data Management and Visualization course assignments. As elaborated and discussed in under the Data Management and Visualization course assignments, I have chosen to focus on the country, Ghana. Hence I will be particularly interested…

Read More Running An Analysis of Variance (ANOVA) – Data Analysis Tools

Python

Python PyQt Gui Basics Button Window PyQt4 Pt1

Bydatapandasadmin September 28, 2016October 23, 2016

This is just some quick scrap notes on the basics of Python GUI building using PyQt. In [1]:

import sys

from PyQt4 import QtGui, QtCore # to help add buttons

#app = QtGui.QApplication(sys.argv)

#window = QtGui.QWidget()

##set the posstion and size of the window

#window.setGeometry(50, 50, 500, 500)

##lets set the title

#window.setWindowTitle('Date Incrementor')

#window.show()

class Window(QtGui.QMainWindow):

def __init__(self):

super(Window, self).__init__()

self.setGeometry(50,50, 500, 500)

self.setWindowTitle("Date incrementor")

# self.setWindowIcon(QtGui.QIcon('logo.png'))

#self.show()

self.home()

def home(self):

#create and name a button

btn = QtGui.QPushButton("Quit", self)

#define what we will do when the button is clicked

#btn.clicked.connect(QtCore.QCoreApplication.instance().quit)

btn.clicked.connect(self.closeapplication)

#lets resisze the button

btn.resize(100,100)

#lets position the button

btn.move(100,100)

#lets show the button

self.show()

#lets create custom methods

def closeapplication(self):

print('Application closed')

sys.exit()

the ‘init‘ statements runs anytime we call call the Window class we have created and it runs all the code in the Window class the _init__ method calls the super constructor and then sets…

Python

Getting Stock Prices from Yahoo and plotting Python 3 Matplolib Urllib

Bydatapandasadmin September 3, 2016October 23, 2016

This is some quick notes about getting stock data from Yahoo and plotting it using Matplotlib . The Python version used is Python 3.5 Credits to sentdex. You can check him out on Youtube. In [11]:

import matplotlib.pyplot as plt

import numpy as np

import urllib

import matplotlib.dates as mdate

import urllib.request

%matplotlib inline

#%matplotlib qt

#%matplotlib notebook

# interactive popout window plot

In [12]:

#getting data from the internet

#define our converter function

def bytespdate2num(fmt, encoding='utf-8'):

strconverter = mdate.strpdate2num(fmt)

def bytesconverter(b):

s = b.decode(encoding)

return strconverter(s)

return bytesconverter

In [13]:

#define a function which will take the stock data and plot the graph

def graph_data(stock):

stock_data_url = 'http://chartapi.finance.yahoo.com/instrument/1.0/'+stock+'/chartdata;type=quote;range=10y/csv'

source_code = urllib.request.urlopen(stock_data_url).read().decode()

stock_data =[]

split_source = source_code.split('\n') #split the code based on next line

#lets grab only the relevant stock data

for line in split_source:

split_line = line.split(',')

if(len(split_line))==6:

if 'values' not in line and 'labels' not in line:

stock_data.append(line)

date, closep, highp, lowp, openp, volume = np.loadtxt(stock_data, delimiter=',',unpack=True,

converters={0: bytespdate2num('%Y%m%d')})

plt.plot_date(date,closep,'-', label='Price')

plt.xlabel('Date')

plt.ylabel('Price')

plt.title('Interesting stock graph from yahoo')

plt.legend()

In [14]:

1 2	#lets plot TSLA stock prices. graph_data('TSLA')

Want more information like this?

Business Intelligence - BI | Data Science | Python

Save Multiple Pandas DataFrames to One Single Excel Sheet Side by Side or Dowwards – XlsxWriter

Bydatapandasadmin October 5, 2016October 23, 2016

This tutorial is just to illustrate how to save Python Pandas dataframe into one excel work SHEET . You can save it column-wise, that is side by side or row-wise, that is downwards, one dataframe after the other. In [110]:

import sqlalchemy

import pyodbc

from pandas import DataFrame

from bokeh.plotting import figure, output_file, show

import pandas as pd

In [111]:

df = DataFrame

## connect to SqlServer Database and get information.

try:

import urllib

params = urllib.quote_plus("DRIVER={SQL Server Native Client 11.0};SERVER=.\MSSQLSERVER_ENT;DATABASE=Antrak;Trusted_Connection=yes;")

engine = sqlalchemy.create_engine('mssql+pyodbc:///?odbc_connect=%s"' % params)

resoverall = engine.execute("SELECT top 10 * FROM FlightBookingsMain")

df = df(resoverall.fetchall())

df.columns = resoverall.keys()

except (RuntimeError, TypeError, NameError):

print('Error in Conneccting')

print(RuntimeError, TypeError, NameError)

finally:

print("connected")

connected

In [112]:

1 2	#lets look at the head of the dataframe df.head()

Out[112]: First Name Last Name…

Python

Python Pandas Groupby function agg Series GroupbyObject
Bydatapandasadmin September 9, 2016March 4, 2017

Group By FunctionThis is a quick look at Python groupby function. Very powerful and useful function. We will take a simple look at it here. Credits to Data School , creator of Python course materials. lets import sample dataset In [18]:

Default

1

import pandas as pd

In [19]:

Default

1

drinks = pd.read_csv('http://bit.ly/drinksbycountry')

In [20]:

Default

1
2

#lets check the head
drinks.head()

Out[20]: country beer_servings spirit_servings wine_servings total_litres_of_pure_alcohol continent…

Read More Python Pandas Groupby function agg Series GroupbyObject

Python

Matplotlib Pyplot Plt Python Pandas Data Visualization Plotting

Bydatapandasadmin August 26, 2016October 23, 2016

This is some quick notes about graphing or plotting graphs with Matplotlib in Python. Credits to sentdex. You can check him out on Youtube.

import matplotlib.pyplot as plt

%matplotlib inline

# simple syntax is matplotlib plots as plt.plot(x, y)

plt.plot([1,2,3], [5,6,7])

plt.show()

In [4]:

# plotting with variables and basic labelling of our plot

x = [1,2,3]

y = [5,6,7]

##adding a legend to the plot

#add another line vectors first

x2 = [4,6,8]

y2= [10,14,18]

plt.plot(x, y, label='First Line') # you have to give the plots a label before you can call the legends on them

plt.plot(x2, y2, label='Second Line')

plt.xlabel('Plot number')

plt.ylabel('Important var')

plt.title('Plotting\nNice Graph') #\nGives you a sub title

plt.legend() #call the legend

plt.show()

In [10]:

#plotting barcharts and histograms

x3 = [2,4,6,8,10]

y3 = [3,5,7,9,11]

#second set of bars

x4 = [1,3,5,7,9]

y4 = [2,6,6,12,4]

#plot barchart

plt.bar(x3, y3, label='Bar1', color='r')

plt.bar(x4,y4, label='Bar2', color ='c')

plt.xlabel('x3')

plt.ylabel('y3')

plt.title('Bar Chart')

plt.legend()

plt.show()

In [14]:

#plotting histogram.

population_dist = [24,45,5,6,42,65,34,75,75,34,75,7,24,26,87,98,98,67,86, 65,63, 87,98,76,53,35]

#ids = [x for x in range(len(population_dist))] for bar chart plotting

#plot barchart

#plt.bar(ids, population_dist, label='hist')

#histogram works with bins so we will have to create bin 'containers' for our distribution

bins = [10,20,30,40,50,60,70,80,90,100]

plt.hist(population_dist, bins, histtype='bar', rwidth=0.5)

plt.xlabel('x3')

plt.ylabel('y3')

plt.title('Histogram')

plt.legend()

plt.show()

In [20]:

## scatter plot.

#basic skeletion

x = [1,2,3,4,5,6,7,8,9]

y = [4,5,8,3,6,9,5,4,8]

plt.scatter(x,y, label='ScatterShow', color='c', marker='^' , s=60) # s = size of the marker

plt.xlabel('x3')

plt.ylabel('y3')

plt.title('Scatter')

plt.legend()

plt.show()

In [23]:

#stacked plot

days = [1, 2, 3, 4, 5]

sleeping = [6,8,7,3,8]

eating = [6,9,7,3,6]

working = [9,8,5,7,8]

#working around legends for stackplots

plt.plot([],[], color='red', label='eating', linewidth=2)

plt.plot([],[], color='blue', label='sleeping', linewidth=2)

plt.plot([],[], color='yellow', label='working', linewidth=2)

plt.stackplot(days, eating, sleeping, working, color=['red','blue','yellow'])

plt.xlabel('x3')

plt.ylabel('y3')

plt.title('Stacked Plot')

plt.legend()

plt.show()

In [30]:

#pie charts

days = [1, 2, 3, 4, 5]

sleeping = [6,8,7,3,8]

eating = [6,9,7,3,6]

working = [9,8,5,7,8]

slices = [2,4,6,8,1]

colors = ['red','blue','green','brown','pink']

activities = ['sleeping','eating','working','dancing','travelling']

plt.pie(slices, labels=activities, colors=colors, startangle=90, shadow=True, explode=(0,0.1,0,0,0), autopct='%1.1f%%')

Out[30]:

([<matplotlib.patches.Wedge at 0x814e588>,

<matplotlib.patches.Wedge at 0x7fb7080>,

<matplotlib.patches.Wedge at 0x96f9ba8>,

<matplotlib.patches.Wedge at 0x82853c8>,

<matplotlib.patches.Wedge at 0x785a080>],

[<matplotlib.text.Text at 0x82ac4a8>,

<matplotlib.text.Text at 0x96f9438>,

<matplotlib.text.Text at 0x8026390>,

<matplotlib.text.Text at 0x94ac400>,

<matplotlib.text.Text at 0x785ada0>],

[<matplotlib.text.Text at 0x7fb7fd0>,

<matplotlib.text.Text at 0x96f97b8>,

<matplotlib.text.Text at 0x8285f98>,

<matplotlib.text.Text at 0x94aca58>,

<matplotlib.text.Text at 0x65bf860>])

In [48]:

#modifying the matplotlib

#plotting barchart.

x3 = [2,4,6,8,10]

y3 = [3,5,7,9,11]

#second set of bars

x4 = [1,3,5,7,9]

y4 = [2,6,6,12,4]

fig = plt.figure(facecolor='#f0f0f0') #set facecolour or background colour which will be maintained once we save our graph

ax1 = plt.subplot2grid((1,1),(0,0))

#tilt the xlabels

for labels in ax1.xaxis.get_ticklabels():

labels.set_rotation(45)

#plot barchart

ax1.bar(x3, y3, label='Bar1', color='r')

ax1.bar(x4,y4, label='Bar2', color ='c')

ax1.grid(True) #add a grid

plt.xlabel('x3')

plt.ylabel('y3')

plt.title('Bar Chart')

plt.legend()

#customise the legend

leg = ax1.legend(loc=2,ncol=6, prop={'size':10})

leg.get_frame().set_alpha(0.4)

##define spacing around the ticks and labels

plt.subplots_adjust(left = 0.09, bottom=0.20, right=0.94, top= 0.93, wspace = 0.2, hspace = 0)

plt.show()

#lets save the graph

fig.savefig('matplotlib.png', facecolor=fig.get_facecolor())

Want more information like this?

Running a Classification Tree – Decision Tree – Data Analysis and Intrepretation

Running An Analysis of Variance (ANOVA) – Data Analysis Tools

Python PyQt Gui Basics Button Window PyQt4 Pt1

Getting Stock Prices from Yahoo and plotting Python 3 Matplolib Urllib

Save Multiple Pandas DataFrames to One Single Excel Sheet Side by Side or Dowwards – XlsxWriter

Python Pandas Groupby function agg Series GroupbyObject

Matplotlib Pyplot Plt Python Pandas Data Visualization Plotting

Leave a Reply Cancel reply

DataPandas LTS

EXPLORE DataPandas

ImportAnt link

GET IN TOUCH

© 2026 DataPandas

Similar Posts

Leave a Reply Cancel reply

DataPandas LTS

EXPLORE DataPandas

ImportAnt link

GET IN TOUCH

© 2026 DataPandas