ICT110 Introduction to Data Science

Student name
USC Student ID xxxxxx
A Data Analysis Report of the
Underlying Causes of Death in
Queensland (1997 – 2017)
ICT110 Introduction to Data Science
Semester x, year
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
i
Contents
1.0 Introduction ………………………………………………………………………………………………………….. 1
1.1 Authorisation and Purpose …………………………………………………………………………………. 1
1.2 Limitations ………………………………………………………………………………………………………… 1
1.3 Scope ……………………………………………………………………………………………………………….. 1
2.0 Data Setup ……………………………………………………………………………………………………………. 1
3.0 Exploratory Data Analysis ……………………………………………………………………………………….. 5
3.1 Univariate Analyses ……………………………………………………………………………………………. 5
3.1.1 Univariate Analysis 1: Mental and Behavioural Disorders …………………………………. 5
3.1.2 Univariate Analysis 2: Heart Disease ………………………………………………………………. 7
3.2 Bivariate Analyses ……………………………………………………………………………………………… 8
3.2.1 Bivariate Analysis 1: Years and Diseases of the Nervous System ……………………….. 8
3.2.2 Bivariate Analysis 2: Female and Male Genital Cancers ………………………………….. 10
4. Advanced Analysis …………………………………………………………………………………………………. 11
4.1 Clustering ………………………………………………………………………………………………………… 11
4.1.1 Brief Explanation of Cluster Analysis and K-Means ………………………………………… 11
4.1.2 Clustering Analysis: Grouping Years According to Influenza Deaths …………………. 11
4.2 Linear Regression …………………………………………………………………………………………….. 12
4.2.1 Brief Explanation of Linear Regression …………………………………………………………. 12
4.2.2 Simple Linear Regression 1: Drownings Predicted by Melanoma rates …………….. 13
4.2.3 Simple Linear Regression 2: Years Predicting Disorder Fatality Rate ………………… 14
5. Conclusion ……………………………………………………………………………………………………………. 16
6. Reflections ……………………………………………………………………………………………………………. 16
REFERENCES …………………………………………………………………………………………………………….. 18
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 1
1.0 Introduction
1.1 Authorisation and Purpose
This report has been prepared to deliver the findings from analysis of data produced by the
Queensland government pertaining to the causes underlying deaths in the state of
Queensland between the years of 1997 and 2017, inclusive, and the work undertaken in its
preparation and analysis.
No specific goal has been stated for the analysis. This report is intended for researchers,
representatives of business or government agencies with interest in the subject of this data.
1.2 Limitations
Due to this data being published with no consideration of it being imported into an analysis
package such as R, some visual observations of the data are required to ascertain which data
within the file is of sufficient quality and consistency for analysing, and then values or subsets
of insufficient quality will be excluded through necessity.
This report is not intended as comprehensive and will focus on a few causes contained within
one set of data at most. A limiting factor within the data supplied is the absence of other data,
such as the yearly population of the state, which would reveal whether population change
correlates with observed changes within the dataset under scrutiny, for example.
1.3 Scope
For this analysis, a dataset of recorded values showing the causes underlying the deaths of
Queenslanders and their frequency per year which was published in 2017 by the Australian
Bureau of Statistics has been selected.
The health development of Queensland from 1997 until 2017 shown within the data will be
used for analysis and consideration will be given to the findings. The data will first undergo
pre-processing and the constructed data tables loaded into R for statistical analysis in both
univariate and bivariate sets. Appropriate graphs will be produced with R programming, and
the code used will be shown. Advanced analysis will follow in the form of an explanation of
clustering and k-means before cluster analysis of prepared data to group years according to a
cause. Further advanced analysis will then be performed through linear regression following
a brief explanation of the concept.
After conclusions have been drawn, the findings will be reflected upon.
2.0 Data Setup
The file “underlying-cause-death-qld.csv” is loaded into Microsoft Excel, and preliminary ideas
for analysis are formed.
An initial look at the data as supplied shows it to be in a state that would be difficult to load
into R (see figure 1), as it contains many comments and a variety of blank or non-useful cells,
so a clean-up of underlying-cause-death-qld.csv will have to occur manually, with Excel. Note
is made of the numerical values formatted to have a thousands column separator of a comma,
which would force R to consider the values as text, rather than numbers.
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 2
In the file, there are some explanatory notes following some of the values in the “Year”
headings. These are removed (see figure 2).
Figure 1: Initial view of data table
Figure 2: Excess characters are trimmed
Figure 3: Unhelpful rows are removed
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 3
Rows 19, 20 and 31 in the table contain erratic or unhelpful data (see figure 3), and so are
removed along with a value in a cell that is ‘floating’ amongst the data and will not assist
analysis (figure 4).
Within the structure of the data file, some cell contents require editing to improve the layout
for our needs (figure 5).
A set of values of interest is chosen and transposed into a column with a header on a unique
worksheet, as shown in figure 6.
This is followed by a second set,
as two univariate analyses will be
performed.
These columns of values have
unnecessary commas in the values,
so the number format is changed
to “number” with no commas and
no places after the decimal, as all
the values given are integers (see
figure 7).
Figure 4: An aberrant cell is removed
Figure 5: Column heading moved to assist transposition
Figure 6: Rows are transposed to columns of values with a heading
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 4
The 21 observations listed for heart disease
deaths, and deaths attributed to mental and
behavioural disorders are chosen for univariate
analysis.
Initial observation suggests a near linear change in
the values.
In section 3.2, exploratory bivariate analysis will be
conducted, and so datasets are prepared for this
purpose.
The first dataset created has the ordinal
categorical values of each year in the record, listed
with the discrete quantitative values for deaths
related to diseases affecting the nervous system in
each of those years.
The second two-variable dataset contains the
observations for cancers of the reproductive
organs, with the variables of female and male.
All of these separated lists are exported from Excel
as comma-separated value files.
Also exported are the bivariate yearly data
showing deaths caused by influenza for use in
cluster analysis in section 4.1 and a bivariate set
for linear regression analysis in section 4.2
In R, the directory to where the .csv files have been exported to is set to be the working
directory, containing all project data files for analysis by R.
Figure 7: Number formatting is set
Figure 8: Bivariate datasets
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 5
> # Set a working directory for this project, where the datasets for analysis
are stored:
> setwd(“S:/ICT110 Task 2 Data Analysis/R Work”)
Any references within R to files or directories are all relative to this working directory.
To begin analysis, the first two datasets are loaded into R, and assigned to variables.
The R function read.table() has a variant read.csv() to streamline reading .csv files:
> # load prepared data from .csv files and create R datasets
> disorder1 <- read.csv(“MentalDisorders.csv”, header = TRUE)
> disease2 <- read.csv(“HeartDisease.csv”, header = TRUE)
3.0 Exploratory Data Analysis
3.1 Univariate Analyses
3.1.1 Univariate Analysis 1: Mental and Behavioural Disorders
Now that the data has been made more tidy, exploratory data analysis can begin in order to
both get information from and gain some intuitive sense of the data (Pathak 2014).
In order to check the success of loading and see that the values are correctly represented, the
R command head() is used.
> # a quick check of the dataset to see that R is ready.
> head (disorder1)
Mental.and.behavioural.disorders
1 319
2 373
3 406
4 352
5 388
6 468
> # and the second loaded dataset:
> # Quickly examining the first 10 values:
> head (disease2,10)
Heart.Disease
1 5362
2 5319
3 5315
4 5011
5 5040
6 5067
7 4850
8 4904
9 4540
10 4698
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 6
A brief summary of the datasets is obtained with the R command summary().
> # obtain a brief summary of the datasets:
> summary (disorder1)
Mental.and.behavioural.disorders
Min. : 319
1st Qu.: 468
Median : 826

>
> summary (disease2)
Heart.Disease

This gives an idea of the range of the data. The previously observed potential for linear
relationship recommends a barplot of the disorder1 data to confirm this idea.
Adding some titling and axis labels makes the plotted graph more informative, and so the
names (years) observed during the initial views of this data are added to the graph:
> # for the 1st analysis, a barplot will best show the relationship over the 21
observations.
> # concatenate and assign some values to a variable used as names for the plot,
> # in this case, the names are the years involved:
> noms_de_plot <- c(1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006,
2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017)
>
> # now a plot of the first to twenty-first values from the first column:
> barplot (disorder1[1:21,1], names.arg=noms_de_plot, xlab = “Year”, ylab =
“Deaths”, main = “Qld. Deaths from Mental Disorders”, col=”Purple”)
Figure 9: Bar graph of disorder1 vector
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 7
Each bar across the horizontal (“X”) axis represents one of the sampled years, with the
recorded number of deaths on the vertical (“Y”) axis so the height of each bar shows the value
for that year. With the values plotted against each other and scaled relative to the range of
zero to the maximum in the vector, a relationship can be seen among the values. This bar
graph reveals an almost linear increase in deaths due to mental disorders in the 21 years of
data, with an increase from the previous value for most years. As given in the summary, the
number has jumped from 319 in 1997 to 1,921 in 2017, which is an increase of 602%.
> # Calculate a percentage change between the final and first values:
> 1921 / 319
[1] 6.021944
3.1.2 Univariate Analysis 2: Heart Disease
The second univariate analysis is of the data for deaths caused by Ischaemic heart disease.
> # a boxplot of the second set, to show the median and quartiles:
> boxplot.default(disease2[,1],ylab = “Deaths”, main = “# of Qld. Deaths from
Ischaemic Heart Failures 1997 – 2017″, col=”Dark Red”)
In this boxplot (figure 10), the range of values in the vector are shown from a maximum
approaching 5400 deaths and a minimum appearing near 3800, and the median displayed at
just over 4500, agreeing with the previous summary. The median is the 50th percentile, at
which the range has half of the values greater and half lesser. Within the boxed area are the
50th-75th percentile values are represented above the median and the 25th-50th below the
line. Outside the boxed area are the lower 1st-25th percentile (1st quartile) and 75th-100th
percentile (4th quartile) above the box, ending at the whiskers. A single variable boxplot such
as this shows some properties of the data, however when displaying this vector in this way, it
does not lend much to discussion as there is nothing correlated to the range of numbers. From
this plot however, an idea of the distribution of the values within the vector can be found and
so it is a useful tool for awareness at this initial exploratory stage of the analysis.
Median
Figure 10: Boxplot of disease2 vector
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 8
The change over the time of observation can once again be calculated by R:
> # Calculate a percentage change between the first and final values:
> disease2[21,]/disease2[1,]
[1] 0.7403954
A reduction to 74% of the initial amount is revealed.
In order to gain a contrast for further discussion, it is a simple matter to use R’s command
history to recall the settings for the first barplot and edit to represent the second dataset:
# more revealing as an “apples to apples” comparison:
barplot (disease2[1:21,1], names.arg=noms_de_plot, xlab = “Year”, ylab =
“Deaths”, main = “Qld. Deaths from Heart Disease”, col=”Dark Red”)
It can be seen in figure 11 (above) that unlike mental health causes, heart disease has generally
declined in the studied period. Applying the consideration of increasing population to this
result shows that this cause of death has generally diminished in the state of Queensland.
With this knowledge, a second viewing of the boxplot shows by how much the frequency has
declined, and roughly how many deaths from this underlying cause can be expected in a given
year (not a great thing to expect).
3.2 Bivariate Analyses
3.2.1 Bivariate Analysis 1: Years and Diseases of the Nervous System
In this section the relation between datasets having two variables will be analysed and plotted.
R has a scatterplot function and this is entirely appropriate to observe these relationships.
Following a similar initial sequence as the previous section, the prepared bivariate datasets
are loaded into R and assigned to variables with appropriate names.
> # Load the bivariate Nervous System data:
> bivar_YearNerv <- read.csv (“Year-Nervous.csv”, header = TRUE)
Since the relationship between years as categories is already genrally understood, R is queried
for a summary() of the discrete quantities in the second column.
Figure 11: Bar graph of disease2 vector
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 9
> # Get a summary of Nervous disorder data to examine and verify the loading:
> summary(bivar_YearNerv[,2])

The initial summary also displays a significant range and suggests an increase or decrease over
time. The scatterplot will reveal the presence or absence of any trend in the data.
> # Assign a title to a variable then generate a scatterplot for bivar_YearNerv:
> toptitle <- “Deaths caused by Nerve Diseases”
> plot(bivar_YearNerv, main = toptitle, ylab = “Deaths”, type = “o”)
The generated scatterplot suggests a correlation between the increase of years and deaths
caused by diseases of the nervous system. What the reason for this may be is beyond the
scope of this report and some years have shown a reduction from the previous values,
however the trend of increasing frequency is clearly indicated.
The size of the data frame bivar_YearNerv can be found with R command dim()
> # Find the number of rows and columns in bivar_YearNerv:
> dim(bivar_YearNerv)
[1] 21 2
This shows 21 rows and 2 columns. The first column is the year, and the variable whose
change is of interest is the second column of values, deaths from “Diseases of the nervous
system”.
> # Calculate the percentage change between the 1st & the last value for Deaths:
> bivar_YearNerv[21,2] / bivar_YearNerv[1,2]
[1] 2.794118
The number of deaths per year has increased by 279% over the twenty-one years of data.
Figure 12: Line & scatterplot of Deaths from Nerve Diseases
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 10
3.2.2 Bivariate Analysis 2: Female and Male Genital Cancers
The fourth prepared dataset is now loaded into R for analysis.
> # Load the genital cancer deaths data:
> bivar_gencan_MF <- read.csv(“MaleFemale.csv”, header = TRUE)
>
> # Read the the two columns’ observations into separate variables,
> # using the column headings to avoid misassignment:
> F <- bivar_gencan_MF [,”Female.genital.cancer”]
> M <- bivar_gencan_MF [,”Male.genital.cancer”]
> # Variables set for graph titles:
> gendernames <- c(“Male”,”Female”)
> toptitle = “Female vs Male Deaths from Genital Cancers”
>
> # Generate a dual boxplot to show the ranges compared:
> boxplot(M,F,names = gendernames, horizontal = TRUE, main = toptitle, col =
“Pink”, xlab=”Deaths”)
The differences between the gender variables are revealed by a dual boxplot, a version of the
boxplot that demonstrates here that it has relevant application to analysis of multivariate
datasets. Once more, when datasets are presented in the same form, differences become
apparent, and information is revealed.
From this plot, the minimum number of male deaths attributed to genital cancers has, for
every reported year, been higher than the maximum number of female deaths resulting from
this cause, with no outlying years.
Male Female
200 300 400 500 600 700
Female vs Male Deaths from Genital Cancers
Deaths
Figure 13: Dual boxplot of Deaths from Genital Cancers, Female and Male
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 11
4. Advanced Analysis
4.1 Clustering
4.1.1 Brief Explanation of Cluster Analysis and K-Means
In the preceding analysis sections, data has been deliberately segmented and placed into
categories in order to find similarities, differences, patterns or correlations. This has all been
what Provost and Fawcett (2013) denote as “supervised” modelling, but now in this section
instead the focus is on “unsupervised” modelling, specifically cluster analysis, to find
tendencies for discrete variables to form clusters when charted (Akerkar & Sajja 2016). Cluster
analysis methods seek to discover structure in data that occurs through regularities of a
dataset rather than a specific variable being the focus (Provost & Fawcett 2013). Particular
features of data objects are used to divide those objects into clusters according to intrinsic
similarities between the objects (Akerkar & Sajja 2016), rather than labelling by category (Jain
2009).
Jain (2009) identifies K-means to be one of the most popular clustering algorithms. K-means
clustering identifies where to partition data so that the mean of points in the cluster and the
points themselves have the minimum squared error (Jain 2009). The cluster has a centroid
represented by the mean of each cluster member’s dimensions (Provost & Fawcett 2013) and
the number of clusters desired is denoted by K, where for example K=4 means 4 centroids and
therefore 4 clusters (Provost & Fawcett 2013). The K-means algorithm works in multiple
passes upon the data and after initial partitioning it recalculates the centroids in order to
reduce the sum of squared error between the cluster members and centroids, iterating until
the clusters remain unchanged (Jain 2009; Provost & Fawcett 2013).
4.1.2 Clustering Analysis: Grouping Years According to Influenza Deaths
> # Clustering analysis data is loaded:
> YearFlu <- read.csv(“FluCluster.csv”, header = TRUE)
Next, the number of clusters must be decided. With a larger dataset, ideally this could be
calculated in R for optimisation, however with only 21 values for each variable making up a
relatively small dataset in this era of big data, it is desirable to avoid overfitting (Provost &
Fawcett 2013) and so seven is chosen to be the centers value for K in this analysis.
> # Give 8 attempts to find best 7 clusters with lowest SSE:
> KMFlu <- kmeans(YearFlu, 7, iter.max = 5, nstart = 8)
> # Check whether max iterations was reached:
> KMFlu$iter
[1] 2
The algorithm allocated the centroids with only 2 iterations, and so the best possible fit has
been made for seven clusters.
> # Plot the resulting cluster analysis for observation:
> clusplot(YearFlu, clus = KMFlu$cluster, color = TRUE, shade = FALSE, lines =
0, main = “k-Means Clusters of Influenza Deaths in Qld.”, ylab = “Years”, xlab =
“Deaths”, labels = 3)
Data Analysis Report of Underlying Causes of Death in Queensland (1997 -2017)
Student name (USC Student ID xxxxxxxx) 12
The plotted clusters display that extremely high fatalities caused by influenza are seen for a
period of only one year, with three consecutive years of high fatalities being shown clustering
apart from what appears to be groups of typical case frequency.
4.2 Linear Regression
4.2.1 Brief Explanation of Linear Regression
A form of supervised modelling, regression algorithms are used to develop models from
independent input variables which can be qualitative (ordinal or nominal) or quantitative
(continuous or discrete), and output a dependent continuous numeric variable used to predict
a real value (Akerkar & Sajja 2016; Pathak 2014). Linear regression assumes that the
independent and dependent variables have a linear relationship (Pathak 2014). When given
the predictor input variables, linear regression comprises the learning of a function using the
least squares method to predict a value for a target output variable (Pathak 2014).
Simple linear regression has only one dependent target variable

The post ICT110 Introduction to Data Science appeared first on My Assignment Online.

Expert Help With This Assignment — On Your Terms