Data Mining: Deferred/Referred Assignment
page 1 of 21
DATA MINING: DEFERRED/REFERRED ASSIGNMENT
Section 1: INTRODUCTION
1.0 LEARNING OUTCOMES
This assignment will assess the following learning outcomes:
1) Correctly apply and interpret a variety of statistical and data mining techniques using
appropriate data mining software.
2) Effectively communicate your results.
1.1 DEADLINE:
26 August 2016 to be submitted TWICE electronically.
Once as a “SHU Assignment” AND again via turnitin
1.2 ASSESSMENT FOR THIS MODULE
This module will be assessed via a case study. This will involve the analysis of a dataset
that is described below.
1.3 PROBLEM OUTLINE
For this assignment you are required to analyse a data set taken from the data mining
competition prior to the third international conference of Principles and Practices of
knowledge discovery in data bases (PKDD). This conference was held in Prague in 1999‡.
One of the challenges given for the competition was a set of datasets concerning financial
transactions and details for customers at a Czech bank. Several of these files have been
combined to give 4 500 observations of various financial and personal details relating to
different accounts. Details of the variables included are given in the appendix.
1.4 DATASETS
The data you will need to analyse consists of four datasets. The datasets are
czechr.sas7bdat, czechin.sas7bdat, district.sas7bdat and loans2.sastbdat. A full list of the
variables in each of these is given in the appendix.
‡ http://lisp.vse.cz/pkdd99/
Data Mining: Deferred/Referred Assignment
page 2 of 21
1.4.1 CZECH DATASET
The data has a few missing values. The full dataset consists of 4500 observations with the
number having a loan (LOAN=yes) occurring in 682 of the cases (15.16%). These data are
in the czechr.sas7bdat dataset and should be used in 2.1 for questions 1), 2) and 3). A full
list is given in the Appendix.
1.4.2 CZECHIN DATASET
This is a copy of the Czech Dataset but includes dummy variables for the missing values.
These additional variables are given in the Appendix. This dataset will be formed as part of
Enterprise Miner analysis in question 3) below and should be used in question 4). Questions
6)-8) use either this dataset or the Czech dataset following on in the Enterprise Miner
diagram.
1.4.3 DISTRICT DATASET
A number of the variables in the first two datasets relate to the demographic background
based on one of 77 districts that the account holder comes from.
• In addition we also have the name of each district distname.
• In the original dataset the variables unemploy95 and crime95 contained one missing
value. These two values have been imputed using a regression model.
• In addition, the two variables crime95 and crime96 will vary on the region size as
measured by the number of inhabitants. The other variables of this type
(unemploy95, unemploy96 and enter) have been converted to a rate. It seems
sensible to also do this for the crime variables.
Consequently two new variables have been created and used in place of crime95
and crime96:
crime95r = crime95/no_inhab*100
crime96r=crime95/no_inhab*100
This results in the final set of variables for the district dataset. A full list of these is given in
the appendix. Given these data concern the Czech Republic there are also useful
guidelines on finding information about the Czech Republic. This dataset is used in 2.2
and 2.3 and questions 9),10),11),12) and 13).
Data Mining: Deferred/Referred Assignment
page 3 of 21
1.4.4 LOANS2 DATASET
The data set to be analysed contains all the variables as listed in czechr.sas7bdat plus
additional variables that are transformations of these, details of which are given in the
appendix. The target is the variable that records whether the account holder has a loan with
the bank or not.
| LOAN | = = |
yes no |
if account holder has a loan (with this bank) if not |
This dataset is used in 2.4 in questions 14), 15), 16), 17), 18), 19) and 20).
1.4.5 AVAILABILITY OF DATASETS
All datasets will be available on the SAS 9.4 Server in the following path:
E:SHUUsers!SharedDataTeresaRefer BD Assign
1.4.5 – i SAS ENTERPRISE GUIDE
In Enterprise Guide a library is needed to point to the path above. The first line of code in
the ass2r.sas and ass3r.sas creates this library. Ensure you run this line of code before
running code that references this library.
1.4.5 – ii ENTERPRISE MINER
On the SAS Enterprise Miner server the data is in the path above. You will need to create a
library to access the data.
To create a library in SAS Enterprise Miner follow the instructions given in tutorial 2 on page
81 of the first booklet. However ensure that you point to the path above.
1.4.5 – iii SAS/IML STUDIO
Follow the instructions at the start of tutorial 2 on page 78 of the first booklet to launch
IML/Studio but navigate to the path above and open the required dataset.
Data Mining: Deferred/Referred Assignment
page 4 of 21
1.5 LOADING THE SAS MACROS AND SAS PROGRAM
To help you with this assignment programs containing some of the code you will need has
been produced. These require the use of two macros which must also be downloaded. One
macro, label.sas, has been written by Michael Friendly§. The link to this is available on
Blackboard. The other three files are available on Blackboard as direct downloads.
The files you need to download are:
• ass2r.sas
• ass3r.sas
and the two macros
• label.sas (via the link to Michael Friendly’s website) and
• pcait.sas.
Load each of these into SAS Enterprise Guide. Run the macros: label.sas and pacait.sas.
When you run them nothing will have appeared to have happened, but this will load them into
SAS and then compile them. This will enable them to be called from the remaining program.
§ http://www.math.yorku.ca/SCS/sasmac/label.html Michael Friendly Psychology Department
York University, Toronto, Canada
Data Mining: Deferred/Referred Assignment
page 5 of 21
Section 2: ANALYSIS REQUIRED
2.0 INTRODUCTION
There are four parts to this assignment in line with the original four topics for this module.
You need to attempt all parts and obtain a mark of 40% (i.e. 120 marks out of the possible
300) to pass the module. The final mark will be converted to a percentage.
2.1 PART 1: EXPLORING THE DATA
You are required to analyse the Czechr.sas7bdat data in
SAS Enterprise Guide, SAS/IML Studio and Enterprise
Miner as follows:
1) Using either SAS/IML Studio or Enterprise Miner
produce suitable graphs of the data that will aid your
understanding of the variables and also their
relationship to the target variable LOAN. Fully
discuss your results in the report (see details below).
You should discuss your plots and any output fully.
Which variables are symmetrical? Which ones are
likely to be good predictors of the target variable
LOAN? What other features can you discover about
these data?
(11 marks)
2) Load the data into Enterprise Miner. Ensure that all
the variables have the correct role and level.
Produce suitable summary measures (means,
standard deviation, skewness etc.) of the data and
fully interpret your results. For example, what is the
most frequent type of credit? Which is the most
consistent? What about withdrawals? How does
unemployment change? Are there any other
interesting features? Discuss the missing values that
exist in these data.
(8 marks)
Figure 2.1
3) Create an Enterprise Miner stream by adding the Czechr data to a new diagram.
a) Now add a impute node: . Join this to the Czechr node. Leave the
default method for input interval variables set to “mean”. Similarly leave the default
method for class variables as “count”. Under Score set “Hide Original Variables” to
“No”. Also under Score and Indicator Variables set “Type” to “unique” and “Role” to
“input”. The final diagram will look like this: (see below for further nodes to be added).
Data Mining: Deferred/Referred Assignment
page 6 of 21
Figure 2.2
Run the impute node and examine the results.
b) Add a StatExplore node. Right click and select Edit Variables… From here select
the new variables created as part of running this node and the original ones that had
missing values. Click Explore. Examine the table of values, and make sure you
understand what the variables that have been created represent. Include a small
section of this table in your report. (2 marks)
c) In your own words explain what the impute node has done. Do you think the
particular choice of settings used were sensible? Should any settings be changed to
impute any of the categorical variables? Fully justify your answers.
(4 marks)
4) A copy of the resulting dataset czechin.sas7bdat that includes these dummy variables
and imputed variables is available. Using either this dataset and SAS/IML Studio or
using Enterprise Miner and appropriate nodes, produce suitable plots to see if the
indicator variables generated from the replacement node are likely to be good predictors
of LOAN. If you use Enterprise Miner, join subsequent nodes to the impute node, so that
you are using a dataset with the new variables. Join a GraphExplore node to this to
obtain a suitable plot against interval variables (see b) below). Join a StatExplore node,
run it and inside the results window, select ViewPlots Class Variables: Loan for
categorical variables against Loan.
a) Comment on the type of missingness that these plots seem to suggest. Justify your
answer. (3 marks)
b) Also plot these indicator variables against one other interval variable and one
other categorical variable. Again comment on the type of missingness this seems
to suggest. (4 marks)
c) What other plots would be needed to fully investigate the type of missingness?
(3 marks)
5) The data have been analysed using a macro to calculate Little’s test for the interval
variables. This yields the output given below:
Data Mining: Deferred/Referred Assignment
page 7 of 21
The MI Procedure
Model Information
| Data Set Method |
WORK.ONE MCMC |
| Multiple Imputation Chain | Single Chain |
| Initial Estimates for MCMC | EM Posterior Mode |
| Start | Starting Value |
| Prior | Jeffreys |
| Number of Imputations Number of Burn-in Iterations Number of Iterations |
0 200 100 |
| Seed for random number generator | 146279001 |
Missing Data Patterns
Group creditn creditt withdrn withdrt cashcrn cashcrt cashwdn cashwdt bankcolt bankcoln bankrn bankrt othcrn othcrt days no_inhab mu_low
1 X X X X X X X X X X X X X X X X X
2 X X X X X X X X X X X X X X X X X
3 X X X X X . X X X X X X X X X X X
4 X X X X X . X X X X X X X X X X X
5 X X X X . X X X X X X X X X X X X
6 X X X X . X X X X X X X X X X X X
7 X X X X . . X X X X X X X X X X X
8 X X X X . . X X X X X X X X X X X
Missing Data Patterns
——–Group Means——-
Group mu_lmid mu_umid mu_high cities urbanr ave_sal unemploy95 unemploy96 enter crime95 crime96 Freq Percent creditn creditt
1 X X X X X X X X X X X 2685 59.67 90.067412 761230
2 X X X X X X . X X . X 28 0.62 93.142857 782562
3 X X X X X X X X X X X 405 9.00 91.143210 625108
4 X X X X X X . X X . X 3 0.07 61.666667 262904
5 X X X X X X X X X X X 587 13.04 89.621806 661191
6 X X X X X X . X X . X 4 0.09 68.000000 523226
7 X X X X X X X X X X X 775 17.22 89.856774 652148
8 X X X X X X . X X . X 13 0.29 79.000000 928778
SAS Output 2.1: part 1
Data Mining: Deferred/Referred Assignment
page 8 of 21
Missing Data Patterns
———————————————————————–Group Means———————————————————————-
Group withdrn withdrt cashcrn cashcrt cashwdn cashwdt bankcolt bankcoln bankrn bankrt othcrn
1 146.067412 716722 35.265549 559981 97.420484 555482 195041 14.145996 46.931099 157333 40.655866
2 145.392857 736734 45.285714 603856 104.000000 582953 171939 5.750000 39.107143 148046 42.107143
3 143.293827 582119 33.429630 . 96.627160 451503 136283 16.207407 45.071605 127215 41.506173
4 108.666667 235966 24.666667 . 53.333333 110597 49539 12.666667 52.333333 118903 24.333333
5 142.826235 618393 . 506809 96.148211 475222 148308 14.652470 44.691652 138498 40.502555
6 95.000000 482927 . 517894 66.750000 361671 0 0 28.250000 121256 34.500000
7 143.029677 609421 . . 94.596129 462042 138299 15.196129 46.441290 143018 40.696774
8 119.076923 892208 . . 86.153846 772858 261501 7.923077 32.923077 119351 33.692308
Missing Data Patterns
———————————————————————–Group Means———————————————————————-
Group othcrt days no_inhab mu_low mu_lmid mu_umid mu_high cities urbanr ave_sal unemploy95
1 6208.208268 1223.985847 271465 39.476350 21.032030 5.572439 1.740037 5.542272 69.651359 9537.816015 2.870011
2 6768.300000 1242.678571 42821 4.000000 13.000000 5.000000 1.000000 3.000000 48.400000 8173.000000 .
3 5956.532840 1266.148148 294505 37.916049 19.518519 5.279012 1.676543 5.429630 70.872099 9622.735802 2.921901
4 3368.333333 945.000000 42821 4.000000 13.000000 5.000000 1.000000 3.000000 48.400000 8173.000000 .
5 6073.293356 1259.357751 274538 40.683135 22.018739 5.563884 1.758092 5.579216 69.087394 9527.419080 2.907445
6 5332.150000 1190.750000 42821 4.000000 13.000000 5.000000 1.000000 3.000000 48.400000 8173.000000 .
7 5850.351355 1261.165161 258358 41.938065 20.983226 5.569032 1.703226 5.680000 68.363871 9454.548387 2.839419
8 5327.923077 1097.230769 42821 4.000000 13.000000 5.000000 1.000000 3.000000 48.400000 8173.000000 .
Missing Data Patterns
————————-Group Means————————
Group unemploy96 enter crime95 crime96
1 3.458276 121.120298 14702 16376
2 7.010000 124.000000 . 1358.000000
3 3.481654 121.308642 16516 18470
4 7.010000 124.000000 . 1358.000000
5 3.478296 121.407155 14991 16753
6 7.010000 124.000000 . 1358.000000
7 3.434142 120.598710 13903 15529
8 7.010000 124.000000 . 1358.000000
SAS Output 2.1: part 2
Data Mining: Deferred/Referred Assignment
page 9 of 21
Number of Observed Variables = 28
Number of Missing Data Patterns = 8
Summary of Missing Data Patterns (0 = Missing, 1 = Observed)
Frequency | Pattern | d2j
13 | 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 | 251.5981
775 | 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 | 37.74116
4 | 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 | 87.35500
587 | 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 | 40.33753
3 | 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 | 61.13851
405 | 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 | 39.07896
28 | 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 | 523.3387
2685 | 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 | 46.99653
Sum of the Number of Observed Variables Across Patterns (Sigma psubj) = 208
Little’s (1988) Chi-Square Test of MCAR
| Chi-Square (d2) | = | 1087.584 180F 0.000 |
| df (Sigma psubj – p) = | ||
| p-value | = |
SAS Output 2.1: part 3
a) Explain how many missing value patterns exist and what each of these represent. Is
there any evidence in the printed means as to what type of missingness exists?
(4 marks)
b) Interpret the result of Little’s test. What type of missingness exists? (3 marks)
6) Some of the techniques that we will try later require that the variables be symmetrical.
Add a transformation node and find suitable transformations of the interval variables that
could meet this requirement. (To enable comparisons to be made it is best under Score
to set both “Hide” and “Reject” to “No” so that the original variables are still kept.)
a) Explain what transformations have been selected. (2 marks)
b) In some cases the transformation adds 1 first, explain why this is necessary.
(2 marks)
7) Produce suitable plots to check if the transformations appear to have worked. Fully
discuss your results. (6 marks)
8) Using you discussion above, summarize your findings from questions 1)-7). For example,
indicate how each variable should be treated in the subsequent analysis and whether it is
likely to be a good or poor predictor.
(4 marks)
Data Mining: Deferred/Referred Assignment
page 10 of 21
2.2 PART 2: PRINCIPAL COMPONENTS
You are required to analyse the district data in SAS Enterprise Guide, SAS/IML Studio and
Enterprise Miner as described below. First, launch Enterprise Guide and load and run the
macros: label.sas and pcait.sas. When you run them nothing will have appeared to have
happened, but this will load them into SAS and then compile them. This will enable them to
be called from the remaining programs.
9) The missing value for unemploy95 was imputed using a regression model of
unemploy95 on unemploy96. The model was fitted, and then the predicted value was
used to replace the missing value.
a) Use Enterprise Guide and load and compile the macros label.sas and pcait.sas. Load
the ass2r.sas program. Add some code at the start of this to produce a scatter plot of
unemploy95 vs. unemploy96 and comment on the relationship between them.
(2 marks)
b) Run the first section of code in the ass2r.sas program (BLOCK 1) to fit a simple linear
regression of unemploy95 on unemploy96. Fully interpret the output.
(6 marks)
c) Hence or otherwise comment on the suitability of this method to impute the missing
values. (2 marks)
d) A similar process was used to impute the missing value for crime95. Do you think
this is reasonable? Justify your answer. (1 mark)
10) Carry out a principal component analysis of the data. You may use SAS/IML Studio,
SAS Enterprise Miner or the code provided in Enterprise Guide to do this. Examples of
the code can be found in the second block of ass2r.sas. Please ensure that the library
that you have created that contains the district dataset is correctly referenced in the code.
a) Examine the eigenvalues. Also produce a scree plot (if using the code this is done
for you). How many components would you recommend? Fully justify your answer.
(7 marks)
b) Interpret the component loadings for the first three components. What “name” might
you give each? (9 marks)
c) Produce a variety of scatter plots of one component against another. Include labels
for the districts on the edge of the scatter plot and the component loadings. The
macros given you will achieve this, and example code can be found in the second
block of ass2r.sas. You should produce plots that correspond to the number of
components you would keep. Note that the component loadings are superimposed
on the scatter plot rather than as a separate plot.
Data Mining: Deferred/Referred Assignment
page 11 of 21
i) Using your plots explain how the component loadings (in blue) are
represented on the plot. Illustrate this for two variables. How does this
representation help in understanding the components? How does this
representation help in understanding which variables are similar? Use
examples from your plot where necessary. (8 marks)
ii) Examine the scattered points showing each district. What do you notice?
Are there any unusual points? Identify these. Illustrate what features these
district(s) have that might make them unusual. Use your interpretation of the
components in b) above and the component loadings shown on the plot to
achieve this. (5 marks)
d) Given all of your results for question 10) part a) through to c) discuss how well
principal component analysis works for this particular dataset. What interesting
features have you discovered about the data? How might we utilise the results in
subsequent analysis? (5 marks)
2.3 PART 3 CLUSTER ANALYSIS
You are required to analyse this data in SAS Enterprise Guide, SAS/IML Studio and
Enterprise Miner as follows:
11) You are required to carry out variable clustering. First the data needs to be
standardised. To do this run block one in the ass3r.sas program which will also add the
principal components to the data. Now run block two which will perform the variable
clustering.
a) Examine the dendrogram. How many clusters would you pick? Fully justify your
answer.
(5 marks)
b) Examine the printed output. Explain how many clusters the algorithm picks as its
final solution. What criteria has the algorithm used to do this?
(4 marks)
c) Fully discuss the printed output for the final solution. How well does this clustering
seemed to have worked? What variables are in the same cluster? Compare your
answers to the results of principal component analysis from the solution to
assignment 2. Are there any similarities? Does this surprise you?
(12 marks)
12) The next stage is to carryout observational clustering. Run block 3 of the code to
perform Ward’s method of clustering.
Data Mining: Deferred/Referred Assignment
page 12 of 21
a) Examine the printed output. How many clusters would you pick? Give reasons for
your answer utilising the information given in the printed output.
(5 marks)
b) Examine the graphical output, the dendrogram and the plot of the Cubic Clustering
Criteria. How many clusters would each of these plots suggest? Fully explain you
reasoning. (10 marks)
c) Using all the results from 12) a) and b) above, explain how many clusters you would
pick. Justify your answer. (4 marks)
d) Edit the code in block 3 to carry out clustering for one other type of hierarchical
clustering. Fully interpret both the printed output and the graphs produced, again
explaining how many cluster you would pick. Are the results from your chosen
method any clearer than for Ward’s method? Why might this be the case?
(12 marks)
13) The next stage is to use the k-means algorithm to find six clusters for the observations.
Run block 4 and examine the output.
a) Is it possible to produce a dendrogram for this method of clustering? Explain why.
(4 marks)
b) Fully interpret the output such as the R2 values, the R2 ratio, cluster centroids etc.
Using this evidence explain how well this clustering seems to have worked. Is there
any evidence that the clusters overlap?
(14 marks)
c) Various plots can be produced to validate the cluster solution. These are produced
in blocks 5-8. Run each of the blocks and use a suitable selection from these to
discuss the properties of the cluster containing Prague and two other clusters.
Profile these three clusters and explain their features. Compare these profiles to the
results from the solution to part 2 and where each cluster appears on the plot of the
principal components. (Please note that in Enterprise Guide some plots do not
display correctly if you do not use the correct graphics device. Goptions statements
have been inserted in the code – please make sure you run these when running the
code.)
(14 marks)
Data Mining: Deferred/Referred Assignment
page 13 of 21
2.4 PART FOUR PREDICTIVE MODELLING
In the first parts of this assignment you will have already explored the data and discussed its
features. In this section you will attempt to produce a predictive model for the variable
LOAN. In earlier work it could be suggested that various transformations of the variables
could be tried. These have been included in the dataset loans2, but you are free to create
others that you may think appropriate.
You should read the loans2.sas7bdat data into SAS Enterprise Miner checking carefully that
each variable is correctly specified (see appendix). In addition you should partition the data
into training, validation and testing data using a data partition node (found under Sample).
14) Use suitable software to fit two logistic regression models to LOAN. In both cases you
may wish to use suitable methodology to reduce the set of variables used in the model.
(e.g in one case you might use backward elimination and in the other forward
selection). You will only need to discuss one of these models in detail once you have
compared them in part 4) below. Ensure that your settings do produce two different
models. Explain and justify the settings used.
( [fitting/settings] 6 marks)
15) Use suitable software to fit two decision trees. These should be distinguished by the
algorithms and settings you use which should be appropriate for these data, and you
should justify your choices. Again the two decision trees will be compared in part 4)
below, after which you will need to discuss only one of these trees in detail. Explain
and justify the settings used.
( [fitting/settings] 6 marks)
16) Build at least one other model. You will need to experiment with Enterprise Miner to
do this and possibly read the help available. Suggestions include, Neural Network or
Memory Based Reasoning. Briefly explore the software and explain one setting unique
to your chosen method. ( [choice/fitting/settings] 4 marks)
17)
a) Compare the logistic regression models, the decision tree models and the third
model by producing appropriate summary measures of their goodness of fit.
Explain which logistic regression model and which decision tree you would prefer.
Which model would you prefer overall? (12 marks)
b) Compare all five models by producing suitable %Cumulative response,
%Cumulative captured response and ROC charts. For each type of chart illustrate
what it is showing by the interpretation one particular point and discuss how well
Data Mining: Deferred/Referred Assignment
page 14 of 21
the models are performing. Using these charts, explain which logistic regression
model and which decision tree you would prefer. Which model would you prefer
overall? (12 marks)
c) Given your answers to 17) a) and 17) b) above which model would you pick?
Justify your answer. (3 marks)
18) In each case fully discuss the results for these preferred models:
a) For the chosen regression model, discuss the parameter estimates, odds ratios,
summary statistics, statistical tests etc. (12 marks)
b) For the best decision tree discuss the number of leaves, structure, variables, plot of
misclassification against leaves, goodness of fit, etc.
(12 marks)
19) For the selected logistic regression model and the selected decision tree (chosen
models in question 17)c) above) illustrate how the predicted probability of taking a loan
for the first observation in the data is calculated. (16 marks)
20) Use all of your results from questions 14) to 19) select one particular method that you
would recommend to predict LOAN. You should bear in mind practical as well as
statistical considerations. Fully discuss your choice and any other final conclusions
you have resulting from all of your analyses 14) to 19) above. Suggest any further
work that may be useful to build an improved model in the future.
(15 marks)
Total Marks 280
2.5 REPORT
You should write all your findings in a technical report which should include any relevant
output either in the main body of the report or in a suitable appendix (as appropriate).
Marks will be awarded for good English, report format, layout, use of Figures, balance
between appendices and main report etc.
The report should not be of more than 4000 words.
(20 marks)
Total Marks available: 300 marks
Data Mining: Deferred/Referred Assignment
page 15 of 21
Section 3: APPENDIX
3.0 DETAILS OF CZECHR.SAS7BDAT
| Variable | Meaning | Notes |
| account_id | identification of the account | |
| Credit | Total number of credits | § Measured over the period of the transaction data base which is from 1/1/93 – 31/12/98 however many accounts do not show any transactions until much later than the 1/1/93 (the last “first transaction date” is 29/12/97). Similarly for the last transaction. |
| Credit | Total value of credits | |
| Withdrn | Total number of withdrawls | |
| Withdrt | Total value of withdrawls | |
| Cashcrn | Total number of cash credits | |
| Cashcrt | Total value of cash credits | |
| Cashwdn | Total number of cash withdrawals | |
| Cashwdt | Total value of cash withdrawals | |
| Bankcoln | Total number of collections from other banks (i.e. electronic credits) |
|
| Bankcolt | Total value of collections from other banks (i.e. electronic credits) |
|
| Bankrn | Total number of times of remittance to other banks (i.e. electronic debits) |
|
| Bankrt | Total value of remittance to other banks (i.e. electronic debits) |
|
| Othcrn | Total number of other credits | |
| Othcrt | Total value of other credits | |
| Days | Total number of days between first transaction and last |
|
| Sex | Gender of primary account holder | |
| Second | ||
| Frequency | frequency of issuance of statements | “monthly” stands for monthly issuance “weekly” stands for weekly issuance “After_Trans” stands for issuance after transaction |
| Region | Region in which account holder lives | One of : Prague; North Moravia; South Moravia; South Bohemia; Central Bohemia; North Bohemia; West Bohemia; East Bohemia |
=N if no second account holder
=Y if thee is a secondary account holder
Continued on next page…
Data Mining: Deferred/Referred Assignment
page 16 of 21
| no_inhab | no. of inhabitants | Demographic data based on one of the 77 possible districts the account holder lives in. (Therefore a large number of the values of these variables will be the same for several observations corresponding to clients living in the same district). |
| mu_low | no. of municipalities with inhabitants < 499 | |
| mu_lmid | no. of municipalities with inhabitants 500- 1999 |
|
| mu_umid | no. of municipalities with inhabitants 2000- 9999 |
|
| mu_high | no. of municipalities with inhabitants >10000 | |
| Cities | no. of cities | |
| Urbanr | ratio of urban inhabitants | |
| ave_sal | average salary | |
| unemploy95 | unemployment rate ’95 | |
| unemploy96 | unemployment rate ’96 | |
| Enter | no. of entrepreneurs per 1000 inhabitants | |
| crime95 | no. of committed crimes ’95 | |
| crime96 | no. of committed crimes ’96 | |
| LOAN | whether the account holder has a loan with the bank |
= yes if account holder has a loan (with this bank) = no if not |
3.1 DETAILS OF CZECHIN.SAS7BDAT
This dataset contains all the variables of Czech above but in addition has the following
variables:
| Variable | Meaning | Notes |
| IMP_cashcrn | Imputed cashcrn | Copy of cashcrn, but missing values replaced by the mean of the remaining cashcrn values. |
| IMP_cashcrt | Imputed cashcrt | Copy of cashcrt, but missing values replaced by the mean of the remaining cashcrt values. |
| IMP_crime95 | Imputed crime95 | Copy of crime95, but missing values replaced by the mean of the remaining crime95 values. |
| IMP_unemploy95 | Imputed unemploy95 | Copy of unemploy95, but missing values replaced by the mean of the remaining unemploy95 values. |
| M_cashcrn | Indicator for if value of cashcrn is missing |
= 0 if cashcrn not missing = 1 if cashcrn missing |
| M_cashcrt | Indicator for if value of cashcrt is missing |
= 0 if cashcrt not missing = 1 if cashcrt missing |
| M_crime95 | Indicator for if value of crime95 is missing |
= 0 if crime95 not missing = 1 if crime95 missing |
| M_unemploy95 | Indicator for if value of unemploy95 is missing |
= 0 if unemploy95 not missing = 1 if unemploy95 missing |
Data Mining: Deferred/Referred Assignment
page 17 of 21
3.2 DETAILS OF DISTRICT.SAS7BDAT
| Variable | Meaning | Notes |
| Region | Region in which account holder lives | One of : Prague; North Moravia; South Moravia; South Bohemia; Central Bohemia; North Bohemia; West Bohemia; East Bohemia |
| no_inhab | no. of inhabitants | Demographic data based on one of the 77 possible districts the account holder lives in. (Therefore, in the Czech dataset, a large number of the values of these variables will be the same for several observations corresponding to clients living in the same district). In this version of the dataset there is just the data on districts i.e. ONE observation per district. |
| mu_low | no. of municipalities with inhabitants < 499 | |
| mu_lmid | no. of municipalities with inhabitants 500-1999 | |
| mu_umid | no. of municipalities with inhabitants 2000-9999 | |
| mu_high | no. of municipalities with inhabitants >10000 | |
| cities | no. of cities | |
| urbanr | ratio of urban inhabitants | |
| ave_sal | average salary | |
| unemploy95 | unemployment rate ’95 | |
| unemploy96 | unemployment rate ’96 | |
| enter | no. of entrepreneurs per 1000 inhabitants | |
| crime95r | crime rate ’95 as a % | |
| crime96r | crime rate ’96 as a% | |
| distname | district name |
3.2.1 BACKGROUND
Given that the data concerns the Czech Republic and municipalities within it, you may find it
useful to research the structure of this country. A good starting point is:
http://en.wikipedia.org/wiki/Czech_Republic ;
http://en.wikipedia.org/wiki/Regions_of_the_Czech_Republic
and
http://en.wikipedia.org/wiki/Districts_of_the_Czech_Republic
A map of the regions is given overleaf.
Data Mining: Deferred/Referred Assignment
page 18 of 21
Data Mining: Deferred/Referred Assignment
page 19 of 21
3.3 DETAILS OF LOANS2.SAS7BDAT
| Variable | Meaning | Notes |
| account_id | identification of the account | |
| creditn | Total number of credits | =cashcrn + bankcoln+ othcrn ; see also § |
| creditt | Total value of credits | = cashcrt + bankcolt + othcrt ; see also § |
| withdrn | Total number of withdrawls | = cardwdln + cashwdn + bankrn + othwdn ; see also § |
| withdrt | Total value of withdrawls | = cardwdlt + cashwdt + bankrt + othwdt ; see also § |
| clbal | Closing balance at the end of the transaction period. |
For most accounts this is at the end of 1998 i.e. 31/12/98, however some record a last transaction one or two months earlier presumably because there were no transactions in the last one or two months of 1998. |
| cardwdln | Total number of credit card withdrawals | § Measured over the period of the transaction data base which is from 1/1/93 – 31/12/98 however many accounts do not show any transactions until much later than the 1/1/93 (the last “first transaction date” is 29/12/97). Similarly for the last transaction – see previous remark. |
| cardwdlt | Total value of credit card withdrawals | |
| cashcrn | Total number of cash credits | |
| cashcrt | Total value of cash credits | |
| cashwdn | Total number of cash withdrawals | |
| cashwdt | Total value of cash withdrawals | |
| bankcoln | Total number of collections from other banks | |
| bankcolt | Total value of collections from other banks | |
| bankrn | Total number of times of remittance to other banks |
|
| bankrt | Total value of remittance to other banks | |
| othcrn | Total number of other credits | |
| othcrt | Total value of other credits | |
| Days | Total number of days between first transaction and last |
|
| sex | Gender of primary account holder | |
| card | ||
| age | Age of primary account holder |
= no if account holder does not hold a credit
card with this bank
= yes if account holder does hold a credit
card with this bank
Continued on next page…
Data Mining: Deferred/Referred Assignment
page 20 of 21
| second | ||
| frequency | frequency of issuance of statements | “monthly” stands for monthly issuance “weekly” stands for weekly issuance “After_Trans” stands for issuance after transaction |
| Region | Region in which account holder lives | One of : Prague; North Moravia; South Moravia; South Bohemia; Central Bohemia; North Bohemia; West Bohemia; East Bohemia |
| no_inhab | no. of inhabitants | Demographic data based on one of the 77 possible districts the account holder lives in. (Therefore a large number of the values of these variables will be the same for several observations corresponding to clients living in the same district). |
| mu_low | no. of municipalities with inhabitants < 499 | |
| mu_lmid | no. of municipalities with inhabitants 500-1999 | |
| mu_umid | no. of municipalities with inhabitants 2000-9999 | |
| mu_high | no. of municipalities with inhabitants >10000 | |
| cities | no. of cities | |
| urbanr | ratio of urban inhabitants | |
| ave_sal | average salary | |
| unemploy95 | unemploymant rate ’95 | |
| unemploy96 | unemploymant rate ’96 | |
| enter | no. of enterpreneurs per 1000 inhabitants | |
| crime95 | no. of commited crimes ’95 | |
| crime96 | no. of commited crimes ’96 | |
| loan | = yes if account holder has a loan with the bank |
Target variable |
| creditnd | Total number of credits/days | =cashcrnd + bankcolnd+ othcrnd ; see also §2 |
| credittd | Total value of credits/days | = cashcrt + bankcolt + othcrt ; see also §2 |
| withdrnd | Total number of withdrawls/days | = cardwdln + cashwdn + bankrn + othwdn ; see also §2 |
| withdrtd | Total value of withdrawls/days | = cardwdlt + cashwdt + bankrt + othwdt ; see also §2 |
=N if no second account holder
=Y if thee is a secondary account holder
= no if account holder does not have a
loan with the bank
Continued on next page…
Data Mining: Deferred/Referred Assignment
page 21 of 21
| cardwdlnd | Total number of credit card withdrawals/days | §2 Measured over the period of the transaction data base which is from 1/1/93 – 31/12/98 however many accounts do not show any transactions until much later than the 1/1/93 (the last “first transaction date” is 29/12/97). Similarly for the last transaction – see remark on clbal. |
| cardwdltd | Total value of credit card withdrawals/days | |
| cashcrnd | Total number of cash credits/days | |
| cashcrtd | Total value of cash credits/days | |
| cashwdnd | Total number of cash withdrawals/days | |
| cashwdtd | Total value of cash withdrawals/days | |
| bankcolnd | Total number of collections from other banks/days | |
| bankcoltd | Total value of collections from other banks/days | |
| bankrnd | Total number of times of remittance to other banks/days |
|
| bankrtd | Total value of remittance to other banks/days | |
| othcrnd | Total number of other credits/days | |
| othcrtd | Total value of other credits/days | |
| accredit | Average value of credits = Total value of credits /Total number of credits = credit/creditn |
=( cashcrt + bankcolt + othcrt ) / (cashcrn + bankcoln+ othcrn) |
| awithdr | Average value of withdrawals = Total value of withdrawals / Total number of withdrawals = withdrt/withdrn |
=( cardwdlt + cashwdt + bankrt + othwdt )/(cardwdln + cashwdn + bankrn + othwdn) |
| acardwdl | Average value of credit card withdrawals = Total value of credit card withdrawals /Total number of credit card withdrawals = cardwdlt/cardwdln |
§3 Given earlier remarks (see §) average is based on different time periods per observation |
| acashcr | Average value of cash credits = Total value of cash credits /Total number of cash credits = cashcrt/cashcrn |
|
| acashwd | Average value of cash withdrawals = Total value of cash withdrawals /Total number of cash withdrawals = cashwdt/caswdn |
|
| abankcol | Average value of collections from other banks = Total value of collections from other banks /Total number of collections from other banks = bankcolt /bankcoln |
|
| abankr | Average value of remittance to other banks = Total value of remittance to other banks /Total number of times of remittance to other banks = bankrt/bankrn |
|
| aothcr | Average value of other credits = Total value of other credits /Total number of other credits = othcrt /othcrn |
The post Data Mining: Deferred/Referred Assignment appeared first on My Assignment Online.