Página de pruebas

Data analysis

Data analysis is the process of inspecting, cleansing, transforming, and modelling data with a goal of discovering useful information, suggesting conclusions, and supporting decision-making.

Data mining is a particular data analysis technique that focuses on the modelling and knowledge discovery for predictive rather than purely descriptive purposes, while business intelligence covers data analysis that relies heavily on aggregation, focusing on business information.

In statistical applications, data analysis can be divided into:

• Descriptive statistics,
• Exploratory data analysis (EDA), and
• Confirmatory data analysis (CDA).

In particular, data analysis typically includes data retrieval and data cleaning (pre-processing) stages.

Exploratory Data: In statistics, the exploratory data analysis (EDA) is an approach to analyzing data sets to summarize their main characteristics, often with visual methods.

Confirmatory Analysis: In statistics, confirmatory analysis (CA) or confirmatory factor analysis (CFA) is a special form of factor analysis, most commonly used in social research.

Empirical Research:

Statistical Significance:

Descriptive Data Analysis:

• Rather than find hidden information in the data, descriptive data analysis looks to summarise the dataset.
• They are commonly implemented measures included in the descriptive data analysis:
  • Central tendency (mean, mode, median)
  • Variability (standard deviation, min/max)

Exploratory Data Analysis:

• Generate Summaries and make general statements about the data, and its relationships within the data is the heart of Exploratory Data Analysis.

• We generally make assumptions on the entire population but mostly just work with small samples. Why are we allowed to do this??? Two important definitions:
  • Population: A precise definition of all possible outcomes, measurements or values for which inference will be made about.
  • Sample: A portion of the population which is representative of the population (at least ideally).

Types of Variable: https://statistics.laerd.com/statistical-guides/types-of-variable.php

Central tendency

https://statistics.laerd.com/statistical-guides/measures-central-tendency-mean-mode-median.php

A central tendency (or measure of central tendency) is a single value that attempts to describe a set of data by identifying the central position within that set of data.

The mean (often called the average) is most likely the measure of central tendency that you are most familiar with, but there are others, such as the median and the mode.

The mean, median and mode are all valid measures of central tendency, but under different conditions, some measures of central tendency become more appropriate to use than others. In the following sections, we will look at the mean, mode and median, and learn how to calculate them and under what conditions they are most appropriate to be used.

Mean

Mean (Arithmetic)

The mean (or average) is the most popular and well known measure of central tendency.

The mean is equal to the sum of all the values in the data set divided by the number of values in the data set.

So, if we have $n$ values in a data set and they have values $x_{1},x_{2},...,x_{n},$the sample mean, usually denoted by ${\bar {x}}$ (pronounced x bar), is:

${\bar {x}}={\frac {(x_{1}+x_{2}+...+x_{n})}{n}}={\frac {\sum x}{n}}$

The mean is essentially a model of your data set. It is the value that is most common. You will notice, however, that the mean is not often one of the actual values that you have observed in your data set. However, one of its important properties is that it minimises error in the prediction of any one value in your data set. That is, it is the value that produces the lowest amount of error from all other values in the data set.

An important property of the mean is that it includes every value in your data set as part of the calculation. In addition, the mean is the only measure of central tendency where the sum of the deviations of each value from the mean is always zero.

When not to use the mean

The mean has one main disadvantage: it is particularly susceptible to the influence of outliers. These are values that are unusual compared to the rest of the data set by being especially small or large in numerical value. For example, consider the wages of staff at a factory below:

Staff	1	2	3	4	5	6	7	8	9	10
Salary	15k	18k	16k	14k	15k	15k	12k	17k	90k	95k

The mean salary for these ten staff is $30.7k. However, inspecting the raw data suggests that this mean value might not be the best way to accurately reflect the typical salary of a worker, as most workers have salaries in the $12k to 18k range. The mean is being skewed by the two large salaries. Therefore, in this situation, we would like to have a better measure of central tendency. As we will find out later, taking the median would be a better measure of central tendency in this situation.

Another time when we usually prefer the median over the mean (or mode) is when our data is skewed (i.e., the frequency distribution for our data is skewed). If we consider the normal distribution - as this is the most frequently assessed in statistics - when the data is perfectly normal, the mean, median and mode are identical. Moreover, they all represent the most typical value in the data set. However, as the data becomes skewed the mean loses its ability to provide the best central location for the data because the skewed data is dragging it away from the typical value. However, the median best retains this position and is not as strongly influenced by the skewed values. This is explained in more detail in the skewed distribution section later in this guide.

Median

The median is the middle score for a set of data that has been arranged in order of magnitude. The median is less affected by outliers and skewed data. In order to calculate the median, suppose we have the data below:

65	55	89	56	35	14	56	55	87	45	92

We first need to rearrange that data into order of magnitude (smallest first):

14	35	45	55	55	56	56	65	87	89	92

Our median mark is the middle mark - in this case, 56. It is the middle mark because there are 5 scores before it and 5 scores after it. This works fine when you have an odd number of scores, but what happens when you have an even number of scores? What if you had only 10 scores? Well, you simply have to take the middle two scores and average the result. So, if we look at the example below:

65

55

89

56

35

14

56

55

87

45

We again rearrange that data into order of magnitude (smallest first):

14

35

45

55

55

56

56

65

87

89

Only now we have to take the 5th and 6th score in our data set and average them to get a median of 55.5.