S 519 Evaluation of Information Systems Social Statistics

  • Slides: 20
Download presentation
S 519: Evaluation of Information Systems Social Statistics Inferential Statistics Chapter 8: Significantly significant

S 519: Evaluation of Information Systems Social Statistics Inferential Statistics Chapter 8: Significantly significant

Last week

Last week

This week l l What is significance and why it is important Type I

This week l l What is significance and why it is important Type I and Type II errors How inferential statistics works How to select the proper statistical test for your research

The concept of significance l Significance l Any difference between the attitudes of the

The concept of significance l Significance l Any difference between the attitudes of the two groups is due to some systematic influence and not due to chance.

Significance l Example l Hypothesis: There is a significant difference in attitude toward maternal

Significance l Example l Hypothesis: There is a significant difference in attitude toward maternal employment between adolescents whose mothers work and adolescents whose mothers do not work, as measured by a test of emotional state. l There are many other reasons to affect this hypothesis, for example?

Significant level l Significant level is the risk associated with not being 100% confident

Significant level l Significant level is the risk associated with not being 100% confident that the null is true (there is no difference between data or variables) l If the significant finding occurred at the 0. 05 level (p<0. 05), this means that there is 1 chance in 20 (5%) that there is no difference in data (null is true): any differences found were not due to the hypothesized reason, but to some other unknown reasons, or by chance.

Significant level l p-value l l The probability for the null hypothesis to be

Significant level l p-value l l The probability for the null hypothesis to be true The probability for no difference in data or variables p>0. 05 (non significant): more than 5% chance (5% to 99%) that the null is true (no difference in data) – accept null p<0. 05 (statistically significant): less than 5% chance that the null is true (no difference in data) – reject null

Null and research hypothesis l Research hypothesis l l Null hypothesis l l There

Null and research hypothesis l Research hypothesis l l Null hypothesis l l There is a difference in the academic achievement of children who participated in a preschool program and children who did not participate. The two groups are equal to each other on some measure of achievement. As a good researcher, your job to show that any difference that exists between these two groups is due only to the effects of the preschool experience.

Statistical significance l l A result is called statistically significant it is unlikely to

Statistical significance l l A result is called statistically significant it is unlikely to cause by chance A statistically significant difference there is statistical evidence that there is a difference l It does not mean that the difference is necessarily large, important, or significant in common understanding

Null hypothesis l l Either true or false But null cannot be tested directly

Null hypothesis l l Either true or false But null cannot be tested directly (as it is applied to the population) The researchers do not know the real true nature of the null hypothesis, and it is hard to know and to test That is why we need inferential statistics

Table 8. 1 (s-p 179) Accept the null hypothesis Reject the null hypothesis The

Table 8. 1 (s-p 179) Accept the null hypothesis Reject the null hypothesis The null hypothesis is really true , you accepted a null when it is true [there is really no difference between the groups] Type I error: reject a null hypothesis when it is true (represented by the Greek letter alpha, α) [there is really no difference between the groups] The null hypothesis is really false Type II error: accepted a false null hypothesis (represented by Greek letter beta, β) [there really are differences between the two groups] , you rejected the null hypothesis which is false. [there really are differences between the two groups]

Type I error l Defines the risk that you are willing to take in

Type I error l Defines the risk that you are willing to take in any test of the null hypothesis l l l Conventional: 0. 01 ~ 0. 05 Example: if the level of significance is 0. 05 there is a 5% chance you will make the Type I error: to reject it when the null is true. It is not proper to say “on 100 tests of the null hypothesis, I will make an error on only 5” l As it is normally associated with one test

Type I error l p<. 05 or p<. 01 (reject the null) l l

Type I error l p<. 05 or p<. 01 (reject the null) l l The risk to make the Type I error (reject the real true null) is less than 5% or 1% chance p>. 05 or p=n. s. (nonsignificant) (accept the null) l The probability to make the Type I error (reject the real true null) exceeds. 05 Citation counts Degree Closeness Betweenness 0. 4223 0. 5614 0. 7282 p=0. 0060 p=0. 0100 p=0. 0003

Type II error l l The risk of accepting the real false null hypothesis

Type II error l l The risk of accepting the real false null hypothesis It is sensitive to the number of subjects in a sample l l The size of sample increases Type II error decreases If the sample is more closer to the population, the likelihood that you will accept a false null hypothesis decreases

Significance l Statistical significance means l l Example: Group 1 with training to read

Significance l Statistical significance means l l Example: Group 1 with training to read using computer, Group 2 with training to read using classroom teaching A reading test: Group 1=75. 6, Group 2=75. 7 When using t test: result is statistically significant at the. 01 level How to interpret: computers do better than classroom teaching

Inferential statistics l l Descriptive statistics: describe the characters of a sample Inferential statistics:

Inferential statistics l l Descriptive statistics: describe the characters of a sample Inferential statistics: infer something about the population based on the sample’s character

How inference works l Mother-work group and mother-not-work group l l Select representative samples

How inference works l Mother-work group and mother-not-work group l l Select representative samples of two groups Conduct a test for each member in these two groups, calculate the mean scores Select a proper statistical test Draw a conclusion (to a population): l l If statistically significant: the difference is due to moms If not significant: the difference is not due to moms

How to select which test l l Flow chart (s-p 186) http: //rimarcik. com/en/navigator/

How to select which test l l Flow chart (s-p 186) http: //rimarcik. com/en/navigator/

A template for significant test l l l l 1. a statement of the

A template for significant test l l l l 1. a statement of the null hypothesis 2. setting a level of risk associated with the null hypothesis (level of significance or Type I error, p) 3. select a proper statistical test (see Fig 8. 1) 4. set up the sample and experiment, and compute the test statistic value 5. determine the value needed for rejection of the null hypothesis using proper tables – critical value (see appendix) 6. compare the computed value and the obtained value 7. if computed value > critical value: reject the null; if computed value < critical value: accept the null

Exercise l Are the following statements true, and why: l l l A Type

Exercise l Are the following statements true, and why: l l l A Type I error of 0. 05 means that in 5 tests out of 100 tests of the research hypothesis, I will reject a true null hypothesis It is possible to set the Type I error to 0 The smaller the Type I error rate, the better the results