# Stochastics Prof. Dr. Stefan Kooths UE/BiTS Hamburg Summer

Stochastics Prof. Dr. Stefan Kooths UE/BiTS Hamburg Summer term 2018 www.kooths.de/bits-stochastics KOOTHS | UE/BiTS: Stochastics (summer term 2018) 1 Contact data Prof. Dr. Stefan Kooths Head of Forecasting Center Kiel Institute for the World Economy Office Berlin In den Ministergrten 8 10117 Berlin 030/2067-9664

[email protected] www.kooths.de KOOTHS | UE/BiTS: Stochastics (summer term 2018) 2 The Kiel Institute for the World Economy Forecasting Center KOOTHS | UE/BiTS: Stochastics (summer term 2018) 3 Be smarter than your phone KOOTHS | UE/BiTS: Stochastics (summer term 2018) 4

Stochastics Stochastic = randomly determined Tossing a coin Rolling a die Stochastics (science of making guesses) We know more than nothing (possible alternatives) but not which alternative comes out for sure Dealing systematically with risk and uncertainty (making the best use of our limited knowledge) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 5 Statistics and stochastics

Statistics: Description of data (e.g. mean value) Stochastics: Reasoning about data (e.g. expected value) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 6 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I

7. Continuous random variables and their distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 7 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their distribution II

8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 8 Basic concepts Random experiment can be infinitely repeated has a well-defined set of two or more possible outcomes has a concrete outcome that is unknown beforehand refers to any process of observation or measurement

Elementary events are individual, mutually exclusive outcomes of a random experiment Sample space () is the set of all possible distinct outcomes of a random experiment Event is a subset of that is composed of one or more elementary events KOOTHS | UE/BiTS: Stochastics (summer term 2018) 9 Examples of random experiments Tossing a coin Possible outcomes: Head and tail = {head, tail}head, tail}

Rolling a die Possible outcomes: Numbers 1 to 6 = {head, tail}1, 2, 3, 4, 5, 6} Rolling two dice Possible outcomes: = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 10 Mathematical sets Set Collection of different objects forming a whole Typically denoted by capital letters (A, B, C ) Symbol for empty sets:

Element Individual object of a set Sets are used to formally handle random experiments KOOTHS | UE/BiTS: Stochastics (summer term 2018) 11 Set operations KOOTHS | UE/BiTS: Stochastics (summer term 2018) 12 Exercise set operations Definitions

= {head, tail}1, 2, 3, 4, 5, 6, 7, 8, 9, 10} Set A: All even numbers in Set B: All odd numbers in Set C: All multiples of three in Tasks a) Write A, B, and C in set notation b) Write A B, A B, A C, A C, and A\C KOOTHS | UE/BiTS: Stochastics (summer term 2018) 13 Probability trees

Probability tree Represents all possible outcomes of a random experiment From a starting point, draw as many branches as there are elementary events New level for each consecutive event Probabilities are noted next to the branches Example: Single toss of a balanced coin KOOTHS | UE/BiTS: Stochastics (summer term 2018) 14 Exercise probability trees You have a box containing eight intact lightbulbs and two defective lightbulbs. You randomly choose a lightbulb from the box. a)

What is the probability of that the chosen lightbulb is 1) intact? 2) defective? b) Draw a probability tree for this experiment. Draw a probability tree for two tosses of a balanced coin. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 15 Probability Classical definition Deducted form combinatory reasoning if all elementary events are equally likely (theoretical sample) Empirical definition

Derived from experience and observation Subjective definition Reflecting personal degree of belief (derived from experience and intuition) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 16 Examples Classical definition Roll a fair die. Let A = {head, tail}1,2}.

Favorable cases: 1 and 2 Possible cases (): 1, 2, 3, 4, 5, 6 P(A) = 2/6 = 1/3 Empirical definition In the city of Alphaville, 5344 people took the driving test last year. 4530 passed the test. For an individual taking the test, the probability of passing (if no other information is known!) can be estimated as P(passing the driving test) = 4530/5344 0.85. Any ideas for improving the estimation? Subjective definition A soccer trainer estimates that the probability of winning the champions league is 40 percent. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 17

Disjoint and non-disjoint sets Disjoint sets A B Have no element in common A and B are disjoint, if their intersection is the empty set: A B = Mutually exclusive sets (A, B, C, ): All pairs are disjoint Non-disjoint sets A B Have at least one element in common

A and B are non-disjoint, if their intersection is not the empty set: A B KOOTHS | UE/BiTS: Stochastics (summer term 2018) 18 Axioms of Probability Non-negativity The probability of an event is a non-negative number 0 P(A) Unitarity The probability of the certain event is always equal to one P() = 1

Additivity Andrey N. Kolmogorov (1903 1987) The probability of mutually exclusive events is the sum of the individual probabilities. P(A B C D ) = P(A) + P(B) + P(C) + P(D) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 19 Basic rules following from Kolmogorovs axioms 0 P(A) 1 for any subset A of P() = 0 P(A) P(B) if A is a subset of B, and A and B are subsets of P(A) = 1 P(A) with A = \A

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 20 Exercises Rules of Probability 20-sided die KOOTHS | UE/BiTS: Stochastics (summer term 2018) 21 Exercices (cont.) Computer sales

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 22 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018)

23 Adding probabilities Object of interest: Probability of observing event A OR event B P(A B) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 24 Adding probabilities Event A, event B: Any two events in the sample space A B General addition rule:

P(A B) = P(A) + P(B) P(A B) Avoiding double-counting! Example In a city, there are two daily newspapers, The Sun and The Post. 22 percent of all households read The Sun, 35 percent read The Post, and 6 percent read both. You chose a household at random: What is the probability that they read a newspaper? Illustrate your reasoning by a Venn diagram. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 25 Multiplying probabilities Object of interest: Probability of observing event A AND event B P(A B)

Joint-probability KOOTHS | UE/BiTS: Stochastics (summer term 2018) 26 Independent events Definition Two events are called independent if the occurrence of either one does not affect the probability of the other one Example: Tossing a coin twice Two-stage random experiment (first toss, second toss) A = Head in the first toss B = Head in the second toss Probability of head in the first

toss and head in the second toss? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 27 Dependent events Definition Two events are called dependent if the occurrence of one event affects the probability of the other event Example: Drawing two cards from a deck of 32 cards (without putting back the first card) Two-stage random experiment (first draw, second draw) A = King in the first draw

B = King in the second draw Probability of drawing two kings? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 28 Conditional probability Probability of event B given that event A has occurred Additional information! P(B|A) = with: P(A) > 0 Conditional probability Joint-probability

A AB B A No information on occurrence of A P(B|A) (A > B)P(A is related B)toifA KOOTHS | UE/BiTS: Stochastics (summer term 2018)

AB B Information on occurrence of A (A B) is related to A 29 Conditional probability: Example taxi A market research agency studied 50 taxi companies in Alphaville to find out about customer satisfaction. In their survey they differentiated between old (more than 10 years in business) and new (less than 10 years in business) taxi companies. Good service

Bad service Old companies 16 4 New companies 10 20 a) What is the probability that a random passenger catches a taxi from a new company and that the service is good?

b) A passenger enters a taxi from a new company. What is the probability that the service is good? c) A passenger complains about bad service. What is the probability that he caught a taxi from a new company? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 30 Multiplication rule A and B are any two events in the sample space P(A B) = P(A)P(B|A) P(A B) = P(B)P(A|B) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 31

Independent and dependent events revisited Marginal (= unconditional) probability P(A) = Probability of A P(B) = Probability of B Conditional probability P(A|B) = Probability of A given B P(B|A) = Probability of B given A Independent events: Conditional probability = unconditional probability P(A|B) = P(A) P(B|A) = P(B) P(A B) = P(A)P(B) Dependent events Conditional probability unconditional probability P(A B) = P(A)P(B|A) = P(B)P(A|B) KOOTHS | UE/BiTS: Stochastics (summer term 2018)

32 Taxi example (cont.) Good service Bad service Old companies 16 4 New companies 10

20 /nn /nn Marginal probabilities Marginal probabilities a) What is the probability that a random passenger catches a taxi from an old (new) company? b) What is the probability that a random passenger enjoys good (bad) service on the next taxi ride? Stochastic dependency c)

Are service quality and company age stochastically dependent or independent? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 33 Theorem of total probability Sample space () P(A1 B) A1 A4 A8 A6

B A2 A3 A5 A7 A9 A1, A2, , Ak: Partition of the sample space (here: k = 9) Mutually exclusive Their union equals P(Ak B)

P(B) = P(B|A1)P(A1) + P(B|A2)P(A2) + + P(B|Ak) P(Ak) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 34 Theorem of total probability: Example construction The completion of a construction job may be delayed because of a strike. The probabilities are: 60 percent that there will be a strike 85 percent that the construction job will be completed if there is no strike 35 percent that the construction job will be completed it there is a strike Events Event A1: There will be a strike Event A2: There will be no strike Event B: Construction job will be completed

Forecast: What is the probability that the construction job will be completed? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 35 Theorem of total probability: Example bad parts Bayes Theorem KOOTHS | UE/BiTS: Stochastics (summer term 2018) 36 Bayes Theorem Combines the multiplication rule and the theorem of total probability

Interprets statistical dependency as cause and effect and allows to identify probable causes for observed effects Thomas Bayes (1701 1761) It is known that B has occurred. What is the probability that B was caused by Aj? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 37 Interpreting Bayes Theorem using probability trees ) 1

P(A P(A 2) A1 P(B|A1) B A2 P(B|A2) B P(A

j) Aj P(B|Aj) B ) Ak P( P(B|Ak) P(A2)P(B|A2)

Ak P(A1)P(B|A1) P(Aj)P(B|Ajj) B P(Ak)P(B|Ak) The probability that event B was reached via the jth branch of the probability tree is the ratio of the probability associated with the jth branch to the sum of the probabilities associated with all k branches of the tree. KOOTHS | UE/BiTS: Stochastics (summer term 2018)

38 Bayes Theorem: Example HIV test Person: neg. Person: pos. Test result: neg. Test result: pos. Germany 2013 80,000 infected persons out of 82,000,000 inhabitants HIV test 99.7 percent of all infected persons are correctly diagnosed

98.5 percent of all non-infected persons are correctly diagnosed Analysis a) A test result indicates an HIV infection (= event B). What is the probability that the tested person is actually infected (= event A 1)? b) What would be the consequences of a compulsory, country-wide HIV test? Source: ZEIT ONLINE (Math up your life!) : Ein einziger Aids-Test reicht nie zur Gewissheit, 1. Dezember 2014. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 39 Example HIV test (cont.) Sample space () = German population

A1 B (pos. HIV-test) (HIV+) A2 (HIV-) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 40 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions

5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 41 Classical probability: Counting vs. combinatorics P(A) = = Example

Rolling two fair dice. What is the probability to get 7 as the sum of both dice (= event A)? A = {head, tail}(1,6),(2,5),(3,4),(4,3),(5,2),(6,1)} |A| = 6 = {head, tail}(1,1),(1,2), , (6,5),(6,6)} || = 36 P(A) = 6/36 = 1/6 Two options Counting (and counting and counting ) Thinking = combinatorial analysis KOOTHS | UE/BiTS: Stochastics (summer term 2018) 42 Permutations Permutation = Arrangement of n distinct objects

Example Colored cards Distinct objects = three colored cards (n=3) How many different arrangements (combinations) exist for a sequence of these cards? = {head, tail}(red,blue,green),(blue,red,green),(green,blue,red), } First position: 3 possibilities Second position: 2 (remaining) possibilities Third position: 1 (resulting) possibility || = 321 = 3! = 6 Read: 3 factorial KOOTHS | UE/BiTS: Stochastics (summer term 2018) 43 Example Colored cards (cont.) (red, blue, green)

(red, green, blue) (blue, red, blue) || = 6 (blue, green, red) (green, red, blue) (green, blue, red) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 44 Exercise Soccer team line-up How many ways are there to line up the players of a soccer team (6 in the back row, 5 in the front row)?

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 45 Example colored cards (cont.) Choosing only two out of the three cards N = 3 (number of objects to choose from) n = 2 (number of objects in each permutation) Drawing cards from the stack with and without replacement With replacement = a color can occur more than once Without replacement = a color cannot occur more than once How many possibilities exist to choose two cards when a) b) c)

d) cards are not replaced and the order matters? cards are replaced and the order matters? cards are not replaced and the order does not matter? cards are replaced and the order does not matter? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 46 Example colored cards (cont.) Results Without replacement With replacement

Order matters a) b) Order does not matter c) d) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 47

The workhorse of combinatorics: The urn model Mental model to systemize the number of possible permutations if n out of N numbered balls are drawn Sometimes the order of the balls matters, sometimes it does not Sometimes the balls are drawn and put back into the urn (= with replacement), sometimes they are drawn without replacement Without replacement With replacement Order matters Order does not matter

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 48 The Binomial Coefficient Number of ways to choose an (unordered) subset of n elements from a fixed set of N elements Read: N choose n KOOTHS | UE/BiTS: Stochastics (summer term 2018) 49 Example German national lottery Six balls are randomly drawn without replacement from a drum containing 49 numbered balls (numbered 1, 2, , 49). The order is irrelevant.

a) How many possible outcomes are there? b) What is the probability to get all 6 numbers right? c) What is the probability to get 5 out of the drawn 6 right? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 50 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their

distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 51 Random variables Random variable Rule that assigns numbers to the outcomes of a random experiment Denoted by capital letter (e.g. X) Domain: Sample space ()

Codomain: Values of a random variable (realizations, observations) Usually real-valued Denoted by corresponding lowercase letters (e.g. x1, x2, ) X: KOOTHS | UE/BiTS: Stochastics (summer term 2018) 52 Random variables: Examples Example 1 Random experiment: Rolling two dice; = {head, tail}(1,1), (1,2) , (6,6)} Random variable: X = Sum of the numbers shown on both dice X {head, tail}2,3,4,,12}

Example 2 Random experiment: A coin is tossed twice Random Y = Number of heads KOOTHS | UE/BiTS: Stochastics (summervariable: term 2018) 53 Random variables: Examples (cont.) Example 3 Random experiment: Waiting at bus stop (bus frequency: 20 min) Random variable: X = Waiting time until next bus arrives in minutes X [0;20] Example 4 Random experiment: Married couple is surveyed

Random variable: Y = Joint income of both partners in euro Y [0;[ Example 5 Random experiment: Lightbulb is chosen from production process Random variable: Z = Durability of chosen lightbulb Z [0;[ KOOTHS | UE/BiTS: Stochastics (summer term 2018) 54 Discrete and continuous random variables Discrete random variable Domain has a finite (countably many) number of realizations Examples 1 and 2 Continuous random variable Domain has an infinite (uncountably many) number of realizations

Examples 3 to 5 More on continuous random variables in chapters 6 and 7 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 55 Probability functions Probability function f(x) Assigns probability to every value x of a discrete random variable X f(xi) = P(X = xi) = pi Can be a table, a function rule, or a mathematical formula Example: Number of heads in two coin tosses Table: x

0 1 2 f(x) 0,25 0,5 0,25 Function rule: Mathematical formula: f(x) = KOOTHS | UE/BiTS: Stochastics (summer term 2018)

56 Requirements for probability functions (1) For each value x there is exactly one function value f(x) (2) For all values x the function values are between 0 and 1 0 f(x) 1 (3) The sum of all function values of all n realizations is always 1 Exercise a) Find the probability function for example 1 and check whether the above requirements are met. b) Can the function given by f(x) = for x {head, tail}1,2,3,4,5} serve as the probability function of a discrete random variable? KOOTHS | UE/BiTS: Stochastics (summer term 2018)

57 Distribution functions Distribution function F(x) Cumulated probability function Gives the probability that the random variable is less than or equal to some real number x F(x) = P(X x) = F() = 0 and F() = 1 Example: Number of heads in two coin tosses F(0) = f(0) F(1) = f(0) + f(1) F(2) = f(0) + f(1) + f(2) KOOTHS | UE/BiTS: Stochastics (summer term 2018)

58 Example 1 (sum of numbers shown by two dice): f(x) and F(x) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 2

3 4 5 6 7 f(x) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 8 9 10

11 12 F(x) 59 Describing probability distributions Characteristic parameters of a random variable Expected value: E(X) = Variance: V(X) = 2 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 60

Expected value E(X) Corresponds to arithmetic mean in descriptive statistics Represents the average value we would get it we repeated the random experiment many times Weighs all realizations by their probabilities E(X) = = Properties E(c) = c with c = const. E(X + c) = E(X) + c E(cX) = cE(X) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 61

Variance and standard deviation V(X) Describes the scatter of realizations are around the expected value Average squared deviation from the expected value if the random experiment was repeated many times Weighs all squared deviations by their probabilities V(X) = 2 = E((X - )2) = = - = E(X2) Standard deviation: = Properties V(c) = 0 with c = const. V(X + c) = V(X) V(cX) = c2V(X) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 62 Exercise: Average deviation from expected value

Why does the simple (= non-squared) average deviation from the expected value not tell us anything about the scatter of realizations around their mean? = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 63 Example 1: Mean and variance kooths-stochastics-Chapter4-Example1.xlsx KOOTHS | UE/BiTS: Stochastics (summer term 2018) 64

Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 65 Usefulness of specific discrete distributions Family of similar random experiments (principle)

Application to concrete problems by adapting parameters Distributions Binomial Hypergeometric Poisson KOOTHS | UE/BiTS: Stochastics (summer term 2018) 66 Bernoulli experiment Random experiment Two disjoint elementary events (success, failure) Any two experiments are statistically independent (drawing with replacement) Probability of success p is constant (consequently, probability of failure q = 1-p = const.)

Jakob Bernoulli (1655 1705) Example: Repeated flipping of a coin (fair or biased) Two disjoint outcomes: Head or Tail Coin has no memory, so repeated runs are independent The probability of Head is const, so is the probability of Tail o p = q = 0.5 (fair coin) o p 0.5, q = 1-p 0.5 (biased coin) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 67 Experimental learning Rolling a fair die 8 times (or 8 fair dice simultaneously) X = number of dice showing a 6 https://www.random.org/dice/

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 68 Experimental learning Round Number of dice showing 6 relative frequency 1 2 3 4 5 6

7 8 9 10 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 69 Binomial distribution Repeated Bernoulli experiments p = const. = probability of success n = number of experiments Random variable X = number of successful outcomes P(X=x) : probability of x successful outcomes Solution Multiplication rule for independent events Combinatorics (urn model: irrelevant order, without replacement)

P(X|n,p) = pxqn-x = px(1-p)n-x number of possible outcomes for X=x probability of one possible outcome for X=x KOOTHS | UE/BiTS: Stochastics (summer term 2018) 70 Bernoulli experiment: Urn model interpretation Urn with pN black and (1-p)N white balls Probability of drawing x black balls order does not matter with replacement Note: Illustration for random experiment, NOT for combinations of possible outcomes! KOOTHS | UE/BiTS: Stochastics (summer term 2018)

71 Binominal distribution: Table Probabilities of all x for typical values of n and p Example customers: The probability that a randomly selected customer makes a purchase is 20 percent a) What is the probability that 4 out of 6 customers make a purchase? b) What is the probability that no more than 3 customers decide to buy? c) What is the probability that

at least one customer makes the purchase? http://health.uottawa.ca/biomech/courses/apa3381/Binomial%20table.pdf KOOTHS | UE/BiTS: Stochastics (summer term 2018) 72 Binomial distribution: Expected value and variance Expected value: E(X) = np Variance: V(X) = npq = np(1-p) Exercises a) Calculate and interpret the expected value and the variance for the customers example on the previous slide b) Calculate and interpret the expected value and the variance for o p=1 o p=0

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 73 Hypergeometric distribution Urn with M black and N-M white balls Drawing n balls Success = drawing a black ball, X = number of successes Probability of drawing X=x black balls order does not matter without replacement different from binomial distribution

p is no longer constant as events are statistically dependent P(X|N,M,n) = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 74 Hypergeometric distribution: Examples Example employees Out of 6 employees, three have been with the company for more than 5 years. If we now randomly select 4 employees, what is the probability that exactly two of them have been with the company for more than 5 years? What is N, what is M, what is n? Example lottery 6 out of 49 What is the probability to guess four numbers right in a lottery where 6 out of 49 numbered balls are drawn?

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 75 Hypergeometric distribution: Expected value and variance Expected value: E(X) = n Variance: V(X) = n - ) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 76 Poisson distribution Probability of a number of events in a fixed interval of time/space/distance for which the average number of can be expected Two events do not occur at exactly same time

The probability for an event in a very short interval is very small (distribution of rare events) Denis Poisson The probability of a success is proportional to (1781 1840 ) the size of the interval For mutually exclusive intervals the number of successes are independent P(X|) = E(X) = V(X) = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 77 Poisson distribution: Examples

1) Events: Phone calls arriving at a telephone hotline. = average number of calls arriving per hour. 2) Events: Defects in the insulation of an undersea cable. = average number of defects per kilometer of cable. 3) Events: Houses sold by a real estate company. = average number of houses sold per day. 4) Events: Electrons emitted by a piece of radioactive material. = average number of electrons emitted per minute 5) Events: Defects in sheet metal. = Average number of defects per square meter of metal. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 78 Poisson distribution for approximations Binomial distribution n (high number of repetitions, n > 10) p0

(small probability of each event, p < 0.05) = np Hypergeometric distribution n is large (high number of repetitions, n > 10) M/N is small (M/N < 0.05) N is large relative to n (n/M < 0.05) = n KOOTHS | UE/BiTS: Stochastics (summer term 2018) 79 Poisson distribution: Exercise subway crime It is known that in the subway system of Alphaville, there are on average four pickpocketing incidents per hour. What is the

probability a) that there are no pickpocketing incidents in a given hour? b) that there are seven pickpocketing incidents in a given hour? c) that there are 50 incidents in a given day? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 80 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their distribution II 8. Estimation and confidence intervals

9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 81 Discrete and continuous random variables (recap) Discrete random variable Domain has a finite (countably many) number of realizations Examples 1 and 2 (slide 53) Each value of X can be assigned a probability P(X=x) = f(x) Probability function

Continuous random variable Domain has an infinite (uncountably many) number of realizations Examples 3 to 5 (slide 54) Each individual value of X has a probability of zero focus on intervals Density function KOOTHS | UE/BiTS: Stochastics (summer term 2018) 82 Density function Definition Areas under the curve of this function give the probabilities associated

with the corresponding intervals along the x-axis Example bus stop Busses depart every ten minutes from a bus stop. If you arrive at the bus stop at a random time, the waiting time X until the next bus departs is a continuous random variable which can take any value between zero minutes and then minutes. f(x) = Sketch f(x) and determine (visually) the probability that the waiting time is between five and seven minutes KOOTHS | UE/BiTS: Stochastics (summer term 2018) 83 Density function f(x) and distribution function F(x) Density function P(a x b) =

f(x) 0 P(- x +) = = 1 Distribution function F(x) = P(X x) = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 84 Expected value and variance Expected value E(X) = = Variance V(X) = 2 = E((X - )2) = = Example bus stop (cont.)

Calculate the expected value and the variance for the waiting time at the bus stop. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 85 Uniform distributions Definition Density function is constant over some interval [a;b] f(X) = Expected value E(X) = = = Variance V(X) = 2 = E((X - )2) = =

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 86 Normal distribution (= Gauss distribution) Cornerstone of modern statistical theory Many random processes follow a normal distribution Other distributions can be approximated by the normal distribution (under certain conditions) All expected values are normally distributed for sufficiently large sample sizes (n > 30) Discovered in the 18th century (typical pattern for measurement errors) Abraham de Moivre (1667 1754)

Carl Friedrich Gau (1777 1855) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 87 Normal distribution: The (magic) formula A random variable X has a normal distribution (= is normally distributed) if its density function is given by N(X|; 2) = f(x) = Expected value: Variance: 2 Bell-shaped graph KOOTHS | UE/BiTS: Stochastics (summer term 2018)

88 How and affect the shape of the Gaussian density function Example: = 4 and = 1.5 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 89 How and affect the shape of the Gaussian density function shifts the graph horizontally KOOTHS | UE/BiTS: Stochastics (summer term 2018) 90 How and affect the shape of the Gaussian density function stretches or compresses the graph around its center

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 91 Z-transformation (standardization): The standard normal distribution Z-transformation: Z = Expected value: E(Z) = 0 Variance: V(Z) = 1 Simplifies tabulation of values of the normal distribution Step 1: Standardize normally distributed X Step 2: Lookup probabilities of the standardized normal distribution KOOTHS | UE/BiTS: Stochastics (summer term 2018) 92

Standard normal distribution: Table KOOTHS | UE/BiTS: Stochastics (summer term 2018) 93 Exercises X is a N(10; 25)-distributed variable. Use the table of the standard normal distribution and determine: a) P(0 X 11) b) P(8 X 12) c) P(X 15) A radio station has conducted a survey among 760 listeners. The result is that a certain rock program is listened to for 10 minutes on average. The standard deviation is 2 minutes.

a) Why is the information on the number of surveyed listeners important? b) What percentage of all the listeners is tuned in for a duration between 9 and 11 minutes? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 94 Exercise electronic parts The life span of an electronic part is normally distributed and the expected use time is 2,000 hours while the standard deviation is 200 hours. a) b) c)

d) e) f) g) Illustrate the z-transformation that turns this distribution into a standard normal distribution with the help of a sketched diagram. Determine the probability that one randomly chosen part has a use time between 2,000 and 2,400 hours. Do this both graphically with your sketched diagram and with the help of the table of the standard normal distribution. What is the probability that the life span is less than 1,800 hours? What is the probability that the life span is greater than 2,200 hours? What is the probability that the life span is greater than 1.900 hours and less than 2.300 hours? Which use time will not be exceeded with a probability of 90 percent? What is the maximum use time of those 20 percent of parts with the lowest quality (i.e. use time).

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 95 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 96

Exponential distribution Random variable T Time that elapses between two single events that follow a Poisson distribution P(T t) = 1 e-t = 1 E(T) = V(X) = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 97 Examples On average an office receives 5 calls per hour. What is the probability that there will be one incoming call in the next 30 minutes?

A speed control identifies 4 speeding incidents per hour on average. What is the probability that they find one speeder in the next 20 minutes? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 98 Approximations of distributions Generally used if the actual distribution is difficult to compute. The quality of an approximation always depends on the parameters of the actual distribution. Trade-off between accuracy and calculation effort Of course, an approximation is never perfect (trade-off between accuracy and calculation effort). But for most purposes and under

most circumstances it is well enough and it saves a lot of work. Review: Poisson distribution (slide 79) Binomial distribution Hypergeometric distribution KOOTHS | UE/BiTS: Stochastics (summer term 2018) 99 Approximations using the normal distribution Binomial distribution (for large sample sizes) n 30 np 5 nq = n(1p) 5 Poisson distribution (for large )

10 KOOTHS | UE/BiTS: Stochastics (summer term 2018) Normal distribution = np = Normal distribution = = 100 Approximations for the Binomial distribution: Example 1 Random experiment: 50-time coin toss. What is the probability to obtain 20 tails? Exact solution

P(X=20|n = 50, p = 0.5) = 0.520 0.530 = 0.0419 Approximation n = 50 30 np = 500.5 = 25 5 nq = 500.5 = 25 5 P(X=20|n = 50, p = 0.5) P(19.5 X 20.5| = 25, = ) z1 = = -1.56 z2 = = -1.27 P(z1 Z z2) = 0.4406 0.3980 = 0.0426 KOOTHS | UE/BiTS: Stochastics (summer term 2018)

101 Approximations for the Binomial distribution: Example 2 Random experiment: 50-time coin toss. What is the probability to obtain anything between 20 and 35 tails? Exact solution (very tiring) Approximation n = 50 30 np = 500.5 = 25 5

nq = 500.5 = 25 5 P(20 X 35|n = 50, p = 0.5) P(19.5 X 35.5| = 25, = ) z1 = = -1.56 z2 = = 2.97 P(z1 Z z2) = 0.4406 + 0.4985 = 0.9391 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 102 Approximations for the Binomial distribution: Example 3 The call of a sales agent ends with a sale in 20 percent of the cases. If he makes 30 phone calls, what is the probability that he makes at least 10 sales? KOOTHS | UE/BiTS: Stochastics (summer term 2018) 103

Poisson distribution for increasing values of KOOTHS | UE/BiTS: Stochastics (summer term 2018) 104 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I 7. Continuous random variables and their distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 105 Statistical inference Sample statistical inference Population Collecting a sample and analyzing the sample data to infer properties (e.g. mean value) about a population, which is larger than the observed sample data set Point estimate Interval estimate Hypothesis testing (next chapter)

Less precise than a full census, but cheaper/faster/feasible KOOTHS | UE/BiTS: Stochastics (summer term 2018) 106 Samples and populations Sample (size = n) statistical inference Samples have statistics (Latin letters) _ Sample mean: x

Sample variance: s2 Sample standard deviation: s Random variables! KOOTHS | UE/BiTS: Stochastics (summer term 2018) Population (size = N) Populations have parameters (Greek letters) Mean: Variance: 2 Standard deviation: Typically unknown but of interest 107

Sample statistics and point estimation Sample mean: = Random variable Random variables 2 Sample variance: s = Sample standard deviation: meaningful s = for n>1 only When we use the value of a sample statistic to estimate a population parameter, we call this point estimation and we refer to the statistic as a point estimator of the parameter. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 108

Desired characteristics of point estimators Unbiasedness: Expected value of point estimator = population parameter (no systematic tendency to underestimate or overestimate the truth) Small standard error* (= high efficiency): High concentration of point estimator around population parameter *standard error = standard deviation of the point estimator KOOTHS | UE/BiTS: Stochastics (summer term 2018) 109 Example There are five houses in Betastreet of Alphaville (=

population). The number of children per house is as follows: Population mean: = 1 For budgetary reasons, a survey company can only sample two houses in the street (n=2). If the randomly chosen houses are #3 and #4, then the sample mean is x_ = 0.5 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 110 Unbiasedness of a point estimator: Example (cont.) Expected value of point estimator = population parameter _ = 1 = x_is an unbiased estimator of E(x) KOOTHS | UE/BiTS: Stochastics (summer term 2018)

111 Law of large numbers and standard error The larger the sample size the higher the precision of the point estimators Law of large numbers: The sample mean approaches the population mean as the sample size grows Standard error of sample means (= standard deviation of the distribution of the sample means): _X = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 112 Interval estimation: Confidence intervals for the mean

Point estimator says nothing about its precision Interval estimation: Finding the range (based on a sample) that contains the true parameter of the population with a given probability (= confidence level ) Requires information on the distribution of the point estimator (here: sample mean) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 113 The statisticians workhorses Normal distribution for the sample mean Sample size is large (n 30): Central limit theorem (see next slide) Sample size is small (n < 30) and the population is normally distributed and is known

Students t-distribution for the sample mean Sample size is small (n < 30) and the population is normally distributed and is unknown KOOTHS | UE/BiTS: Stochastics (summer term 2018) 114 Central limit theorem Draw a random sample of size n from any population with mean and standard deviation . _ When n > 30, the random variable X (= sample mean) follows a normal distribution with _X = _X =

regardless of the population distribution from which the data are sampled. KOOTHS | UE/BiTS: Stochastics (summer term 2018) 115 Normal distribution and standard normal distribution z -3 -2 KOOTHS | UE/BiTS: Stochastics (summer term 2018)

-1 0 +1 +2 +3 116 Solving Z-transformation for population mean _ Z-transformation: Z = = X Xz _ _ _

Confidence level z-value (from table) 1 = 90 percent z1 = 1.64 2 = 95 percent z2 = 1.96 3 = 99 percent z3 = 2.58 Confidence intervals _ _ P(X z1 X + _Xz1) = 0.90 _ _ _ P(X

z X + Xz2) = 0.95 2 _ _ _ P(X Xz3 X + Xz3) = 0.99 _ X _ X _

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 117 Confidence intervals: Example exam results An exam was taken by 194 students (= population) Arithmetic mean of results: = 64.12 Standard deviation of results: = 27.73 10 samples of size n = 30 Confidence intervals for confidence levels of 90 %, 95 %, 99 % KOOTHS | UE/BiTS: Stochastics (summer term 2018) 118 Confidence intervals: Example exam results (cont.)

With increasing confidence level the intervals become larger Intervals differ for each sample True value not always enclosed by interval KOOTHS | UE/BiTS: Stochastics (summer term 2018) 119 Vertrauen ist gut, Konfidenz ist besser Source: Oestreich and Romberg (2010) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 120 Survey design: Precision and necessary sample size Solving the Z-transformation for n (if is known) Z= = =

Precision (= acceptable deviation) _ Degree of certainty n = 2 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 121 Excursus: Students t-distribution (n < 30, unknown) Standard normal distribution t-distribution Previous plots for

t-distribution degrees of freedom v = n-1 Source: https://en.wikipedia.org/wiki/Student%27s_t-distribution KOOTHS | UE/BiTS: Stochastics (summer term 2018) 122 Outline 1. Probabilities I 2. Probabilities II 3. Permutations and combinations 4. Discrete random variables and distributions 5. Specific discrete distributions 6. Continuous random variables and their distribution I

7. Continuous random variables and their distribution II 8. Estimation and confidence intervals 9. Statistical hypothesis test KOOTHS | UE/BiTS: Stochastics (summer term 2018) 123 Statistical inference Sample statistical inference Population Collecting a sample and analyzing the sample data to infer properties (e.g. mean value) about a population, which is

larger than the observed sample data set Point estimate Interval estimate Hypothesis testing Similar calculations Less precise than a full census, but cheaper/faster/feasible KOOTHS | UE/BiTS: Stochastics (summer term 2018) 124 General approach to hypothesis testing Hypothesis for the value of a population parameter Example: Mean filling level of beer bottles is 0.5 litres (0) Take random sample and calculate sample parameter (x) Example: 100 bottles are checked for filling level, x = 0.498

Check whether sample parameter is in line with 0 Using probability distributions one evaluates deviations between hypothetical value and sample value How likely is the observed sample value? Goal: Rejecting the so-called null hypothesis (H0) in favor of the alternative hypothesis (H1) Court analogy: Not guilty (H0) vs. guilty (H1) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 125 Error types and level of significance Reality Test result

H0 is true H1 is true Reject H0 Type I error P(Type I error) = Correct decision Do not reject H0 Correct decision Type II error

P(Type II error) = Stochastical evidence is based on probability distributions, hence the jugdement is never for 100 percent sure! = level of significance (= error probability) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 126 Test design: Two-sided and one-sided Two-sided tests H0: = 0 H1: 0 Error probability is /2 for values from the sample that are too high, and /2 for values from the sample that are too low One-sided tests (upper bound)

H0: = 0 H1: < 0 Error probability is for values from the sample that are too low One-sided tests (lower bound) H0: = 0 H1: > 0 Error probability is for values from the sample that are too high KOOTHS | UE/BiTS: Stochastics (summer term 2018) 127 Rationale of test statistics The test is based on a sample statistic that estimates the population parameter that appears in the hypotheses. Usually this is the same estimate that we would use in a confidence interval for the parameter. If H0 is true then the estimate should take a value near the parameter value specified by H0.

Values of the estimate far from the parameter value specified by H0 give evidence against H0. The alternative hypothesis determines which directions count against H0. To assess how far the estimate is from the parameter, we must standardize the estimate. In many common situations the test statistic (standardized estimate) has the form: test statistic = KOOTHS | UE/BiTS: Stochastics (summer term 2018) 128 One-sided tests: Example KOOTHS | UE/BiTS: Stochastics (summer term 2018) 129 Critical value and rejection region

Test design (H0, H1) Level of significance ( = probability of type I errors) Assumptions on probability distribution in population Critical value for rejecting H0 (separates rejection region from the rest of the data space) _ Value of test statistic (xL) Value of standardized test statistic (z) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 130 Critical value: Example KOOTHS | UE/BiTS: Stochastics (summer term 2018)

131 Critical value: Example (cont.) KOOTHS | UE/BiTS: Stochastics (summer term 2018) 132 p-value approach Critical value Given level of significance critical value if H0 holds Compare value of test statistic with critical value p-value Probability of observing test statistic if H0 holds (= p-value) Compare p-value with level of significance Identical decision for rejection of H0

KOOTHS | UE/BiTS: Stochastics (summer term 2018) 133 Critical values and p-value approach: Example KOOTHS | UE/BiTS: Stochastics (summer term 2018) 134 Trade-off between Type I and Type II errors Calculation of type II errors requires crisp value for alternative hypothesis Example H0: = 170 H1: = 180 = 0.05

= 65 n = 400 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 135 Summary: Steps for hypothesis testing 1) Formulate hypotheses (H0 and H1, one-sided or two-sided) 2) 3) 4) 5) 6) Sample data from the population of interest Compute the sample statistics you need from the data Compute the test statistic from the sample statistics

Convert the test statistic into a probability (p-value) Formulate a conclusion regarding H0 KOOTHS | UE/BiTS: Stochastics (summer term 2018) 136

## Recently Viewed Presentations

• Trust Positions: Policy and Recommendations Trust Personnel* a. Identification of Trust Positions: The following shall be considered positions of trust: the Executive Secretary for Integral Development, designated Director General of the IACD, as well as the Assistant Secretaries, the advisors...
• 3D Animation Rendering 3D Scene and Motion Sequence of Frames Rates: Video 30fps, Film 24fps Persistence of Vision Animator must create .. Illusion of Life Weight Animation Almost every property of every object in the scene can be animated (changed...
• A modified Harvard architecture machine is very much like a Harvard architecture machine, but it relaxes the strict separation between instruction and data while still letting the CPU concurrently access two (or more) memory buses. The most common modification includes...
• Strategy for Growth in International Tourism to South Africa Presentation to June 2003 DEPARTMENT OF ENVIRONMENTAL AFFAIRS AND TOURISM Presentation Structure South African Tourism is reconfiguring itself to drive tourism's contribution to the nation's transformation programme Domestic and International Short-Haul...
• Walk through a day in PK or KG to include cafe and recess . procedures. PK and K students will all go to cafe for first two weeks to meet their teacher then walk down to classroom as a class....
• Particle Interactions Concept of exchange particles to explain forces between elementary particles The electromagnetic force; virtual photons as the exchange particle. The weak interaction limited β-, β+ decay, electron capture and electron-proton collisions; W+ and W- as the exchange particles.
• International diplomacy examples. Definition. Nonverbal communication refers to behaviors that form a socially shared coding system. ... Chronemics. Taking More of a Communication Perspective. Shifting emphases: From individuals to dyads and interaction.
• Essential idea: Electric cells allow us to store energy in a chemical form.. Nature of science: Long-term risks: Scientists need to balance the research into electric cells that can store energy with greater energy density to provide longer device lifetimes...