5. Introduction to Probability Theory

The statistician is basically concerned with drawing conclusions (or inference) from experiments involving uncertainties. For these conclusions and inferences to be reasonably accurate, an understanding of probability theory is essential.

In this section, we shall develop the concept of probability with equally likely outcomes.

Experiment, Sample Space and Event

Experiment: This is any process of observation or procedure that:

(1) Can be repeated (theoretically) an infinite number of times; and

(2) Has a well-defined set of possible outcomes.

Sample space: This is the set of all possible outcomes of an experiment.

Event: This is a subset of the sample space of an experiment.

Consider the following illustrations:

Experiment 1: Tossing a coin.

Sample space: S = {Head or Tail} or we could write:

S = {0, 1} where `0` represents a tail and `1` represents a head.

Experiment 2: Tossing a coin twice.

Sample Space:

S = {HH, TT, HT, TH} where

`"H"` represents head and
`"T"` represents tail.

Some possible events:

E₁ = {Head},

E₂ = {Tail},

E₃ = {All heads}

Experiment 3: Throwing a die.

Sample space:

S = {1, 2, 3, 4, 5, 6} or S = {Even, odd}

Some events:

Even numbers, E₁ = {2, 4, 6}

Odd numbers, E₂ = {1, 3, 5}

The number `1`, E₃ = {1}

At least `3`, E₄ = {3, 4, 5, 6}

Experiment 4: Defective items

Two items are picked, one at a time, at random from a manufacturing process, and each item is inspected and classified as defective or non-defective.

Sample space:

S = {NN, ND, DN, DD} where

N = Non-defective

D = Defective

Some events:

E₁ = {only one item is defective} = {ND, DN}

E₂ = {Both are non-defective} = {NN}