Key Probability Definitions and Rules
Sample Space Definition
A sample space is a collection or a set of possible outcomes of a random experiment. The sample space is represented using the symbol S. The subset of possible outcomes of an experiment is called an event. A sample space may contain a number of outcomes that depend on the experiment. If it contains a finite number of outcomes, then it is known as a discrete or finite sample space.
Probability of an Event
An event in probability can be defined as a set of outcomes of an experiment. In other words, an event in probability is the subset of the respective sample space.
Types of Probability Events
Impossible and Sure Events
If the probability of occurrence of an event is 0, such an event is called an Impossible Event. If the probability of occurrence of an event is 1, it is called a Sure Event.
Simple Events
Any event consisting of a single point of the sample space is known as a Simple Event in probability. For example, if S = {56, 78, 96, 54, 89} and E = {78}, then E is a simple event.
Compound Events
Contrary to the simple event, if any event consists of more than one single point of the sample space, then such an event is called a Compound Event.
Independent and Dependent Events
If the occurrence of any event is completely unaffected by the occurrence of any other event, such events are known as an Independent Event in probability. Events which are affected by other events are known as Dependent Events.
Mutually Exclusive Events
If the occurrence of one event excludes the occurrence of another event, such events are Mutually Exclusive Events.
Exhaustive Events
A set of events is called Exhaustive Events if all the events together consume the entire sample space.
Conditional Probability Explained
The conditional probability of A given B, denoted P(A|B), is the probability that event A has occurred in a trial of a random experiment for which it is known that event B has definitely occurred. It may be computed using the following formula:
Rule for Conditional Probability:
P(A|B) = P(A ∩ B) / P(B)
Independent Events Formula
Events A and B are independent if:
P(A ∩ B) = P(A) ⋅ P(B)
If A and B are not independent, then they are dependent.
Understanding Bayes’ Rule
Bayes’ Rule is a fundamental rule in data science. It is the mathematical rule that describes how to update a belief, given some evidence. In other words – it describes the act of learning.
The equation itself is not too complex:
There are four parts:
- Posterior probability
- Prior probability
- Likelihood
- Marginal probability
It is named after Thomas Bayes, an 18th-century English theologian and mathematician. Bayes originally wrote about the concept, but it did not receive much attention during his lifetime.
Mathematical Expectation (Expected Value)
Mathematical expectation, also known as the expected value, is the summation or integration of possible values from a random variable. It is also known as the product of the probability of an event occurring, denoted P(x), and the value corresponding with the actual observed occurrence of the event. The expected value is a useful property of any random variable. Usually notated as E(X), the expected value can be computed by the summation over all the distinct values that the random variable can take.
The mathematical expectation is given by the formula: E(X) = Σ xipi = x1p1 + x2p2 + … + xnpn, where xi represents a distinct value of the random variable, pi is the probability of that value occurring, and n is the total number of distinct values.
The mathematical expectation of an indicator variable can be zero if there is no occurrence of an event A, and the mathematical expectation of an indicator variable can be one if there is an occurrence of an event A. Thus, it is a useful tool to find the probability of event A.