Bayesian statistics is a mathematical approach to calculating probability in which conclusions are subjective and updated as additional data is collected. This approach can be contrasted with classical or frequentist statistics, in which probability is calculated by analyzing the frequency of particular random events in a long run of repeated trials, and conclusions are considered to be objective.
Statistical inference, in general, is the process of drawing conclusions from a large data set by analyzing smaller sets of sample data. Bayesian data scientists first analyze sample data and draw a conclusion. This is called the prior inference. Then, they analyze another sample and revise their conclusion. The revised conclusion is called a posterior inference. Using the knowledge of prior events to predict future events is known as Bayesian logic.
Bayesian statistics is named for Thomas Bayes, an 18th-century clergyman and mathematician, who was interested in probability as a way to measure one's belief in a certain hypothesis. Although the Bayesian theory has roots in the 18th-century, the concept took flight in the mid-20th century and has become more popular in recent decades for applications including animal breeding in the 1950s, education measurement in the 1960s and 1970s, spatial statistics in the 1980s, and marketing and political science in the 1990s.
Its iterative approach allows data scientists to make more precise predictions than would be possible by using either data set alone. Today, Bayesian statistics play an important part in machine learning because of the flexibility it provides data scientists working with big data. Bayesian models and methods are used in many industries, including financial forecasting, weather forecasting, medical research and information technology (IT).