Browse Definitions:
Definition

Markov model

Contributor(s): Matthew Haughn

A Markov model is a stochastic method for randomly changing systems where it is assumed that future states do not depend on past states. These models show all possible states as well as the transitions, rate of transitions and probabilities between them. 

Markov models are often used to model the probabilities of different states and the rates of transitions among them.  The method is generally used to model systems. Markov models can also be used to recognize patterns, make predictions and to learn the statistics of sequential data.

There are four types of Markov models that are used situationally:

  • Markov chain - used by systems that are autonomous and have fully observable states
  • Hidden Markov model - used by systems that are autonomous where the state is partially observable. 
  • Markov decision processes - used by controlled systems with a fully observable state.
  • Partially observable Markov decision processes - used by controlled systems where the state is partially observable.

Markov models can be expressed in equations or in graphical models. Graphic Markov models typically use circles (each containing states) and directional arrows to indicate possible transitional changes between them. The directional arrows are labeled with the rate or the variable one for the rate. Applications of Markov modeling include modeling languages, natural language processing (NLP), image processing, bioinformatics, speech recognition and modeling computer hardware and software systems.

Markov models are named after their creator, Andrey Markov, a Russian mathematician in the late 1800s to early 1900s.

An introduction to Markov models:

This was last updated in July 2017

Continue Reading About Markov model

Join the conversation

1 comment

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

What mathematical concept do you wish you'd paid more attention to in high school?
Cancel

-ADS BY GOOGLE

File Extensions and File Formats

Powered by:

SearchCompliance

  • smart contract

    A smart contract, also known as a cryptocontract, is a computer program that directly controls the transfer of digital currencies...

  • risk map (risk heat map)

    A risk map, also known as a risk heat map, is a data visualization tool for communicating specific risks an organization faces. A...

  • internal audit (IA)

    An internal audit (IA) is an organizational initiative to monitor and analyze its own business operations in order to determine ...

SearchSecurity

  • evil maid attack

    An evil maid attack is a security exploit that targets a computing device that has been shut down and left unattended.  An evil ...

  • Common Body of Knowledge (CBK)

    In security, Common Body of Knowledge (CBK) is a comprehensive framework of all the relevant subjects a security professional ...

  • rootkit

    A rootkit is a program or, more often, a collection of software tools that gives a threat actor remote access to and control over...

SearchHealthIT

  • value-based healthcare

    Value-based healthcare, also known as value-based care, is a payment model that rewards healthcare providers for providing ...

  • health informatics

    Health informatics is the practice of acquiring, studying and managing health data and applying medical concepts in conjunction ...

  • clinical trial

    A clinical trial, also known as a clinical research study, is a protocol to evaluate the effects and efficacy of experimental ...

SearchDisasterRecovery

  • crisis communication

    Crisis communication is a method of corresponding with people and organizations during a disruptive event to provide them with ...

  • Zerto

    Zerto is a storage software vendor that specializes in enterprise-class business continuity and disaster recovery in virtual and ...

  • crisis management plan (CMP)

    A crisis management plan (CMP) is a document that outlines the processes an organization will use to respond to a critical ...

SearchStorage

  • hard disk

    A hard disk is part of a unit -- often called a disk drive, hard drive or hard disk drive -- that stores and provides relatively ...

  • cache memory

    Cache memory, also called CPU memory, is high-speed static random access memory (SRAM) that a computer microprocessor can access ...

  • RAID 10 (RAID 1+0)

    RAID 10, also known as RAID 1+0, is a RAID configuration that combines disk mirroring and disk striping to protect data.

SearchSolidStateStorage

  • hybrid hard disk drive (HDD)

    A hybrid hard disk drive is an electromechanical spinning hard disk that contains some amount of NAND Flash memory.

Close