Browse Definitions :
Definition

SequenceFile

Contributor(s): Matthew Haughn

A SequenceFile is a flat, binary file type that serves as a container for data to be used in Apache Hadoop distributed computing projects. SequenceFiles are used extensively with MapReduce.

Since Hadoop functions best with larger files, SequenceFiles are used to store and compress files that are smaller than the optimum size for operating efficiently with Hadoop, which can help reduce required disk space capacity and I/O  requirements.

SequenceFiles serve as a container for a sequence of files. Keys are listed for reference and values, and the contents of the file are referenced in a given key. SequenceFiles support a Writer, a Reader and a Sorter class for respective functions in relation to keys. As an example, a SequenceFile might contain a massive number of log files for a server where the key would be a timestamp and the value would be the entire log file. Normally, the small text files would be very inefficient in Hadoop. After packaging into SequenceFiles, however, they can be used effectively.

Beyond packaging files into a manageable size for Hadoop, SequenceFiles support compression of the keys, the values or both.  When both are compressed, the file keys and values are collected into blocks and separately compressed. The type of compression chosen determines the file format.

This was last updated in December 2017

Continue Reading About SequenceFile

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

What steps are you taking to monitor Hadoop processing and ensure that big data workloads run efficiently?
Cancel

-ADS BY GOOGLE

File Extensions and File Formats

SearchCompliance

  • regulatory compliance

    Regulatory compliance is an organization's adherence to laws, regulations, guidelines and specifications relevant to its business...

  • privacy compliance

    Privacy compliance is a company's accordance with established personal information protection guidelines, specifications or ...

  • data governance policy

    A data governance policy is a documented set of guidelines for ensuring that an organization's data and information assets are ...

SearchSecurity

  • asymmetric cryptography (public key cryptography)

    Asymmetric cryptography, also known as public-key cryptography, is a process that uses a pair of related keys -- one public key ...

  • Evil Corp

    Evil Corp is an international cybercrime network that uses malicious software to steal money from its victims' bank accounts.

  • Plundervolt

    Plundervolt is a method of hacking that involves depriving an Intel chip of power so that processing errors occur.

SearchHealthIT

  • telemedicine (telehealth)

    Telemedicine is the remote delivery of healthcare services, such as health assessments or consultations, over the ...

  • Project Nightingale

    Project Nightingale is a controversial partnership between Google and Ascension, the second largest health system in the United ...

  • medical practice management (MPM) software

    Medical practice management (MPM) software is a collection of computerized services used by healthcare professionals and ...

SearchDisasterRecovery

SearchStorage

  • M.2 SSD

    An M.2 SSD is a solid-state drive (SSD) that conforms to a computer industry specification written for internally mounted storage...

  • RAID (redundant array of independent disks)

    RAID (redundant array of independent disks) is a way of storing the same data in different places on multiple hard disks or ...

  • cache memory

    Cache memory, also called CPU memory, is high-speed static random access memory (SRAM) that a computer microprocessor can access ...

Close