Browse Definitions :
Definition

SequenceFile

Contributor(s): Matthew Haughn

A SequenceFile is a flat, binary file type that serves as a container for data to be used in Apache Hadoop distributed computing projects. SequenceFiles are used extensively with MapReduce.

Since Hadoop functions best with larger files, SequenceFiles are used to store and compress files that are smaller than the optimum size for operating efficiently with Hadoop, which can help reduce required disk space capacity and I/O  requirements.

SequenceFiles serve as a container for a sequence of files. Keys are listed for reference and values, and the contents of the file are referenced in a given key. SequenceFiles support a Writer, a Reader and a Sorter class for respective functions in relation to keys. As an example, a SequenceFile might contain a massive number of log files for a server where the key would be a timestamp and the value would be the entire log file. Normally, the small text files would be very inefficient in Hadoop. After packaging into SequenceFiles, however, they can be used effectively.

Beyond packaging files into a manageable size for Hadoop, SequenceFiles support compression of the keys, the values or both.  When both are compressed, the file keys and values are collected into blocks and separately compressed. The type of compression chosen determines the file format.

This was last updated in December 2017

Continue Reading About SequenceFile

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

What steps are you taking to monitor Hadoop processing and ensure that big data workloads run efficiently?
Cancel

-ADS BY GOOGLE

File Extensions and File Formats

Powered by:

SearchCompliance

SearchSecurity

  • GPS jamming

    GPS jamming is the process of using a frequency transmitting device to block or interfere with radio communications.

  • time-based one-time password (TOTP)

    A time-based one-time password (TOTP) is a temporary code, generated by an algorithm, for use in authenticating access to ...

  • Security Operations Center (SOC)

    A security operations center (SOC) is a command center facility for a team of IT professionals with expertise in information ...

SearchHealthIT

SearchDisasterRecovery

  • disaster recovery team

    A disaster recovery team is a group of individuals focused on planning, implementing, maintaining, auditing and testing an ...

  • cloud insurance

    Cloud insurance is any type of financial or data protection obtained by a cloud service provider. 

  • business continuity software

    Business continuity software is an application or suite designed to make business continuity planning/business continuity ...

SearchStorage

  • SSD (solid-state drive)

    An SSD (solid-state drive) is a type of nonvolatile storage media that stores persistent data on solid-state flash memory.

  • NAND flash memory

    NAND flash memory is a type of nonvolatile storage technology that does not require power to retain data.

  • storage class memory (SCM)

    Storage class memory (SCM) is a type of NAND flash that includes a power source to ensure that data won't be lost due to a system...

Close