Browse Definitions :
Definition

Avro (Apache Avro)

Contributor(s): Matthew Haughn

Apache Avro is a row-oriented object container storage format for Hadoop as well as a remote procedure call and data serialization framework. Hadoop is a free, Java-based programming framework that supports the processing of large data sets in a distributed computing environment. Avro is optimized for write operations and includes a wire format for communication between nodes.

Avro makes translation between different nodes by way of the data definition and serialized permanent data. Avro uses JavaScript object notation to define the data types and protocols. The data is streamed in an efficient and compact binary format. An Avro container file consists of a header and one or multiple file storage blocks.

The header is made up of:

  • 4 bytes of ASCI “OBJ1”
  • File metadata including the schema definition
  • A sync marker: 16 bytes of randomly generated code

Avro also includes its own interface descriptor language (IDL) also named Avro, aside from JSON to define data types and protocols. IDL eases adoption by users who are used to more common traditional IDLs, which have a syntax more like C/C++.

Avro is a top-level project sponsored by the Apache Software Foundation (ASF).

This was last updated in January 2018

Continue Reading About Avro (Apache Avro)

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

-ADS BY GOOGLE

File Extensions and File Formats

Powered by:

SearchCompliance

  • risk management

    Risk management is the process of identifying, assessing and controlling threats to an organization's capital and earnings.

  • compliance as a service (CaaS)

    Compliance as a Service (CaaS) is a cloud service service level agreement (SLA) that specified how a managed service provider (...

  • data protection impact assessment (DPIA)

    A data protection impact assessment (DPIA) is a process designed to help organizations determine how data processing systems, ...

SearchSecurity

  • quantum key distribution (QKD)

    Quantum key distribution (QKD) is a secure communication method for exchanging encryption keys only known between shared parties.

  • identity theft

    Identity theft, also known as identity fraud, is a crime in which an imposter obtains key pieces of personally identifiable ...

  • cybercrime

    Cybercrime is any criminal activity that involves a computer, networked device or a network.

SearchHealthIT

SearchDisasterRecovery

  • disaster recovery plan (DRP)

    A disaster recovery plan (DRP) is a documented, structured approach that describes how an organization can quickly resume work ...

  • business continuity plan (BCP)

    A business continuity plan (BCP) is a document that consists of the critical information an organization needs to continue ...

  • disaster recovery team

    A disaster recovery team is a group of individuals focused on planning, implementing, maintaining, auditing and testing an ...

SearchStorage

  • logical unit number (LUN)

    A logical unit number (LUN) is a unique identifier for designating an individual or collection of physical or virtual storage ...

  • NVMe over Fabrics (NVMe-oF)

    NVMe over Fabrics, also known as NVMe-oF and non-volatile memory express over fabrics, is a protocol specification designed to ...

  • CIFS (Common Internet File System)

    CIFS (Common Internet File System) is a protocol that gained popularity around the year 2000, as vendors worked to establish an ...

Close