fast data

Fast data is the application of big data analytics to smaller data sets in near-real or real-time in order to solve a problem or create business value.

The goal of fast data is to quickly gather and mine structured and unstructured data so that action can be taken. As the flood of data from sensors, actuators and machine-to-machine (M2M) communication in the Internet of Things (IoT) continues to grow, it has become more important than ever for organizations to identify what data is time-sensitive and should be acted upon right away and what data can sit in a database or data lake until there is a reason to mine it.

The term fast data is often associated with self-service BI and in-memory databases.  The concept plays an important role in native cloud applications that require low latency and depend upon the high I/O capability that all-flash or hybrid flash storage arrays provide.

In the future, it is expected that some fast data applications will rely on rapid batch data while others will require real-time streams. Potential use cases for fast data include:

• Smart grid applications that can analyze real-time electric power usage at tens-of-thousands of locations and automatically initiate load shedding to balance supply with demand in specific geographical areas.     
• Smart window display applications that can identify a potential customer’s demographic profile and generate a discount code or other special offer for him when he enters the store.
• Smart surveillance cameras that can continuously record events and use predictive analytics to identify and flag security anomalies as they occur.