Why Data Pros Have a Need for Speed

 
 
By Dennis McCafferty  |  Posted 06-29-2016 Email
 
 
 
 
 
 
 
 
 
  • Previous
    Why Data Pros Have a Need for Speed
    Next

    Why Data Pros Have a Need for Speed

    Tech tools that enable near real-time data pipelines are needed in order to keep up with a digital world that’s moving faster than ever.
  • Previous
    Immediate Access
    Next

    Immediate Access

    65% of survey respondents said their organization has near real-time data pipelines already in production, and 24% said they plan to have this before the end of this year.
  • Previous
    Five-Minute Rule
    Next

    Five-Minute Rule

    60% define real-time streaming as the processing of data within or under five minutes of ingesting the data.
  • Previous
    Impactful Outcomes
    Next

    Impactful Outcomes

    71% said they need to leverage "fast data/big data" to power core customer-facing apps, and 68% said this is required to provide analytics to optimize internal business processes.
  • Previous
    Open Up
    Next

    Open Up

    91% said their company leverages open-source distributions in its apps.
  • Previous
    Location, Location, Location
    Next

    Location, Location, Location

    40% host their data pipeline in the cloud and 32% host this on-premise, while 28% do so both in the cloud and on-premise.
  • Previous
    Information Stream
    Next

    Information Stream

    92% said their organization plans to increase its investment in steaming data in the next year.
  • Previous
    Scaling Back
    Next

    Scaling Back

    79% said their company plans to reduce or eliminate investment in batch-only processing.
  • Previous
    Top Data Processing Tools Deployed
    Next

    Top Data Processing Tools Deployed

    Apache Spark: 70%, MapReduce: 50%, Apache Storm: 27%
  • Previous
    Top Data Sink Tools
    Next

    Top Data Sink Tools

    HDFS: 54%, Cassandra: 42%, Elasticsearch: 38%
  • Previous
    Top Message Broker Tools
    Next

    Top Message Broker Tools

    Apache Kafka: 86%, Apache Flume: 22%, RabbitMQ: 21%
  • Previous
    Unsettled Situation, Part I
    Next

    Unsettled Situation, Part I

    55% of survey respondents said the frequency of code changes/pushes cause instability in the data pipeline, while 44% cite a lack of experience.
  • Previous
    Unsettled Situation, Part II
    Next

    Unsettled Situation, Part II

    38% said an inadequate capacity to handle surges creates instability in the data pipeline, while 30% said constant version updates do.
 

As a means of achieving what's called "fast data/big data," a majority of organizations are deploying tech tools which enable near real-time data pipelines, according to a recent survey from OpsClarity. The resulting report, titled "2016 State of Fast Data and Streaming Applications," indicates real-time standards require the processing of data within five minutes of the ingesting of the data. With this, survey respondents say they're better positioned to power customer-facing apps while delivering analytics to optimize internal business processes. In seeking better—and faster—ways to improve data management, companies are planning to boost data streaming capabilities within the next year while hopefully overcoming obstacles in the form of frequent code changes and an inability to effectively respond to data surges. "The digital world is moving faster than ever—with massive volumes of data, more devices, users and applications," according to the report. "To keep up with the pace, large volumes of dynamically changing data must be processed at high velocity. Companies cannot afford to wait for analytics that only kick in at midnight to make critical decisions that impact their business. Real-time insights are imperative. Streaming or fast data applications are driving a new wave of data revolutions." More than 4,000 developers, data architects, DevOps team members, senior IT managers and other IT pros took part in the research.

 
 
 
 
 
Dennis McCafferty is a freelance writer for Baseline Magazine.

 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login Register