March 30, 2011

Short-request and analytic processing

A few years ago, I suggested that database workloads could be divided into two kinds — transactional and analytic. The advent of non-transactional NoSQL has suggested that we need a replacement term for “transactional” or “OLTP”, but finding one has been a bit difficult. Numerous tries, including high-volume simple processing, online request processing, internet request processing, network request processing, short request processing, and rapid request processing have turned out to be imperfect, as per discussion at each of those links. But then, no category name is ever perfect anyway. I’ve finally settled on short request processing, largely because I think it does a good job of preserving the analytic-vs-bang-bang-not-analytic workload distinction.

The easy part of the distinction goes roughly like this:

Where the terminology gets more difficult is in a few areas of what one might call real-time or near-real-time analytics. My first takes are: 

Indeed, one of my top trends to watch these days is the integration of short request and analytic processing. Several different approaches come to mind.

You can do everything in a single instance of a general-purpose DBMS such as Oracle, DB2, Microsoft SQL Server, Sybase ASE, or MySQL. For sufficiently small enterprises with sufficiently undemanding workloads, that’s the best approach. Tokutek evidently aspires to be an improved version of the same thing.

You can do it in an analytic DBMS that is sufficiently strong in user concurrency, update speed, and so on. This is the sweet spot of Teradata’s market. It’s also where SAP HANA is alleged to be going.

You can tie together DBMS optimized for short-request and analytic processing (or use something like Hadoop for the analytics, whether or not it should be considered as a DBMS). E.g., Membase (now Couchbase) has integration stories with Hadoop and Vertica, at a couple of clients each. I think this is a major untapped opportunity in the MySQL world, and have been raising that point with various companies for some time.

You can graft short request processing onto analytic system. That’s the point of HBase.

You can superpose analytics on a short request processing system. That’s the point of DataStax Brisk.

Sybase RAP, depending on how it is configured, can fit several of these models. The same could be said of Oracle (especially Exadata) or DB2.

Fun times.


33 Responses to “Short-request and analytic processing”

  1. Starcounter | DBMS 2 : DataBase Management System Services on April 13th, 2011 12:55 pm

    […] seems to be offering an in-memory object-based/object-oriented/whatever short-request DBMS that also talks SQL. I haven’t been briefed at this point, and hence don’t have detail […]

  2. Notes on short-request scale-out MySQL | DBMS 2 : DataBase Management System Services on April 19th, 2011 4:52 am

    […] that sounds as if he was asking about scale-out relational DBMS in general, MySQL or otherwise, short-request or analytic, it turned out that he was asking just about short-request scale-out MySQL. My thoughts and […]

  3. The Ted Codd guarantee | DBMS 2 : DataBase Management System Services on July 31st, 2011 5:44 pm

    […] never quite worked out that way. For most of the history of tabular DBMS, the best-performing short-request and analytic DBMS have been designed quite differently from each other.* Non-relational systems — from […]

  4. “Enterprise-ready Hadoop” | DBMS 2 : DataBase Management System Services on June 19th, 2012 6:14 pm

    […] across-the-board competitive with analytic relational DBMS. (And the same goes for HBase vs. short-request alternatives.) But the real question is whether its features are good enough for a variety of […]

  5. Notes on HBase 0.92 | DBMS 2 : DataBase Management System Services on June 19th, 2012 6:15 pm

    […] you do short-request and MapReduce processing against the same HBase database, the MapReduce part is usually still done […]

  6. Clustrix 4.0 and other Clustrix stuff | DBMS 2 : DataBase Management System Services on July 18th, 2012 5:54 pm

    […] makes a short-request processing […]

  7. Notes on some basic database terminology | DBMS 2 : DataBase Management System Services on August 7th, 2012 6:25 am

    […] that are designed strictly for analytic uses. Examples include Sybase IQ, Vertica, Greenplum, Aster, Infobright, SAND, ParAccel, Exasol, […]

  8. Realising the Benefits of Big Data - TUMRA | Big Data Science on September 30th, 2012 9:54 pm

    […] batch processing truly massive structured and semi-structured data sets. Hadoop’s weakness is short-request processing but tools such as Storm and HBase are starting to address […]

  9. Integrated internet system design | DBMS 2 : DataBase Management System Services on October 11th, 2012 5:57 pm

    […] challenges, with considerable success. But the success is usually one silo at a time — a short-request application here, an analytic database there. When it comes to integration, unsolved problems […]

  10. NewSQL thoughts | DBMS 2 : DataBase Management System Services on January 5th, 2013 1:04 pm

    […] (OnLine Transaction Processing)/short-request-oriented SQL DBMS that are newer than […]

  11. Analytic application themes | DBMS 2 : DataBase Management System Services on April 25th, 2013 4:42 am

    […] analytics at short-request speeds is an obvious data-management-related challenge, and not yet comprehensively […]

  12. Some stuff I’m working on | DBMS 2 : DataBase Management System Services on May 20th, 2013 6:06 am

    […] Numerous vendors are blending SQL and JSON management in their short-request DBMS. It will take some more work for me to have a strong opinion about the merits/demerits of various […]

  13. task management software on July 19th, 2013 11:40 am

    Thanks for the information. I have been working with the analysis and databases. Apply both OLAP and analysis software easy project.

  14. The refactoring of everything | DBMS 2 : DataBase Management System Services on July 23rd, 2013 7:17 am

    […] “real-time analytics”. I expected the short-request/analytic distinction to blur, but even so I’m astonished by the number of NoSQL and NewSQL vendors who’ve […]

  15. web application development services on July 31st, 2013 4:34 am

    and I on the contrary, it is not convenient to use OLAP

  16. What matters in investigative analytics? | DBMS 2 : DataBase Management System Services on October 7th, 2013 4:08 am

    […] availability and security.* This is more crucial for short-request applications than analytic ones, but even your analytic systems shouldn’t leak data or […]

  17. Comments on the 2013 Gartner Magic Quadrant for Operational Database Management Systems | DBMS 2 : DataBase Management System Services on November 8th, 2013 11:46 am

    […] Systems is out. “Operational” seems to be Gartner’s term for what I call short-request, in each case the point being that OLTP (OnLine Transaction Processing) is dubious term when […]

  18. Vertica 7 | DBMS 2 : DataBase Management System Services on January 31st, 2014 9:15 am

    […] sounds like an alternate query execution capability for short-request queries, the big point of which is that it saves them from being broadcast across the whole cluster, hence […]

  19. NoSQL vs. NewSQL vs. traditional RDBMS | DBMS 2 : DataBase Management System Services on March 28th, 2014 10:09 am

    […] it is a new kid — but the general class of questions keeps coming. And that’s just for short-request use cases; similar questions for analytic systems arise even more […]

  20. DBMS2 revisited | DBMS 2 : DataBase Management System Services on May 17th, 2014 12:42 am

    […] on the performance and/or allowed functionality of joins in scale-out short-request RDBMS, and the relative lack of complaints about […]

  21. Using multiple data stores | DBMS 2 : DataBase Management System Services on June 18th, 2014 12:03 pm

    […] Short-request vs. analytic. […]

  22. 21st Century DBMS success and failure | DBMS 2 : DataBase Management System Services on July 14th, 2014 2:37 am

    […] addressed will include analytic RDBMS (including data warehouse appliances), NoSQL/non-SQL short-request DBMS, MySQL, PostgreSQL, NewSQL and […]

  23. Notes and links, December 12, 2014 | DBMS 2 : DataBase Management System Services on December 12th, 2014 6:05 am

    […] cases run the gamut from short-request to highly analytic; no graph DBMS is well-suited for all graph use […]

  24. Notes on indexes and index-like structures | DBMS 2 : DataBase Management System Services on April 18th, 2015 3:54 pm

    […] There are numerous short-request RDBMS indexing strategies, with various advantages and drawbacks. But better indexing, as a general […]

  25. Which analytic technology problems are important to solve for whom? | DBMS 2 : DataBase Management System Services on May 26th, 2015 12:37 am

    […] is where short-request kinds of data stores — NoSQL or otherwise — are often stressed, especially in the […]

  26. Streaming for Hadoop | DBMS 2 : DataBase Management System Services on January 25th, 2016 5:17 am

    […] This all fits with my view that the Current Hot Subject is human real-time data freshness — for analytics, of course, since we’ve always had low latencies in short-request processing. […]

  27. Some checklists for making technical choices | DBMS 2 : DataBase Management System Services on February 15th, 2016 11:27 am

    […] “adding analytics” to something previously focused on short-request processing, it is common to underestimate the variety of things users will soon want to do. (One common reason […]

  28. kathleen on September 7th, 2018 3:37 am

    Great article and analytic process is essential for dbms.

  29. Jason Barkar on April 22nd, 2022 7:43 am

    Great article, really work out for me. I have referred this article to my friend also.

  30. cody wills on May 3rd, 2022 1:13 am

    Database is essential for modern web development and it promotes the processing requests easily

  31. cloudia on May 3rd, 2022 1:15 am

    SQL and other DBMS has significant influence in modern web applications and other web platforms

  32. Sapphire on November 16th, 2022 6:17 pm

    As always, I appreciate your time and thought.

  33. CDN Solutions group on March 31st, 2023 8:53 am

    Nice Article…Very interesting to read this article. I have learned some new information. Thanks for sharing. Visit Now – custom software application development

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.