March 30, 2011

Short-request and analytic processing

A few years ago, I suggested that database workloads could be divided into two kinds — transactional and analytic. The advent of non-transactional NoSQL has suggested that we need a replacement term for “transactional” or “OLTP”, but finding one has been a bit difficult. Numerous tries, including high-volume simple processing, online request processing, internet request processing, network request processing, short request processing, and rapid request processing have turned out to be imperfect, as per discussion at each of those links. But then, no category name is ever perfect anyway. I’ve finally settled on short request processing, largely because I think it does a good job of preserving the analytic-vs-bang-bang-not-analytic workload distinction.

The easy part of the distinction goes roughly like this:

Where the terminology gets more difficult is in a few areas of what one might call real-time or near-real-time analytics. My first takes are: 

Indeed, one of my top trends to watch these days is the integration of short request and analytic processing. Several different approaches come to mind.

You can do everything in a single instance of a general-purpose DBMS such as Oracle, DB2, Microsoft SQL Server, Sybase ASE, or MySQL. For sufficiently small enterprises with sufficiently undemanding workloads, that’s the best approach. Tokutek evidently aspires to be an improved version of the same thing.

You can do it in an analytic DBMS that is sufficiently strong in user concurrency, update speed, and so on. This is the sweet spot of Teradata’s market. It’s also where SAP HANA is alleged to be going.

You can tie together DBMS optimized for short-request and analytic processing (or use something like Hadoop for the analytics, whether or not it should be considered as a DBMS). E.g., Membase (now Couchbase) has integration stories with Hadoop and Vertica, at a couple of clients each. I think this is a major untapped opportunity in the MySQL world, and have been raising that point with various companies for some time.

You can graft short request processing onto analytic system. That’s the point of HBase.

You can superpose analytics on a short request processing system. That’s the point of DataStax Brisk.

Sybase RAP, depending on how it is configured, can fit several of these models. The same could be said of Oracle (especially Exadata) or DB2.

Fun times.


32 Responses to “Short-request and analytic processing”

  1. Starcounter | DBMS 2 : DataBase Management System Services on April 13th, 2011 12:55 pm

    […] seems to be offering an in-memory object-based/object-oriented/whatever short-request DBMS that also talks SQL. I haven’t been briefed at this point, and hence don’t have detail […]

  2. Notes on short-request scale-out MySQL | DBMS 2 : DataBase Management System Services on April 19th, 2011 4:52 am

    […] that sounds as if he was asking about scale-out relational DBMS in general, MySQL or otherwise, short-request or analytic, it turned out that he was asking just about short-request scale-out MySQL. My thoughts and […]

  3. The Ted Codd guarantee | DBMS 2 : DataBase Management System Services on July 31st, 2011 5:44 pm

    […] never quite worked out that way. For most of the history of tabular DBMS, the best-performing short-request and analytic DBMS have been designed quite differently from each other.* Non-relational systems — from […]

  4. “Enterprise-ready Hadoop” | DBMS 2 : DataBase Management System Services on June 19th, 2012 6:14 pm

    […] across-the-board competitive with analytic relational DBMS. (And the same goes for HBase vs. short-request alternatives.) But the real question is whether its features are good enough for a variety of […]

  5. Notes on HBase 0.92 | DBMS 2 : DataBase Management System Services on June 19th, 2012 6:15 pm

    […] you do short-request and MapReduce processing against the same HBase database, the MapReduce part is usually still done […]

  6. Clustrix 4.0 and other Clustrix stuff | DBMS 2 : DataBase Management System Services on July 18th, 2012 5:54 pm

    […] makes a short-request processing […]

  7. Notes on some basic database terminology | DBMS 2 : DataBase Management System Services on August 7th, 2012 6:25 am

    […] that are designed strictly for analytic uses. Examples include Sybase IQ, Vertica, Greenplum, Aster, Infobright, SAND, ParAccel, Exasol, […]

  8. Realising the Benefits of Big Data - TUMRA | Big Data Science on September 30th, 2012 9:54 pm

    […] batch processing truly massive structured and semi-structured data sets. Hadoop’s weakness is short-request processing but tools such as Storm and HBase are starting to address […]

  9. Integrated internet system design | DBMS 2 : DataBase Management System Services on October 11th, 2012 5:57 pm

    […] challenges, with considerable success. But the success is usually one silo at a time — a short-request application here, an analytic database there. When it comes to integration, unsolved problems […]

  10. NewSQL thoughts | DBMS 2 : DataBase Management System Services on January 5th, 2013 1:04 pm

    […] (OnLine Transaction Processing)/short-request-oriented SQL DBMS that are newer than […]

  11. Analytic application themes | DBMS 2 : DataBase Management System Services on April 25th, 2013 4:42 am

    […] analytics at short-request speeds is an obvious data-management-related challenge, and not yet comprehensively […]

  12. Some stuff I’m working on | DBMS 2 : DataBase Management System Services on May 20th, 2013 6:06 am

    […] Numerous vendors are blending SQL and JSON management in their short-request DBMS. It will take some more work for me to have a strong opinion about the merits/demerits of various […]

  13. task management software on July 19th, 2013 11:40 am

    Thanks for the information. I have been working with the analysis and databases. Apply both OLAP and analysis software easy project.

  14. The refactoring of everything | DBMS 2 : DataBase Management System Services on July 23rd, 2013 7:17 am

    […] “real-time analytics”. I expected the short-request/analytic distinction to blur, but even so I’m astonished by the number of NoSQL and NewSQL vendors who’ve […]

  15. web application development services on July 31st, 2013 4:34 am

    and I on the contrary, it is not convenient to use OLAP

  16. What matters in investigative analytics? | DBMS 2 : DataBase Management System Services on October 7th, 2013 4:08 am

    […] availability and security.* This is more crucial for short-request applications than analytic ones, but even your analytic systems shouldn’t leak data or […]

  17. Comments on the 2013 Gartner Magic Quadrant for Operational Database Management Systems | DBMS 2 : DataBase Management System Services on November 8th, 2013 11:46 am

    […] Systems is out. “Operational” seems to be Gartner’s term for what I call short-request, in each case the point being that OLTP (OnLine Transaction Processing) is dubious term when […]

  18. Vertica 7 | DBMS 2 : DataBase Management System Services on January 31st, 2014 9:15 am

    […] sounds like an alternate query execution capability for short-request queries, the big point of which is that it saves them from being broadcast across the whole cluster, hence […]

  19. NoSQL vs. NewSQL vs. traditional RDBMS | DBMS 2 : DataBase Management System Services on March 28th, 2014 10:09 am

    […] it is a new kid — but the general class of questions keeps coming. And that’s just for short-request use cases; similar questions for analytic systems arise even more […]

  20. DBMS2 revisited | DBMS 2 : DataBase Management System Services on May 17th, 2014 12:42 am

    […] on the performance and/or allowed functionality of joins in scale-out short-request RDBMS, and the relative lack of complaints about […]

  21. Using multiple data stores | DBMS 2 : DataBase Management System Services on June 18th, 2014 12:03 pm

    […] Short-request vs. analytic. […]

  22. 21st Century DBMS success and failure | DBMS 2 : DataBase Management System Services on July 14th, 2014 2:37 am

    […] addressed will include analytic RDBMS (including data warehouse appliances), NoSQL/non-SQL short-request DBMS, MySQL, PostgreSQL, NewSQL and […]

  23. Notes and links, December 12, 2014 | DBMS 2 : DataBase Management System Services on December 12th, 2014 6:05 am

    […] cases run the gamut from short-request to highly analytic; no graph DBMS is well-suited for all graph use […]

  24. Notes on indexes and index-like structures | DBMS 2 : DataBase Management System Services on April 18th, 2015 3:54 pm

    […] There are numerous short-request RDBMS indexing strategies, with various advantages and drawbacks. But better indexing, as a general […]

  25. Which analytic technology problems are important to solve for whom? | DBMS 2 : DataBase Management System Services on May 26th, 2015 12:37 am

    […] is where short-request kinds of data stores — NoSQL or otherwise — are often stressed, especially in the […]

  26. Streaming for Hadoop | DBMS 2 : DataBase Management System Services on January 25th, 2016 5:17 am

    […] This all fits with my view that the Current Hot Subject is human real-time data freshness — for analytics, of course, since we’ve always had low latencies in short-request processing. […]

  27. Some checklists for making technical choices | DBMS 2 : DataBase Management System Services on February 15th, 2016 11:27 am

    […] “adding analytics” to something previously focused on short-request processing, it is common to underestimate the variety of things users will soon want to do. (One common reason […]

  28. kathleen on September 7th, 2018 3:37 am

    Great article and analytic process is essential for dbms.

  29. riiupw afrass on December 11th, 2020 3:43 pm

    Charts, reports, and analytics – all this is somehow present in the back-office of any, even a very small, enterprise. When ordinary spreadsheets in Excel / Numbers / Libre are getting cramped, but data is still not very big, traditional solutions for internal company needs are often built using relational databases and using software design methodology Why not use services to help marketers visualize data? Any ideas?

  30. Bruno Araujo on March 18th, 2021 2:07 pm

    Do you need to analyze more data? You can simply use a third-party software for this purpose. Here are some examples. It is faster than build your own scripts.

  31. Sophia on November 25th, 2021 1:59 am

    Web design and development Since 2014, India has been an Indian-based software outsourcing company with offices in the United States. Web/mobile app development, e-Commerce development, and custom software creation are among their specialties. Aside from that, they have a broad range of services, including AR & VR solutions, eLearning & online training, Omnichannel solutions, MVP development, IoT solutions, SaaS development services, Blockchain development, and so on.

  32. Nicholas Winston on December 27th, 2021 1:47 am

    Excellent tutorial on DBMS. In this post, you have really cleared all the doubts. Thanks you so much

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.