May 1, 2014

MemSQL update

I stopped by MemSQL last week, and got a range of new or clarified information. For starters:

On the more technical side:

And finally, MemSQL’s column-store compression story — which I mangled in a previous post — goes like this:

Comments

17 Responses to “MemSQL update”

  1. Michele Chambers on May 2nd, 2014 1:37 am

    Don’t forget that MemSQL can spin up on in AWS in 10 mins!

    Companies love the simplicity and flexibility of multiple deployment options on commodity hardware.

  2. John Hugg on May 2nd, 2014 9:00 am

    To clarify the VoltDB misinformation, VoltDB can be run by end users without writing a single stored procedure or ever directly interacting with Java. If used this way VoltDB runs single-statement transactions and does not support external transaction control, exactly like MemSQL. There is literally no “there” there to the Java and stored procedure argument when comparing VoltDB to a system without multi-statement transactions (like MemSQL or most NoSQL).

    If a user does use simple stored procedures written in Java or Groovy, we support complex procedural logic with many SQL statements in a single, ACID-compliant transaction. This is something users can’t do with MemSQL, so it’s weird they’re eager to call attention to it.

  3. John on May 2nd, 2014 11:01 am

    Thanks Curt. Very informative.So why does world need another columnar MPP DBMS?

    It is very interesting to see Memsql pivoting towards Analytics and do a columnar. Sounds me too. Aren’t there already enough (ParAccel, Vertica etc) columnar MPP solutions in the market which are way more mature than memsql?

    WRT the aggregator nodes, doesn’t seem to be intuitive architecture. Let’s take an example. Assume 10 leaf nodes and 4 aggregator nodes. Assuming that first phase aggregate happens on the leaf node and then results are shipped to aggregator nodes and final aggregation takes place. How does this architecture scale in case of high cardinality aggregates or count distincts?

    Looks like Memsql is trying to figure out the market and is confused which way to go (OLTP or Analytics or both).

  4. Jason Block on May 2nd, 2014 12:13 pm
  5. John Hugg on May 2nd, 2014 12:42 pm

    Jason, unless you can also link to term sheets, I’m not sure there’s a judgement to make here.

    In the meantime, I’m happy to make any comparisons based on the value of the products to their users. VoltDB 4.2 is our fastest, lowest-latency-est, most space efficient and most featureful release yet. Of course we do internal comparisons to systems like MemSQL and love what we see in all of those dimensions, but we heartily encourage users to do the same.

  6. Curt Monash on May 2nd, 2014 4:10 pm

    John,

    I think a lot of the action in data management the past few years has been around doing some level of analytics on very fresh data. The associated marketing term is commonly some variant of “real-time”.

    But then, even classical OLTP apps typically have a lot of analytics in them. As I often point out, SAP told me almost a decade ago that over half of their apps’ processing was actually analytic (reporting, planning, whatever). Indeed, there are plenty of kinds of OLTP workflow that use reports as their starting point.

  7. John Hugg on May 2nd, 2014 5:10 pm

    Curt,

    I’m not sure if the point your trying to make is that transactions/procedures are not valuable in analytic workloads, or if you think VoltDB is an OLTP-focused system. I’ll address both.

    VoltDB has been messaging Real-Time Analytics for some time, and has stepped up this messaging for our 4.0 launch significantly. The majority of our users are using VoltDB to glean insight from their live data. We’ve put significant engineering time into SQL-accessible data structures to enable powerful decisioning, such as materialized views, ranking indexes and function-based indexes. For aggregations and leaderboards, VoltDB’s performance is unmatched. We’re rolling out an example in 4.3 next week showing instant responses to queries on pre-aggregated moving time-windows.

    It’s also fair to say that a majority of our users are using VoltDB as an OLTP store, often as the system of record. This is enabled by VoltDB’s tremendous performance, mature high-availabilty and disk-durability features. While some users use VoltDB as a consistent key-value or document store, it seems natural that many of our users are leveraging both OLTP and Real-Time Analytical models at the same time. These use cases typically involve high update volume combined with live reporting / altering on the OLTP store.

    So why are transactions and procedures useful in Real-Time Analytics? Mostly because running a SQL insert statement (or bulk-loader) is just such a waste of an opportunity to make decisions at the point of ingestion.

    We have multiple customers processing sensor networks directly with VoltDB. In one example, an app de-dups and filters RFID sensor triggers acceding to business logic, making a data firehose into a data stream. In another, an app updates a running calculation based on new sensor data using complex math software in the Java portion of the procedure.

    Besides filtering, processing and enriching, stored procedures enable alerting and chained actions beyond what’s possible using SQL triggers. Statistics about a significant portion of all IP traffic in Japan is going though a single VoltDB node who’s only job is to detect DDOS attempts faster and cheaper than other systems.

  8. John Hugg on May 2nd, 2014 5:10 pm

    Ouch. A “your” vs “you’re” in the first sentence. You win Friday. You win.

  9. Curt Monash on May 2nd, 2014 5:22 pm

    Not to mention “statistics … is”. ;)

    As for the rest — it’s been a while since I talked with you guys. Maybe we should fix that. Do you have a competent and civil marketing guy these days, or should we keep it strictly to engineering? :)

  10. John Hugg on May 2nd, 2014 8:40 pm

    Yeah, rushing to get home on a Friday night.

    I’ll talk to the new marketing people and get back to you. ;)

  11. ddorian43 on May 3rd, 2014 2:43 pm

    what does this mean ?

    At customer Shutterstock, 1000s of non-MemSQL nodes are monitored by 4 MemSQL machines.

  12. John on May 5th, 2014 8:01 am

    Thanks Curt. Memsql being downloadable, product can be tested easily :)). With their current version,I would love to see the actual query and rows in each table where they claim they ran 7 to 8 way joins. At the same time, it is a bit strange that without any workload management, Memsql claims to be able to run analytics and OLTP on same cluster. I guess analytics is a being used loosely .

    Out of the 3 vendors mentioned in this post,Clustrix seem to have most comprehensive coverage for SQL and ACID capabilities. Any thoughts on that product?

  13. John on May 5th, 2014 8:11 am

    Curt, one more clarification , post mentions “Dozen of TB across 500 Nodes”. Can you please elaborate on this? What was the node configuration for these and concurrency for analytics? Sounds a bit too much HW for dozen of TB. Reason could be inherent Memsql engine issues requiring this much HW.

  14. Curt Monash on May 5th, 2014 6:41 pm

    ddorian,

    Increasingly, a use for short-request data stores, SQL or NoSQL as the case may be, is to monitor devices. That can get up into the millions of simple devices. In the Shutterstock use case, it seems to be thousands of more complex ones.

  15. Curt Monash on May 5th, 2014 6:42 pm

    John,

    We’re talking about software that was, until recently, in-memory only. I’m not seeing the same arithmetic problem you are.

  16. John on October 18th, 2014 1:33 am

    I don’t know why the voltdb guys to make it a point that they also have traditional jdbc support in lieu of stored procedures, the fact is that you even admit, that using jdbc will result in much worse performance vs. stored java procedures. What makes memsql neat, if true, is that, you can get the high-performance of an in-memory database with ACID, without being limited by stored procedures in your application.

  17. dmitry on November 21st, 2014 1:31 pm

    John Hugg,
    Is VoltDB written in Java?
    Apologies, I can’t find that info on VoltDB’s site.

    Thanks

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.