Speculation about the JPMorgan Chase authentication database outage
Edit: Subsequent to making this post, I obtained more detail about the JPMorgan Chase database outage.
I was just contacted for comment about the Chase database outage, about which they’ve released remarkably little information (they’ve even apologized for their terseness). About all Chase has said is:
A third-party database company’s software caused a corruption of systems information, disabling our ability to process customer log-ins to chase.com. This resulted in a long recovery process,
and even that quote is a bit hard to find. From other reporting, we know that ATM machines, bank branches, and the call centers continued to work, but various web and mobile access applications were disabled.
Of course, that quote is pretty ambiguous. My thoughts on it include:
- Presumably, the database Chase uses to authenticate log-ins was screwed up.
- That’s consistent with what the quote says.
- It’s also consistent with the stories as to what did or didn’t work. After all, ATM authentication — validating ATM cards and PINs at known endpoints — is very likely to be run differently than, say, web authentication using conventional passwords.
- Note that authentication is commonly run off of purpose-built LDAP (Lightweight Directory Access Protocol) database systems rather than relational DBMS.
- We’re Chase customers (for credit cards). Their authentication is pretty annoying. So are other aspects of their technology. (And so are their business practices, but that’s a different story.)
- Surely you’ve heard the term “security theater.” Well, “authentication theater” happens too. More on that in a future post.
- The Chase quote is ambiguous as to whether this was failure of purchased software Chase runs in-house, or whether it was a failure of an outsourced system. The former theory seems more likely.
- I guess it’s possible that there’s some sort of third-party (outsourced or inhouse) identity validation/data cleaning going on, which feeds the authentication database, and that this is what crashed rather than the main system. But again, the Chase quote is — well, I guess it’s both ambiguous AND vague.
Comments
11 Responses to “Speculation about the JPMorgan Chase authentication database outage”
Leave a Reply
One aspect of their explanation puzzles me: Why would a busted authentication system affect online bill payments?
According to Chase’s “apology” page, Online Bill Payments scheduled for Monday or Tuesday didn’t get sent until Wednesday.
I’m no database expert, so I ask you: Does it seem reasonable that some software tool related to authentication would also disable previously scheduled transactions?
I’ve gotten the explanation. Will blog it soon.
Chase online banking is horribly unreliable. This is not the first time that Chase has suffered a lengthy outage of its online banking system. For example, Chase online banking was down for 15 hours on August 7 through August 8. This article on the USA Today web site mentions that August outage:
JPMorgan Chase’s Online Banking Outage Sparks Questions:
http://content.usatoday.com/communities/technologylive/post/2010/09/investigators-seek-specific-trigger-to-jpmorgan-chases-online-banking-outage/1
If that URL does not work, Google the article’s headline:
“JPMorgan Chase’s Online Banking Outage Sparks Questions”
[…] posting my speculation about the JP Morgan Chase database outage, I was contacted by – well, by somebody who wants to be referred to as “a credible source close […]
Any chance that this was related to the Zeus bot that came to the surface last week. Today Zone Alarm announced a new zeus.zbot.aoaq trogan that specifically keylogs banking usernames and passwords. ESET reported last month on the theft of over 1 million pounds from 100,000 banking customers in England.
http://www.eset.com/resources/podcasts/081110_ESET_Zeus.mp3
Chase has 16.9 million online users so if it is an attack of that nature it would be the largest bank heist ever. Creepy.
There debit card system was down in Canada.
Chase isn’t very popular in Canada but for some reason Subway had one and it failed during authentication today around noon.
Cost me $2 in service fees for having to run to the shitty ATM next door.
A week later, in a different country, in a very different application, that’s probably an unrelated outage.
[…] me: Because my previous speculative post about the JP Morgan Chase outages had shown up in the search engines and looked pretty […]
[…] 8:32 am (data, design, engineering, musings) The blogosphere is abuzz about JPMC outage (1, 2, 3). The basic reason people cite for long recovery time is a big, ambitious database design – […]
[…] JP Morgan Chase 的 Oracle database 掛掉 導致線上服務停擺近兩天。銀行系統專用資料庫還有更好的選擇嗎? […]
[…] to do before they can confidently isolate the specific trigger. Monash Research principal Curt Monash calls the bank’s explanation “ambiguous and […]