The HackerNews commentary is, as usual, overflowing with ignorance. This guy claims he prefers carefully tuning InnoDB to prevent Out of Memory (OOM) situations and for its reliability. But in fact a system like LMDB can never cause an OOM situation, even though it requires no tuning. And InnoDB's reliability is so bad that usually the entire DB is unreachable after a power failure. https://news.ycombinator.com/item?id=39324205
So while he's doing extra administrative work (fine tuning to search for safe memory allocation parameters) he's getting orders of magnitude slower performance using #InnoDB than #LMDB, and none of the crash reliability that he claims to value. http://www.lmdb.tech/bench/memcache/
(Note that they claimed to have found 1 flaw in #LMDB that occurred in 0.05% of their test runs, but it was actually a bug in the ext3 fs driver. One that had been fixed in Linux 2+ years before they began their research. In fact LMDB's reliability is flawless.) LMDB is the only DB that is perfectly crash-proof.
@hyc mmap performs awesomely well; for the now >10yo AURORA (used to be sold as IBM Tivoli Netcool Netflow Performance Analyzer & embedded in the big Storwize V7000 "shark" SAN) the time series were objects mmap'ed into mem, thus OS takes care of flushing things to disk when it had mem pressure or found things dirty, and one can use more men than actual mem without the program knowing about the complexity that the kernel deals with perfectly fine as it has the right knowledge.
@jeroen yes. The Brown U paper is correct that things can go horribly wrong if you use it wrong, but that's true of anything. The big problem with the paper is claiming it's impossible to use it right, or that using it right doesn't give simplicity, efficiency, or reliability, which is total BS.
@peterhoneyman frankly the thought never crossed my mind. Probably because of the covid mindset, not participating in any gatherings. But yeah, good suggestion.
@peterhoneyman Also I suspect a conflict of interest; Andy Pavlo's now pushing his Ottertune product, which is supposed to automate tuning optimization for big DBMSs. An mmap-based DB engine like LMDB completely obviates a product like that, since it has no cache/performance tuning parameters.
Add comment