08 April 2007 - 07:48Some Numbers: Fedora Directory Server vs OpenLDAP
The boys in the back room (Howard Chu and Quanah Gibson-Mount) have been off playing with slamd again. There will be a much more detailed look at these runs on the Symas Web site soon, but here's a sniff.
Pictures below, but OpenLDAP 2.3.34 delivered 66% to 72% faster load times and authentication rates from 3.5 to almost 3.8 times that of Fedora Directory Server 1.0.4. So much for the old wives tale that FDS is faster than OpenLDAP. BTW ... this SHOULD represent the performance of Red Hat Directory Server, too (unless they've been mucking about with the code in proprietary source mode). This clearly substantiates Howard's strongly stated belief that OpenLDAP is the fastest directory server on the planet!
And now the pictures. There are three graphs, the two behind the numbers quoted above and a third showing the search rate where the entire database was searched. FDS did better at 250K on this task with OpenLDAP only returning 46% more entries per second. FDS got a lot worse on the larger directories.
All these runs were made on the same server. There were no changes to the hardware or OS configuration between any of the runs. The runs were made between the 4th and the 7th of April, 2007. All the detailed data is shown on the graphs.
This graph shows the load time for each size directory. OpenLDAP pretty consistently takes 30% less elapsed time to load the data. This is good, and convincingly faster but not earth-shattering. I mean, how often do you do a load. Besides, at eight and a half or eleven and a half minutes once in a long while, it's not really a huge deal.
Here's the meat. OpenLDAP is consistently over 3.5 times faster than FDS performing authentications. This is real work. And, if you have a directory authenticating a large population like some of our customers, this might mean the difference between needing a few extra replica servers for performance or not. This was a really happy outcome. We didn't expect the difference to be quite this big.
This was a search that retrieved the DN of every entry in the database. The major drop in performance for each server occurs when the query data exceeds the server's entry cache. For whatever reason the FedoraDS server is much less memory efficient than OpenLDAP, and we could not keep 1 million entries in its entry cache without causing the process size to exceed the system's 8GB of RAM, causing extensive swapping to occur. So while OpenLDAP is answering all queries from its entry cache at 1 million entries, we had to drop FedoraDS's cache down to only 630,000 entries in order to get it to run at all. The remainder of query data was in the BerkeleyDB cache. At 2.5 million entries, both servers are memory starved, and both are set with entry caches of 630,000 entries to keep them from swapping. So at this size, both servers' speeds are dominated by how efficiently they can retrieve data from the underlying BerkeleyDB. But even with severely constrained resources, OpenLDAP makes better use of the available memory than FedoraDS. While FedoraDS suffers a 4.7:1 drop in throughput when it runs out of entry cache, OpenLDAP only shows a 2.4:1 drop. This chart is the clearest evidence of the overwhelming superiority of the design and algorithms in OpenLDAP.
... Marty and Howard
No animals were harmed during these tests. Detailed slamd data is available for review (if you care). Contact us for access.
eleven comments:
What, are you allowed to post benchmarks? 
I have been asked to post this by “the management:
Mr. Howard, your comment raises serious questions about whether the relevant blog entry violates the anti-humiliation provisions of the GPL. Believe me, I take these matters very seriously, almost as seriously as LUNCH! I will get back to you in the fullness of time with an update.
Sincerely,
Jordan H. Heyman, VP Commenter Satisfaction, Dog
Now if only the OpenLDAP developers would implement RFCs 2891 and 2696 (Server-side sorting and paged results, respectively). 2891 been on an official OpenLDAP To-do list for quite a while, but there seems to be little developer interest in actually adding these features. Which is a shame, because there’s no point in storing millions of records anywhere if you can’t sort and page through that data somehow.
When I asked about this on IRC, I was told that it would reduce performance. Of course it would, when actually used. But if there are millions of records in an LDAP store and you need to sort and page through them, it’s much less efficient to retrieve all of that information through a socket, massively increase the resource utilization of your program in the process, just so you can sort the data and then only view a small slice of it.
On the other hand, Fedora DS actually has both RFCs already implemented. That might account for some of the search speed differences, especially if the sorting and paging algorithms are used for all searches.
I love OpenLDAP’s speed. But I’m probably going to stop using it just to get these features.
@Troy Davis:
Uh… OpenLDAP has had RFC2696 support since 2002. What are you talking about?
It’s probably worth noting that the initial code for PagedResults was contributed by an engineer at IBM.
http://www.openldap.org/its/index.cgi/Co..
There are several important lessons here: OpenLDAP is community driven. If you want something, then do something about it. Participate. Developer interest only reflects the priorities of a handful of people. If your priorities are different, you’re welcome to contribute and put your stamp on things. Open Source Software Communities are not servants to individuals or marketing departments. Don’t expect them to jump just because you want something that you’re unwilling to invest effort in yourself. We’d love to have some help of any kind and participants, contributors, have clout.
Server side sorting is kind of necessary to support the Samba 4 effort so it’s somewhere in the mix behind a STABLE 2.4, an Enterprise grade back-ndb, and a few other irons already in the fire. The other lesson is that real Open Source Communities keep developing and don’t give up because the green eye-shade types decide there’s no future in it.
@hyc: It’s good to know that there is 2696 support. I guess I never bothered to try it, it’s not terribly useful to page through unsorted data. And I take your comments about participating vs. whining seriously, but don’t get snippy with me, 2891 has been marked as a “large project” by the developers. That means I’m not likely to get very far trying to implement that on my own. I can affirm this, I downloaded the openldap source last year and tried to scope the work. I wouldn’t just need to understand a significant portion of the existing code base, I’d have to get inside the heads of several of the primary developers to figure out why the code is the way it is. I’d be happy to help, but this is really something a core developer should coordinate for the sake of not wasting the time of new contributors. Our time is important too.
@Troy Davis: Contributing doesn’t necessarily mean contributing code. While at Stanford University, we paid Symas to implement several features we wanted to see in OpenLDAP. This allowed the whole community to benefit.
http://www.wotlkgold.org http://www.lowxx.com http://www.wotlk-powerleveling.com
adfa
[url http://www.adasd.com]adf[/url]]
sadf
Great!wow gold Campus of the University who has clearly inherited the brain remnants of these shares of lonely trend!
I recently came accross your blog and have been re… I recently came accross your blog and have been reading along. I thought I would leave my first comment. I dont know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
I am glad to talk with you and you give me great help! Thanks for that,I am wonderring if I can contact you via email when I meet problems.
No trackbacks: