by Oleg Prosyanik
Today we’ll be testing several servers and software configurations to obtain summary test reports and compare the results. Let’s say for example we plan to open a new website with regular, ongoing streaming of data, images and video clips, for instance, a news site.
In the process of planning the project, we decided to try two blog configurations in parallel.
WordPress vs. Drupal
Both have advantages and disadvantages. Making the right decision without any pre-testing or reference for possible results is difficult and not recommended. Running these tests are important as we will then be able to better assess between the two configurations and select our best option.
Before you start testing or getting results, let’s set up our two configurations, add some test content to the websites and finally, assess possible actions of potential users. Then, we need to design the test scripts and proceed to test the configurations.
Step 1. Test Server Configuration.
Because we need obtain as many results as possible, let’s configure our servers as real-world scenarios. First, configure the hardware, install and prep all necessary server software. Next, we prepare the demo sites for testing.
To get the load testing results, we connected to our New Relic account - our partners for application performance monitoring (APM) - to the server applications to monitor performance during the load test.
*Note- To use New Relic’s APM service, a downloaded component must be installed on the servers and be linked to your New Relic account.
The next necessary action is the configuration of both websites and the uploading of the test materials to simulate the service.
We already configured both our Drupal and WordPress blogs and installed all necessary modules, optimized the sites and added in the content; a few posts, some static content pages and basic forms.
So we are starting with two nearly identical sites in terms of content and functionality, albeit located on two separate servers with the same hardware configuration. When we run our aggregated test , we’ll get a better idea of the which system configuration is better for our purposes.
Step 2. The Creation of Load Scripts.
When we configure our blogs, we can begin creation of our JMeter test scripts for each configuration which will then be combined into a single test.
The test plan consists of basic user actions reading through a blog post, viewing other site pages etc.
Step 3. Creation of Aggregated Tests.
Now that we have configured the demo sites and test scripts we can get down to it.
Comparing our Drupal blog with our WordPress blog and creating an aggregation test.
An aggregation test is a test that contains master and slave tests. Master load tests run and controls related slave tests. Running tests in this way allows us to easily and effectively compare results about efficiency of the CMS for Drupal and WordPress. In our case, the Aggregation test results contain information from several test sessions.
To create the tests, login to your BlazeMeter account, then click “Add Test”.
Set the test name, traffic origin location and JMeter version. Then, upload the script files and configure virtual user activity as shown above.
To obtain additional information we can now connect your New Relic account and select the data you wish to monitor in our reports.
To do this, expand the New Relic settings group, connect to your existing New Relic account, and select the required application and metrics to obtain. For example, we want get data pertaining to errors per minute which occurs during test run.
Set the advanced properties of the each load test. In this section we must set, which load test will be the master and which the slave.
We set the Drupal blog as the slave test. To do this we select the corresponding radio button under the ‘Multi Test Type’ options block. This test will now be controlled by the master load test.
During the creation of the second test, of our WordPress blog, again under the advanced test properties and select “Multi Test Type: Master”. When setting up a test as the master test, we need to choose the slave tests that were pre-configured as slaves. Under the “Slaves” checkbox dropdown, we selected the Drupal load test created before.
Once set up and saved, our test set up will now be displayed on the main page under the ‘Load Tests’ table.
Step 4. Run Aggregation Load Test
To start the aggregation test open the WordPress load test, from the Load Tests table and use Start button at the top of the test description page.
While our test runs, we can see interim results. To do this after the test started, just use the top menu link “Load Report” at the load test page. Loading the report of the master test allows us to see results not only for the current test but for all running slave tests.
Now, we start our aggregate test and can see interim results for both the WordPress and Drupal blogs. For clarity we named elements of the Drupal load test as “Drupal_*”.
When we open load report for the master test, aside from the standard tabs Load Results, Monitoring and Logs we can also use the Master Load Results and Master Monitoring tabs which show us all the compiled data about the master and slave test runs.
When we added additional tests, we connected our New Relic account and selected which information from New Relic we wanted to view on BlazeMeter’s dashboard.
In the test configuration we opted to view errors which occurred on our server. But, because there are no errors we just see a straight line passing through –“zero errors/minute”. That’s a good result.
To view results for the slave tests separately, we just open the required test from the main tests page, Load Tests Table. The load report for the slave test will be shown automatically.
When the tests finish, we can open the final aggregated load test report and view the results. To open it, go to main test page “Tests &Reports and open the test report from the Reports list. The report will be named for the load test which was configured.
Step 5. Analyze Results
Now we can analyze the report and decide the winning CMS, Drupal or WordPress, will be better for our purposes. So, we need open the test report to view the results and analyze the graph.
Check out the response time under the results tab/Response Time Graph. The average duration of response for Drupal blog is nearly 14 seconds. For the WordPress blog this number is only 0.52 – 0.53 seconds.
Website response time running by Drupal CMS
Website response time running by WordPress CMS
Because the response time is one of the most important parameters of the site, we can easily and obviously conclude that the WordPress website works much more quickly than Drupal site.
Improvement to the quality of the Drupal site is certainly possible with a number of additional optimizations. But these actions involve additional resources of which we are currently low on. So, WordPress it is, at least for these purposes.
Now we have made a thorough assessment of our simulated sites. These are imperative steps in the development of services.
WordPress vs. Druapl= WordPress is the Winner!
DOWNLOAD BLAZEMETER'S PLUGIN TO WORDPRESS
Join our Sessions (or get the recap)
Learn all about our new WordPress Load Testing plugin with a recap of our live demo and QA session.
Load Testing for Mobile (made easy), the recap: Learn all about our new mobile testing features and how to easily and realistically test mobile apps and websites.
Comments
ophir.prusak replied on Permalink
Does Drupal have any Caching plugins?
technicalknockout replied on Permalink
Yes, drupal core has some default cache systems which you can customize with different cache modules. Drupal core stores its cached data in mysql by default, but can be setup to use memcached, redis, varnish, or anything else really.
Mike replied on Permalink
There are a myriad of ways to cache Drupal. Drupal has a built-in caching mechanism that consists of many levels, but the real performance improvement comes with server-level configuration. This demo does not take that into account. With properly configured server hosting and server environments Drupal can attain and beat the WordPress numbers. The answer to the question posed here is highly, highly dependent on use case and environment. This is impossible to predict without testing. New Relic is great for this (I am not affiliated).
Lucky replied on Permalink
The Boost module (https://drupal.org/project/boost) is an easy way to get static page caching for anonymous visitors. The Memcache module (https://drupal.org/project/memcache) is often used to improve performance for logged in users (requires memcached of course, which means a VPS/dedicated server). The cache backend is pluggable in Drupal, so you could also use APC, etc.
dropchew replied on Permalink
Yes drupal has caching extension plugins (modules) and built in caching. I notice there's a spike at the beginning of the wordpress results, isit cached after that?
yonatan replied on Permalink
First of all, you can set it to cache the created page to the DB, so only a fraction of the normal page processing is done.
secondly - it has a memcached plugin - so you don't even need that roundtrip to the DB...
Lex van Sonderen replied on Permalink
Yes, Drupal is 'too flexible' and that results in poor out-of-the-box performance. Therefore Drupal benefits from caching and knowledgeable server setup. It has various caching solutions, depending on the need (such as anonymous visitors / personalization / faceted browsing). Choose a hoster that really knows Drupal and it will run blazingly fast.
technicalknockout replied on Permalink
Wordpress is simply a lighter CMS with less subsystems & usually less database queries for a vanilla blog. I'm not surprised it wins out in your tests. But, these benchmarks don't compare to when you put varnish in front and can serve pages without hitting the LAMP stack. If you can setup and configure these load tests - setting up varnish should be a piece of cake for you :)
Ben Knowles replied on Permalink
Something is very wrong here - there's no way Drupal would take 14 seconds to load, even when logged in as admin without any caching. You must have a problem with your setup. For any test to be fair you should at least detail the configuration in a basic level, i.e. what caching and modules you are using.
I use both Drupal and WordPress. Drupal comes with some good built-in caching when configured correctly, and also does basic CSS and JS aggregation. WordPress does neither out of the box. Both CMS have good caching plugins.
In most scenarios a Drupal or WordPress site should be easy to get loading in 1 to 2 seconds as long as caching is configured. For logged in users things get a lot harder as it's not possible to serve full cached HTML pages.
Lucky replied on Permalink
Drupal has caching disabled OOTB, as you wouldn't want it enabled when building the site. So what you've tested here is the load performance of a dev site, not a live/production one. If only caching had been enabled (Configure › Development › Performance), the results wouldn't look so horrible.
But I suppose it still wouldn't be able to challenge Wordpress's performance. Drupal is a much larger system OOTB, and this really is a case of comparing apples and oranges.
Ayesh replied on Permalink
Drupal, as per my own experiments and what you have here, is slow compared to Wordpress. But wordpress is a straight blog engine and it's written for blogs. Drupal, in other hand, could be a forum site, a simple blog, a massive corporate site, a recipe site or whatever.
Forms can be altered at module level, at theme level and there are a lot of preprocess functions that need to be run in order to use the max flexibility (of Drupal architecture) and to reuse the code.
In my opinion, when I need a (simple) blog, I don't bother configuring Drupal heavily for that. Wordpress is there. But using wordpress for something that is too different from its scope is worse that using a slow but flexible and smart Drupal site.
kscheirer replied on Permalink
kinda useless metrics unless you actually post what the configurations you used were. Both Drupal and WordPress are built to run on whatever server you have, out of the box. But that's a far cry from even moderately tuning them.
Also, you misspelled Drupal in "WordPress vs. Druapl" :)
And to reply to the previous commenter, yes, Drupal has many caching plugins. I'm sure WordPress does too.
aaron replied on Permalink
Did you even enable the core drupal cache? That's one of the most basic setup items.
Mads replied on Permalink
_14_ seconds average response time? To me that sounds a bit unlikely - that would pretty much render Drupal useless for serving websites. I don't have a lot of experience with Drupla, but I know it's a popular CMS, so a out-of-the-box response time like that can't be right.
Looking at the graphs in the conclusion, puzzles me somewhat - I can see the ~14 seconds response time for the Drupal site, but the Wordpress site seems to average just under 1.4 seconds, not 0.5 seconds?
Javid replied on Permalink
I am glad that I am using Wordpress.
Add new comment