ShvetsGroup archive

Optimizing JavaScript and CSS-files in Drupal

· All articles

Читать по-русски

Speeding up website loading

What a website begins with?

How does the browser render the page?

Until all JS from HEAD is fully loaded the user sees a "white page". When all the external scripts are loaded the user sees the content of the page and the execution of JS starts in the order of elements on the page (top to bottom).

Thus, to increase the speed of page load it’s necessary to:

Optimizing graphics and creating sprites are the responsibilities of a graphic designer or a markup coder, we'll get down to optimizing JavaScript and CSS-files in Drupal.

Let's see how things are going with these files in Drupal.

Situation analysis

JavaScript

In your project the number and the total size will be different.
You can check the size of your JS-files yourself by running:

find . -name '*.js' -exec ls -l {} \; | awk '{s+=$5} END {print s}'

In our project , except for core files, there are also over 1300 (!) JS-files, which are contained in additional modules and themes.

Total size of all JS-files - 14,746,899 bytes.
In the Drupal 6 core I found the following Javascript-files:

  1. modules/comment/comment.js
  2. modules/profile/profile.js
  3. modules/openid/openid.js
  4. modules/taxonomy/taxonomy.js
  5. modules/system/system.js
  6. modules/block/block.js
  7. modules/color/color.js
  8. modules/user/user.js
  9. misc/autocomplete.js
  10. misc/drupal.js
  11. misc/collapse.js
  12. misc/batch.js
  13. misc/farbtastic/farbtastic.js
  14. misc/form.js
  15. misc/tableselect.js
  16. misc/ahah.js
  17. misc/tabledrag.js
  18. misc/textarea.js
  19. misc/progress.js
  20. misc/tableheader.js
  21. misc/teaser.js
  22. misc/jquery.form.js
  23. misc/jquery.js

Fortunately, they don’t load on one page all at once, some are not even used. The module developer is responsible for configuring the conditions of JS and CSS-file connection on this page so that useless code wouldn’t reduce the page loading speed.

CSS-files

In your project the number and the total size will be different.
You can check the size of your CSS files yourself by running:
find . -name '*.css' -exec ls -l {} \; | awk '{s+=$5} END {print s}'

In our project, except for core files, there are also about 450 files contained in additional modules and themes.
Total size of all CSS files - 1,674,793 bytes.
With regard to CSS-files in the Drupal 6 core, here they are:

  1. modules/locale/locale.css
  2. modules/aggregator/aggregator-rtl.css
  3. modules/aggregator/aggregator.css
  4. modules/update/update.css
  5. modules/update/update-rtl.css
  6. modules/poll/poll.css
  7. modules/poll/poll-rtl.css
  8. modules/comment/comment-rtl.css
  9. modules/comment/comment.css
  10. modules/tracker/tracker.css
  11. modules/forum/forum-rtl.css
  12. modules/forum/forum.css
  13. modules/book/book.css
  14. modules/book/book-rtl.css
  15. modules/profile/profile.css
  16. modules/search/search-rtl.css
  17. modules/search/search.css
  18. modules/openid/openid.css
  19. modules/node/node-rtl.css
  20. modules/node/node.css
  21. modules/taxonomy/taxonomy.css
  22. modules/system/system-menus-rtl.css
  23. modules/system/admin-rtl.css
  24. modules/system/admin.css
  25. modules/system/maintenance.css
  26. modules/system/defaults-rtl.css
  27. modules/system/defaults.css
  28. modules/system/system-rtl.css
  29. modules/system/system-menus.css
  30. modules/system/system.css
  31. modules/block/block.css
  32. modules/color/color.css
  33. modules/color/color-rtl.css
  34. modules/help/help.css
  35. modules/help/help-rtl.css
  36. modules/dblog/dblog.css
  37. modules/dblog/dblog-rtl.css
  38. modules/user/user.css
  39. modules/user/user-rtl.css
  40. misc/print-rtl.css
  41. misc/farbtastic/farbtastic.css
  42. misc/print.css
  43. themes/bluemarine/style.css
  44. themes/bluemarine/style-rtl.css
  45. themes/garland/print.css
  46. themes/garland/style.css
  47. themes/garland/minnelli/minnelli.css
  48. themes/garland/color/preview.css
  49. themes/garland/style-rtl.css
  50. themes/garland/fix-ie.css
  51. themes/garland/fix-ie-rtl.css
  52. themes/pushbutton/style.css
  53. themes/pushbutton/style-rtl.css
  54. themes/chameleon/common-rtl.css
  55. themes/chameleon/style.css
  56. themes/chameleon/marvin/style.css
  57. themes/chameleon/marvin/style-rtl.css
  58. themes/chameleon/common.css
  59. themes/chameleon/style-rtl.css

The total size of CSS files is a lot smaller than the size of JS files, but we have to keep in mind the fact that there are a lot more CSS files on the page than JS files (about 2 times more). In addition, the styles tend to load for all pages (these are theme styles), and only the module styles can load for certain pages. Therefore both the style files and the scripts require our equal attention.

Limitations of Internet Explorer

IE 6-8 has a limitation on the number and size of CSS files :

Limitations of HTTP protocol

We wonder whether there is a limit to the number of AJAX connections in a browsers

According to the HTTP 1.1 specification the browser must establish not more than 2 simultaneous connections (and it works for IE6/7) to one host. In Firefox and Opera, this parameter is configurable and equals to no less than 4 by default. According to some reports the IE8 has 6 connections to one host.

Source: Raising network.http.max-persistent-connections-per-server?

Results of the Analysis

Let’s sort out the glossary.

Optimization types

Script minimization is the removal of all non-essential characters from the code in order to reduce the size of the script file and its loading speed. In the minimized code all the comments and insignificant spaces, line breaks, tab characters are deleted. In case of JavaScript, this reduces the page load time, since the file size decreases. Two of the most popular tools for minimizing JavaScript include JSMin è YUI Compressor.

obfuscated_javascript_code_0.png

Obfuscation is an alternative way of reducing the source code. As well as the minimization, it removes the spaces and the comments, and additionally changes the code itself. For example, during the obfuscation function names and variables are replaced by shorter names, which makes the code more compact but less readable. Typically, this technique can be used to complicate the program’s reverse engineering. Obfuscation also helps to reduce the code much more than by simple minimization. It’s not that easy to find the best JavaScript obfuscation software, but I think that the most common tool is the Dojo Compressor (ShrinkSafe).

Minimization of JavaScript is a safe and relatively simple process. On the other hand, obfuscation may produce errors in the code because of its complexity. Obfuscation also requires changes to your code by highlighting API-functions and other elements that should not be changed.

Aggregation is the grouping of multiple files on a page into one. Aggregation is as safe as minimization, because the code is not changed. Sometimes "gluing" files may fail due to improper comments at the end of the files. Aggregation reduces the number of files connected to the page. Starting with Drupal 6 aggregation of CSS and JS files is built into the core, previous versions require the installation of an additional module.

HTTP-compression is a way to compress the entire content (mainly text), which is rendered from the Web servers to browsers via the Internet. The main advantage of compression is that it reduces the number of bytes transferred and thus increases performance. HTTP-compression uses publicly available compression algorithms to encrypt HTML, XML, JavaScript, CSS and other file formats on the server side. This method of compressed content delivery is based on the standards and is included in HTTP/1.1 and all modern browsers support the HTTP/1.1 protocol. This way they can decompress compressed files automatically on the client side. This means that no additional software or user interaction on the client side is necessary.

Server request without compression

HTTP_request.png

Server request and server response with compressed content

HTTP_request_compressed.png

For cascading style sheets only minimization, aggregation and HTTP-compression can be applied. Obfuscation is not used for CSS files. JavaScript files can be optimized in any of the described ways.

Overview of the existing Javascript-compressors.

The degree of compression of a file can be compared by using the JavaScript Compressor and Comparison Utility online service.

The degree of compression is different for all compressors, because the compression methods are different. Compression of small files, usually gives a very small degree of compression, files of more than 10Kbytes are compressed very well. The chart below shows the relation between the degree of compression and the file size:

js_compression_levels_chart.png

Solutions to the Problem

So, we have a fairly large number of files that can be compressed and we need to sort out what Drupal has to offer (in the core and additional modules) and find the best solution.

Let’s start with an analysis of “out of the box” solutions we already have.

Standard Drupal Features

JavaScript and CSS files aggregation, and HTTP-compression are implemented in the core of Drupal 6.
On the "Performance" page ("Performance" - admin admin/settings/performance) there is an option to enable/disable the optimization of CSS and JavaScript files:

If CSS and JS file optimization is disabled - this means that the aggregated file cache can not be written to files. Check whether the path to the folder files and write permissions for this folder on the File System page (File System - adminadmin/settings/file-system) are correct.

CSS and JS file optimization on Performance configuration page.

How does the optimization of JS and CSS-files work in the Drupal core?

To start using this opportunity you need to enable JavaScript or CSS-file optimization on Performance configuration page. Then during the theming of the page from the function template_preprocess_page() the drupal_get_css() function will be called. This function does most of the job for CSS-files.

How does the drupal_get_css() function work?

What does the optimization comprise of?

Thus, the optimization offered by Drupal core is actually minimization with aggregation into a single file.

Disadvantages of optimization techniques in Drupal core

The optimization technique used in Drupal core is safe, meaning it does not lead to errors in the code. But this technique is not as effective as it seems.

As a matter of fact, a page may contain about twenty different scripts, which are collected in a unique file to be cached. Scripts on the page can be loaded, depending on access rights or any other conditions that increases the number of options for a single page.

General scripts in aggregated files
Thus we create a large file that aggregates scripts and the browser caches it for this page. On the next page a few scripts can be added or removed and a unique file, aggregating scripts for this page will be created. Browser will cache it too, but these 2 large files contain a lot of common code, that is rendered twice and cached by the browser separately in files with different names. Thus, the effective caching (if enabled), this technique may produce even more traffic.

JavaScript optimization in the core of Drupal

JavaScript optimization is similar to optimizing the CSS-files.

This way of gluing can produce errors in the aggregated file, when the last string in the end of the script was a comment.

When optimizing the JavaScript-files, they are merged into one large file in a little different way - ";\n" is added after each file.
The JS-file code itself does not change - it can be seen in the code of function drupal_build_js_cache(), but is simply merged into one big file.

That is Drupal core does not even minimize JavaScript!

Using HTTP-Compression

On the Performance configuration page HTTP-compression can be enabled and disabled:
HTTP-traffic Gzip compression on the Performance configuration page.

Function page_set_cache() saves compressed page content in the cache, if:

Drupal caches the page content, and in case if the above conditions are met, compresses and then caches the page content..

If the browser supports HTTP-compression, it specifies what types of compression it can work with in a page request:

Accept-Encoding: gzip,deflate

A simple online web page compression / deflate / gzip test tool is a service that checks whether the server returns a file using HTTP-compression and the degree of compression.

If the server uses HTTP-compression, it will compress and give the browser a compressed version of the content.
However, with the help of zlib library Drupal is able to compress and cache an already compressed version of HTML-pages, but not JS and CSS-files.
This means that if your web server is configured to compress the content and the browser supports such compression, the server will compress JS and CSS-files straight away. But it would be much more effective to store compressed copies of these files with the highest possible degree of compression, and not compress them straight away (in which case the default is not the maximum degree of compression).

Additional modules

JavaScript Aggregator Module

The module minimizes JS-files using JSMin.
The module can compress aggregated and minimized JS-files using the gzip-compression. Compressed copy is stored in a file on the server and is given to the browser instead of compressing the file by the server.

Example of compression:

FILENAME                                       SIZE              DESCRIPTION
6e13ccb6262e06b9f890414db56d3b1f.js            289.558 Bytes  >  Aggregated by Drupal Core
6e13ccb6262e06b9f890414db56d3b1f.jsmin.js      173.243 Bytes  >  Minified with JSMin
6e13ccb6262e06b9f890414db56d3b1f.jsmin.js.gz    47.618 Bytes  >  Minified and gzipped

CSS Gzip module

css_gzip_0.png

Parallel module

Parallel1.png

IE CSS Optimizer module

ie-css-optimizer-screenshot.png

IE Unlimited CSS Loader module

"don’t use @import" - an article on why using @import is not desirable in terms of performance.

The module does about the same thing as IE CSS Optimizer module, but a little bit differently – it uses @import to solve the 31 CSS-file limit of IE.

CSSTidy module

What is optimized:

CSS-hacks that work:

Testing page loading speed

We measured the page loading time to figure out how the average download time changes depending on different methods of JS-file optimization. The test site has over 1300 JS-files, almost 450 CSS-files and 227 installed modules (including core modules). We studied different methods of optimization:

Test conditions

Browser Settings

Server Settings

Drupal Settings

Test Results

Average page load time

Results of testing the speed of various optimization modes

Chart of page load times in different modes

YSlow_tests_diagram.png

Test results analysis

Therfore we experimentally found that the optimization of JS-files by the Drupal core doesn’t always show good results. In our case (without the use of JavaScript Aggregator module) we received a slight increase in loading speed (by reducing the number of requests) and an increase (compared with the original files) in the size of aggregated (or compressed) file.

Given that many pages load the same JS and CSS files, with only a few specific files added or removed, caching a large file won’t give a performance boost. In this case, the caching of separate CSS files and scripts on the same page will save time on loading these same files on other pages.
A user typically does not load the same page several times, and goes on surfing, thus browser caching of small script and CSS files can be more efficient than aggregation into a single file. But this is only for the user, it’s better for the server to have as few requests as possible and give aggregated files instead of piles of small files.

What can be done to improve the situation?

Moving JavaScript to footer

At the very beginning of this article, we found that a browser loads all CSS and JS from header and only then displays the page to the user and starts executing JS. That's why it makes sense to move the JS-files to the footer, to make the page content appear faster in the browser. This will reduce the time during which the user sees a "white page" and pages will visually load faster (the actual loading speed will remain the same).

JS-files in Drupal are connected as follows:
drupal_add_js($data = NULL, $type = 'module', $scope = 'header', $defer = FALSE, $cache = TRUE, $preprocess = TRUE)
here
$scope (optional) The location in which you want to place the script. Possible values are 'header' and 'footer' by default. If your theme implements different locations, however, you can also use these.

By default $scope is equal to "header" and, unless otherwise stated, all scripts are loaded in the header...
Thus the need to change $scope = 'header' to $scope = 'footer' in all the modules.

Just moving <?php print $scripts ?> to the footer won’t bring any result, because inline-scripts that will remain in the header will expect jquery.js and other scripts in the header and will return errors during execution.

For the 7th version of Drupal, there has been a discussion about the possibility to make 'footer' the default for $scope (Make 'footer' the default scope for drupal_add_js() so that pages render fast), but most likely this option will be implemented in the 8th version of Drupal.

Links

Script compresors

HTTP-compression

Order of loading page elements by the browser

Transferring Javascript to footer

Miscellaneous