From 3cd5b4bc0621a40b73af4ece196df854790eece2 Mon Sep 17 00:00:00 2001 From: Yannick Warnier Date: Wed, 4 Feb 2015 09:16:13 -0500 Subject: [PATCH] Add DB compression flag configuration - refs #7510 --- documentation/optimization.html | 29 +++++++++++++++++++++++++---- main/inc/global.inc.php | 20 ++++++++++++-------- main/install/configuration.dist.php | 8 ++++++++ 3 files changed, 45 insertions(+), 12 deletions(-) diff --git a/documentation/optimization.html b/documentation/optimization.html index 9eff6a97ff..1a86cac9e5 100755 --- a/documentation/optimization.html +++ b/documentation/optimization.html @@ -51,6 +51,7 @@
  • Speeding file downloads with mod_xsendfile
  • IGBinary for faster courses backups and better sessions
  • Removing files download permissions check
    • +
  • MySQL/MariaDB compression

    • 1. Using xCache or APC

    @@ -355,12 +356,14 @@ Don't have time or resources to optimize your Chamilo installation yourself? Hir Valid CSS
    +

    Memory considerations for high numbers of users

    Some administration scripts *have to* handle lists of all users, and this might have a considerable impact on portals with very high numbers of users. For example, the main/admin/add_users_to_session.php script that handles the registration of users into a specific session, if used with the (non-default) full list of users, will devour about 3KB per user, which, for 100,000 users, translates into the need for around 300MB of RAM just to show this page, and to around 3GB for 1,000,000 users.
    This mode is not loaded by default, but could still be selected, leading to a "Fatal error: Allowed memory size ... exhausted" message.
    The only non-scripted solution here is to allow for the corresponding amount of RAM for your PHP configuration (memory_limit = 300M) or your specific VirtualHost if you use mod-php5 (php_value memory_limit 300M).
    -

    Avoiding non-fixed values

    + +

    Avoiding non-fixed values

    Many things in Chamilo are written focusing on the ease of use, even for the administrator. Sometimes, these settings are weighing a little bit more on the system. This is the case, between others, of the mail.conf.php file (being loaded unconditionally) and its CONSTANT "IS_WINDOWS_OS", which is defined by a function call (api_is_windows_os()) at the beginning of main_api.lib.php. The definition of this constant (which is executed at *every* page load) can easily be avoided, and the only place where it is used inconditionally (mail.conf.php) can be modified to set the line as you expect it (depending on whether you use sendmail/exim or smtp). @@ -373,7 +376,7 @@ $platform_email['SMTP_MAILER'] = 'mail'; In fact, the complete loading of mail.conf.php can also be avoided if loaded conditionally (with require_once) when sending an e-mail (which is the only case where it is useful).
    -

    Speeding file downloads with mod_xsendfile

    +

    Speeding file downloads with mod_xsendfile

    It might have come to your attention that file downloads through Chamilo might get slow, under default conditions, in particular using Apache 2.

    There are several ways to fix this, one of which is removing the .htaccess inside the courses/ directory. This, however, will remove all permissions checks on the files contained in this directory, so... most of the time, not ideal unless your portal is *really* open to the world.

    Another technique, revealed to us by VirtualBlackFox on this Stackoverflow post, is to use the X-SendFile module for Apache 2.2+ (other web servers might offer other solutions, or avoid the problem initially).

    @@ -398,11 +401,13 @@ $_configuration['enable_x_sendfile_headers'] = true; Done! Now your downloads should go substantially faster. This is still a feature in observation. We're not sure the benefits are sufficient, so don't hesitate to let us know in the related issue in Chamilo's tracking system

    -

    IGBinary for courses backups and better sessions management

    +
    +

    IGBinary for courses backups and better sessions management

    IGBinary is a small PECL library that replaces the PHP serializer. It uses less space (so less memory for serialized objects) and is particularly efficient with memory-based storages (like Memcached). Use it for course backups (see issue 4443) or to boost sessions management.

    -

    Removing files download permissions check

    +
    +

    Removing files download permissions check

    This measure is not cumulative with mod_xsendfile explained above. It is not *recommended* either, as it removes an important security layer.

    @@ -435,13 +440,29 @@ RewriteRule ([^/]+)/work/(.*)$ /main/work/download.php?file=work/$2&cDir=$1 [QSA
    This is easy, doesn't require a server reload and you should see the results pretty quickly. As mentioned above, if security of your content is an issue, though, you should avoid using this technique.

    +
    +

    MySQL/MariaDB compression

    +If your database server is separate from your web server, you have to play with bandwidth, firewalls, and network restrictions in general.
    +In particular, when dealing with large-scale portals, the time a SQL query will take to return to the web server will take longer and, eventually, in the most critical cases, will take too long, and your web servers will be completely overloaded (load average very high because the system is waiting for I/O operations, but processors usage not being very high is a clear sign of this).
    +To solve this kind of issues, MySQL and MariaDB offer a data compression mechanism, which will reduce the amount of data passed between PHP and the database server. Ultimately, this reduction will lower bandwidth usage and reduce the impact of numerous and heavy data requests (and save you).
    +In 1.10.0, we have added the possibility to enable this compression very easily, from the configuration.php file, uncommenting the following line: +

    +//$_configuration['db_client_flags'] = MYSQL_CLIENT_COMPRESS;
+
    +This should have an immediate effect on the load average on your server.

    Authors

    +
    +Don't have time or resources to optimize your Chamilo installation yourself? Hire an official Chamilo provider and get it sorted out professionally by specialists. + Valid XHTML 1.0 Transitional + + Valid CSS + diff --git a/main/inc/global.inc.php b/main/inc/global.inc.php index ce64215464..4fe264f79e 100755 --- a/main/inc/global.inc.php +++ b/main/inc/global.inc.php @@ -115,14 +115,18 @@ if (empty($_configuration['statistics_database']) && $already_installed) { global $database_connection; // Connect to the server database and select the main chamilo database. // When $_configuration['db_persistent_connection'] is set, it is expected to be a boolean type. -if (!($conn_return = @Database::connect( - array( - 'server' => $_configuration['db_host'], - 'username' => $_configuration['db_user'], - 'password' => $_configuration['db_password'], - 'persistent' => $_configuration['db_persistent_connection'] - ))) -) { +$params = array( + 'server' => $_configuration['db_host'], + 'username' => $_configuration['db_user'], + 'password' => $_configuration['db_password'], + 'persistent' => $_configuration['db_persistent_connection'] +); +// $_configuration['db_client_flags'] can be set in configuration.php to pass +// flags to the DB connection +if (isset($_configuration['db_client_flags']) && !empty($_configuration['db_client_flags'])) { + $params['client_flags'] = $_configuration['db_client_flags']; +} +if (!($conn_return = @Database::connect($params))) { $global_error_code = 3; // The database server is not available or credentials are invalid. require $includePath.'/global_error_message.inc.php'; diff --git a/main/install/configuration.dist.php b/main/install/configuration.dist.php index 2bb8bcdd62..7036e44a02 100755 --- a/main/install/configuration.dist.php +++ b/main/install/configuration.dist.php @@ -35,6 +35,14 @@ $_configuration['db_user'] = '{DATABASE_USER}'; // Your MySQL password $_configuration['db_password'] = '{DATABASE_PASSWORD}'; +// Persistent connections may have profound effects (not always positive) on +// your database server. Use with care. +//$_configuration['db_persistent_connection'] = false; +// For separate web and DB servers, reduce the bandwidth used by compressing +// data returning from the DB server. By default, it is ignored. Uncomment +// the following to enable compression. +//$_configuration['db_client_flags'] = MYSQL_CLIENT_COMPRESS; + /** * Database settings */