Move download button and select below table

If you are working with flexibile tables (lib/tablelib.php) in Moodle and have table download enabled, you can move the “Download table data as” select and button below the table by adding the following line before finishing your table setup:

$table->show_download_buttons_at(array(TABLE_P_BOTTOM));

By default they appear above the table but this doesn’t always look so good and takes up screen space.

Working with dates before epoch

If you use a custom user profile field with a date and want to store a date that falls before epoch, 01/01/1970 (e.g. a date of birth), the data will be stored as a negative number
e.g. 20/11/1959 is stored as -319284000 which means 319, 284, 000 seconds BEFORE 01/01/1970.

With MySQL you can convert these with the following SQL which works for both scenarios (before and after epoch, 01/01/1970):

select date_add(from_unixtime(0), interval data second) from mdl_user_info_field

Where in the above “data” is the column that returns the stored user profile field value e.g. date of birth from the mdl_user_info_field record.

If you just want the date (no timestamp):

select date(date_add(from_unixtime(0), interval data second)) from mdl_user_info_field

Getting the Moodle base / root directory

The moodle root directory $CFG->dirroot is established in the file lib/setup.php along with other values in the global $CFG object which combines configuraton from config.php as well as the entries in the database table mdl_config.

The value is set as follows:

$CFG->dirroot = dirname(dirname(__FILE__));

And you can use it reliably establish the base/root moodle directory absolute path and use it for things like file requires in your plugin.

Some other useful entries in lib/setup.php from the code:

$CFG->wwwroot  - Path to moodle index directory in url format.
$CFG->dataroot - Path to moodle data files directory on server's filesystem.
$CFG->dirroot  - Path to moodle's library folder on server's filesystem.
$CFG->libdir   - Path to moodle's library folder on server's filesystem.
$CFG->tempdir  - Path to moodle's temp file directory on server's filesystem.
$CFG->cachedir - Path to moodle's cache directory on server's filesystem (shared by cluster nodes).
$CFG->localcachedir - Path to moodle's local cache directory (not shared by cluster nodes).

Lots more in there have a dig around in lib/setup.php.

Make a select form field required

When adding a form rule for select (or multiple select) fields, these need to be applied on the client side (not server side). Here’s an example of how to add a rule for a select field:

        $mform->addRule(
            'selectfield', 
            get_string("selectfieldrequiredmessage", "local_yourplugin"), 
            'required',
            '',
            'client'
        );   

The key is the 5th parameter above which is set to client instead of the default of server.

For more help, see the Moodle Forms library developer documentation.

Scheduled Tasks: A lock was created but not released

Schedule tasks can use two types of locks:
* Database locks — lock_db table
* File locks — dirroot/lock

The follow error can occur when running a cron through the CLI for example:

!!! Coding error detected, it must be fixed by a programmer: A lock was created but not released at:
[dirroot]/lib/classes/task/manager.php on line 483

 Code should look like:

 $factory = \core\lock\lock_config::get_lock_factory('type');
 $lock = $factory->get_lock(Resource id #585);
 $lock->release();  // Locks must ALWAYS be released like this.

As a starting point, trying disabling the schedule task causing the error and see if you can get the cron to run successfully.

You can try clearing out the locks as well from the database/filestore (whichever is relevant) and running the cron again.

Setting up a Moodle Development Environment with Cloud9 IDE

Cloud9 is a cloud based development environment. Behind the scenes it uses Docker containers to provide an awesome IDE together with a Workspace using Ubuntu. What this means it that you can you set up a fully integrated Moodle development environment on Cloud9!

This tutorial uses Moodle 3.1 for because as of May 2017 Cloud9 ships by default with PHP 5.5.9 which doesn’t meet the minimum requirements for Moodle 3.2 (https://docs.moodle.org/dev/Moodle_3.2_release_notes). You can work around this but it is more involved so for now, lets stick with Moodle 3.1.

To start with, signup for a new (free account). Note while the account is free, it will require you to enter a credit card just to verify your identity.

https://c9.io/

Once you have an account and have logged in, add a new Workspace.

  • Enter your workspace name (e.g. Moodle)
  • A description e.g. Moodle 3.1 Development Environment
  • Choose Private (you get one free private workspace) or public
  • Choose a template and select PHP, APache & MySQL
  • Create your workspace

Note most of the steps that follow are just regular linux commands and set up instructions for installing Moodle.
Nothing particularly specific to Cloud9, but there are a few instances where the commands vary slighly e.g. for accessing the MySQL CLI.

Cloning Moodle via Git

Once you see your IDE, go to the terminal at the bottom. The first step is to clone moodle via git.

You should be in ~/workspace (if not cd ~/workspace).

git clone git://git.moodle.org/moodle.git moodle

This will clone it to ~/workspace/moodle

Once finished, checkout the MOODLE_31_STABLE branch

git checkout MOODLE_31_STABLE

Now we want to move everything in moodle to the ~/workspace folder as that is the base directory used when you choose Run Project.

cd ~/workspace/moodle
mv * ../
mv .* ../
cd ..
rmdir moodle

Note you will see the following message which can be ignored. Its just to get all the .* files into the workspace folder.

mv: cannot move ‘.’ to ‘../.’: Device or resource busy
mv: cannot move ‘..’ to ‘../..’: Device or resource busy

Now you have everything that was cloned in~/workspace/moodle in the ~/workspace folder and it should show you are in the MOODLE_31_STABLE branch

<user>~/workspace (MOODLE_31_STABLE) $ 

Setting up the Database

Next we need to set up a MySQL database. In the terminal type:

mysql-ctl cli

Note you may need to type mysql-ctl install if it is not already installed

Create your moodle database, user and give the user full access to the database.

mysql> create database moodle character set utf8 collate utf8_general_ci;
mysql> create user [email protected] identified by 'moodle';
mysql> grant all on moodle.* to [email protected];

mysql> exit

Setting up the data directory

We’ll put this under ~/moodledata
Create this directory:

mkdir ~/moodledata

( Note you could do this via the file manager on the left hand side of the IDE if you click on the small cog in the left pane and choose “Show Home in Favourites”. )

Install Moodle

Ok, you are ready to hit “Run Project” to fire up Apache & PHP.
This will open in a new tab and give you a URL to where it is serving e.g.

https://moodle-yourusername.c9users.io/

Go ahead and visit that page or click on the link and choose open. Hopefully you see the Moodle installation screen. Note on the first few screens before the license the stylesheet/layout won’t look great, but it will sort itself out later on.

Follow the prompts:

  • Pick your language
  • Your web address, Moodle directory (workspace) and Data directory (moodledata) should all default
  • Database drive is MySQL (native/mysqli)
  • Database host is localhost (default) name is moodle (default) enter user as moodle and password as moodle and choose next (prefix/port/socket all stay at defaults).
  • You should see the installation screen and license, choose continue.
  • All server checks should pass, so choose continue
  • The installation will take a while, let it run through. When finished you will see a continue button on the bottom of the page (may need to scroll to the end).
  • When finished you should see a screen where you can set up your admin user account. Make sure you enter a valid email here.
  • Finally you can enter your front page settings; the full site name e.g. Moodle 3.1 Development Environment and short name e.g. moodle31dev. Update your timezone and save.

That’s it, you have a working Moodle 3.1.x environment that you can now development against with the Cloud 9 IDE!

XMLDB Editor and Reserved Names

One (of the many) good reasons to use the XMLDB editor in Moodle rather than hacking the db/install.xml file directly is that it warns you if you use a reserved name. For example:

XMLDB Reserved name error

The red Reserved text after [XML] in the screenshot is indicating that the word “column” is reserved and shouldn’t be used. It would be nice if it stopped you from saving such a change (it doesn’t) but at least it indicates a problem.

The Reserved link points to the following location if you ever want to look up the currently used reserved words:

<http://www.yourmoodle.com>/admin/tool/xmldb/index.php?action=view_reserved_words

This link also shows you potential existing problems in your XMLDB that may need to be fixed.

Replacing strings in the database through the CLI

There is a tool provided with Moodle to replace strings in your database with another value. This is very useful for things such as site address or domain name changes, and changes from non-SSL (http) to SSL (https).

The tool can be accessed via the following relative URL as a site administrator.
/admin/tool/replace/

While you can certainly run it from the browser / site, it can timeout and cause problems to do so. Therefore the CLI version is a far better option and available in the moodle code tree under:

/admin/tool/replace/cli/

php admin/tool/replace/cli/replace.php

Search and replace text throughout the whole database.

Options:
--search=STRING       String to search for.
--replace=STRING      String to replace with.
--shorten             Shorten result if necessary.
--non-interactive     Perform the replacement without confirming.
-h, --help            Print out this help.

Note you might need sudo above. So lets say you change your site form http://yoursite.com to https://yoursite.com you could run it like so:

php admin/tool/replace/cli/replace.php 
--search=http://yoursite.com --replace=https://yoursite.com  --non-interactive

This will scan through tables and give you results in the console as it is going. The non-interactive flag prevents prompting (assuming you are comfortable with the change). If you aren’t you probably shouldn’t use the tool or at least use it on a test site first. The shorten result flag shouldn’t be necessary in most cases.

Resetting a user’s password through the CLI

If you have access to the Moodle server, you can use the admin/cli/reset_password.php script to reset passwords for users with the manual authentication method.

To run this script (from the moodle root code directory), note you may need to sudo as a user with write privileges to the $CFG->dataroot directory.

php admin/cli/reset_password.php

== Password reset ==
Enter username (manual authentication only)
: moodletips
Enter new password
: moodletips

Note the password you specified has to match the password rules configured in Moodle for example the above password moodletips by default would throw:

Passwords must have at least 1 digit(s).
Passwords must have at least 1 upper case letter(s).
Passwords must have at least 1 non-alphanumeric character(s) such as as *, -, or #.

Very handy if you are unable to get into the Moodle site for any reason, or need a quick way to reset a user’s password.

Set User IDs that will always see debug messages

There’s a handy feature in Moodle configuration called $CFG->debugusers = '<moodle_user_ids>' that allows you to set certain user IDs to always see debug messages regardless fo the Site administration > Development > Debugging settings for the site. This can be a comma separated list of Moodle User Ids (from the user table). Set this in the config.php or see config-dist.php for more details.

Useful if you need to troubleshoot an error but don’t want to increase debug level for the entire system and catch messages for the other users in the system too.

Course file system space usage query

The following query provides a summary of the file space usage (in bytes) and number of files used by each course in the system from the mdl_files meta table for the data directory. This is just for the space used in the data directory (not the database).

select 
    f.contextid,
    x.instanceid, 
    c.fullname as course_full_name,
    c.shortname as course_short_name,
    sum(f.filesize) as size_in_bytes,
    sum(case when (f.filesize > 0) then 1 else 0 end) as number_of_files
from
    mdl_files f inner join mdl_context x
    on f.contextid = x.id
    and x.contextlevel = 50
    inner join mdl_course c
    on c.id = x.instanceid
group by 
    f.contextid, x.instanceid
order by 
    sum(filesize) desc
;

It may need some refinement. Context level 50 refers to courses.

Overriding PHP Settings

There are a few ways to override global PHP (php.ini) settings with local settings. For example you can use a .htaccess file or local php.ini file depending on what options exist for your site hosting.

Another way to do it, is to override the settings in config.php. This has the benefit of being part of the standard moodle code configuration setup. All you need to do is use the ini_set() function. For example, to increase the max_execution_time from 30s to 60s you can do this in config.php:

ini_set('max_execution_time', 60);

The best place for a change like this is towards the end of the config.php, and perhaps wrapped in comments indicating custom configuration.

Getting PHP Info for your Moodle Site

There is a handy link under Site adminstration > Server > PHP Info ( or browse to http://yoursite.com/admin/phpinfo.php ) that will return the standard PHP Info phpinfo() results for your server.

Very handy for checking things like installed modules and core parameters if you are overriding them ( e.g. error_log, max_execution_time, post_max_size, upload_max_size etc )

Moodle Debug Log Level Values

These are the values you can set for the debug level under Site administration > Development > Debugging for Debug message (debug).
You can also query this from the mdl_config table looking at the key debug.

  • 0 = NONE: Do not show any errors or warnings
  • 5 = MINIMAL: Show only fatal errors
  • 15 = NORMAL: Show errors, warnings and notices
  • 30719 = ALL: Show all reasonable PHP debug messages
  • 32767 = DEVELOPER: extra Moodle debug messages for developers

Those are values you can set in the database for the debug entry in mdl_config if you need to check or change this on the database. Good idea to purge the cache when you do too.

These are defined with PHP error constants in lib/setuplib.php as follows (note this might change depending on versions):

DEBUG_NONE = 0
DEBUG_MINIMAL = E_ERROR | E_PARSE
DEBUG_NORMAL = E_ERROR | E_PARSE | E_WARNING | E_NOTICE
DEBUG_ALL = E_ALL & ~E_STRICT
DEBUG_DEVELOPER = E_ALL | E_STRICT

Extended characters in moodle usernames

If you need to use extended characters in usernames, there is a setting under Site Administration > Security > Site Policies called Allow extended characters in usernames extendedusernamechars.

By default moodle usernames must be lowercase and are restricted to alphanumeric characters, the underscore, hypen, period and @ symbol.

You don’t need this if you are using email addresses as usernames (as the @ symbol is allowed) but you might run into a few other edge cases with email addresses:

  • Emails that are mixed case e.g. [email protected] need to be lowercased to [email protected]
  • Emails can have other characters such as an apostrophe (e.g. jack.o’[email protected]) and if you want to preserve such characters you’ll need to turn this setting on. Alternatively you can remove such characters from the username.
  • Emails might have extended language characters in them which required this setting to be turned on as well if you want to match the username to the email addresse exactly.

Also worth noting is that the lib/classes/user.php definition uses PARAM_USERNAME (not PARAM_RAW) anymore in recent versions of moodle. This is used for things like the core user web services. So you will hit invalid parameter exceptions if you specify a username in say mixed case format or with extended characters (and the extendedusernamechars setting is off). This can be confusing it seems like you have specified the wrong parameters but what’s happening is that your parameter does not follow the definition of PARAM_USERNAME. To see the code used to test PARAM_USERNAME have a look in lib/moodlelib.php -> clean_param()

Verify your Moodle Database Schema against XMLDB

There is a very handy CLI tool admin/cli/check_database_schema.php which compares the structure of your Moodle database against the XMLDB metadata and looks for any issues.

These might includes issues such as:

  • Missing tables – they exist in XMLDB but not in your DB (not good!)
  • Unexpected tables – tables in your database not defined in XMLDB (e.g. created outside of Moodle)
  • Mismatches between table column definitions in the database and XMLDB

Some things may be acceptable to ignore, but it pays to do a check, particularly after an upgrade to make sure the upgrade process worked correctly and in particular you aren’t missing any core or plugin database tables that are defined in XMLDB, as these will pop up as database errors in your application sooner or later (e.g. “Error reading from database”).

If you need to build any missing core tables, remember they are defined under Developer > XMLDB Editor. From here find lib/db (core database tables) and choose Load, followed by Edit. Find the table you need and choose Edit. You can then use View SQL code to get the relevant SQL commands to create the table if it is missing from your database.

Also as a developer, make sure you are correctly utilising XMLDB for defining your plugin’s database tables. Taking shortcuts such as directly creating tables in the database without defining them in XMLDB will mean that they will pop up as unexpected when checking the database schema.

CLI Maintenance Mode

Newer versions of Moodle now provide the ability to put a site into maintenance mode through the CLI:

This can be found under the standard admin CLI location

php admin/cli/maintenance.php

Options include

php admin/cli/maintenance.php --enable
php admin/cli/maintenance.php --disable

NOTE this disables all web access to the site and can only be restored by disabling maintenance mode through the CLI again. This is exactly what you want if you need to take a proper site backup. The message displayed will be the maintenance mode message set up under Site Administration > Server > Maintenance Mode > Optional maintenance message (maintenance_message).

There is also a cool feature to enable it later e.g.

php admin/cli/maintenance.php --enablelater=60

Which would enable it after 60 minutes. This will tell exactly when maintenance mode will be enabled too. Good for making sure the site is in maintenance mode when you are ready to start an upgrade for example.

Finally there is an option called “enableold”

php admin/cli/maintenance.php --enableold

This is meant to do the same thing as the user interface Site Administration > Server > Maintenance Mode > Maintenance mode setting. That is site admins still have access to the site but the site is otherwise in maintenance mode for everyone else. This works in Moodle 3.2 but I’ve had problems with it in prior versions of moodle. It could possibly be around caching and might be important to also clear the moodle cache at the same time.

Table formats from Antelope to Barracuda

On your environments page, as of around Moodle 2.6 you might see a message like this:


unsupported_db_table_row_format if this test fails, it indicates a potential problem

Your database has tables using Antelope as the file format. You are recommended to convert the tables to the Barracuda file format. See the documentation Administration via command line for details of a tool for converting InnoDB tables to Barracuda.

If you are running MySQL/MariaDB, this message is advising you should move to the better InnoDB Barracuda format.

To perform the table upgrades, you use the Moodle cli (i.e. admin/cli) and run the following commands (prefix with sudo if relevant):

php admin/cli/mysql_compressed_rows.php -l

This lists tables that need compacting/compressing e.g.

mdl_data                        Compact     (needs fixing)
mdl_data_fields                 Compact     (needs fixing)
mdl_enrol_paypal                Compact     (needs fixing)
mdl_lti                         Compact     (needs fixing)
mdl_user                        Compact     (needs fixing)
mdl_user_info_field             Compact     (needs fixing)

To fix, run the following. Note this can be database intensive so avoid running during high load periods or when there are a lot of active users on the site.

php admin/cli/mysql_compressed_rows.php -f

mdl_data                         ... Compressed
mdl_data_fields                  ... Compressed
mdl_enrol_paypal                 ... Compressed
mdl_lti                          ... Compressed
mdl_user                         ... Compressed
mdl_user_info_field              ... Compressed

To confirm all is well, run the first info command again or check the admin/environment.php page again.

Updating the Moodle Frontpage Summary through the database

You might have the need to update the Moodle front page summary through the database rather than through the UI. For example you might get an error editing the summary after a clone, or you might want to script the cloning process and change the frontpage text through the database.

Here are the steps:

Update the moodle course sections summary field for the relevant course and summary (you might need to find these first) for your frontpage.
The course ID is probably 1 for your frontpage, but you’ll need to confirm this. You can find the section id by looking at the URL that is used to edit the section (e.g. http://localhost/course/editsection.php?id=xxx).

update mdl_course_sections 
set summary = 'Your text/HTML' 
where course = x 
and section = y;

You’ll probably find that even after you purge the moodle cache this doesn’t take effect. You’ll need to also clear your course section cache with the following SQL.

update mdl_course set sectioncache = '' where id = x;

Alternatively with newer versions of moodle you may also need to purge your moodle cache.

Upgrading database schema using upgrade.php

If you need to upgrade your database schema for a custom plugin, e.g. adding a new table or changing the properties of an existing table, you should do it through db/upgrade.php per the suggestions in the MoodleDocs UpgradeAPI.

This isn’t a bad idea to do even when iterating through development as it isn’t a lot of work to keep things synchronised through the XMLDB editor and you won’t miss something later.

These are the basic steps involved:

  • Use the Moodle XMLDB editor to adjust your schema: Site adminstration > Development > XMLDB editor
  • Copy the relevant changes into your install.xml and upgrade.php straight from the editor
  • Update your plugin version and test.

The Moodle XMLDB editor actually makes this very easy.

  • Open the XMLDB editor
  • Find your plugin (assuming it is already installed etc). If you haven’t managed to get this far, at least install what you have so it will show up in XMLDB editor. Note you do not need to have anything DB related defined for this to work.
    Load the DB for you plugin or if nothing is defined using Create.
  • Follow the UI to create your table, fields, keys and indexes
  • When you are ready use View PHP code to get the code to go into db/upgrade.php
  • Go all the way back to the main XMLDB editor screen (where you can see all plugin databases) and use save to save the XML to db/install.xml as well.

How to get a list of the Moodle Core Git Branches

Use the following git command to get a list of all the Moodle core git branches.

git ls-remote git://git.moodle.org/moodle.git | grep head

This returns a bunch of hashes and the refs/heads/[MOODLE_XX_STABLE] branches. You can guess these, so this is just to check what they are without having to clone the entire repository. Plus you’ll know when a new stable branch is available in the repository.

To clone a specific branch (e.g. say I only want Moodle 2.2), use this command in the relevant target directory (note the -t to include tags so you can go to specific tags like v2.2.2):

git clone -b MOODLE_22_STABLE git://git.moodle.org/moodle.git

$DB->get_record_sql only returning a single row

So you’ve written a query, passed it to $DB->get_records_sql() and, what’s this – only one row? What’s going on, where’s the rest of the query data?

First off, make sure you are using the plural form (get_records_sql() and not get_record_sql()).

If that’s not the issue then read on.

The reason this can happen is that Moodle uses the first column in your query as a unique id and then filters results down to that id field. However, if your first column isn’t unique across your data set, you won’t get every row.

For example, your query may select user enrolments for a course. But if you put your course id as the first column, you’ll only get one row, even if there are 30 enrolments in that course, because all of those enrolments belong to that one course id and that is the unique identifier used.

The fix is simple, find (or if you need to, create) a unique id so that the first column is unique for every row in your query. In the user enrolments example you might use the user ID or the user_enrolments ID.

Moodle User Management URLs

Some handy user management and search URLs, particularly if the site admin block is hidden.

User search

Note a %20=means a blank space, encoded to work in a URL.

user/index.php?id=1&search=
http://yourmoodle.com/user/index.php?id=1&search=fred
http://yourmoodle.com/user/index.php?id=1&search=Fred%20Jones

Create a new user

user/editadvanced.php?id=-1
http://yourmoodle.com/user/editadvanced.php?id=-1

Edit a user if you know their moodle internal id

user/editadvanced.php?id={moodle_user_id}
http://yourmoodle.com/user/editadvanced.php?id=16

Delete an existing user

This requires a session key (which is a godo thing!) so you need to do this once in the user edit area.

admin/user.php

If you don’t see a delete action for a user it usually means they are a site admin so you need to take them out of site admin first.

Assigning site administrators

admin/roles/admins.php
http://yourmoodle.com/admin/roles/admins.php

Uninstalling a Moodle Plugin and forcing reinstallation

The easiest way to get moodle to re-run installation is to update the version in version.php. However, if you want to be sure it makes any schema changes etc then remove the plugin version from the mdl_config_plugins table. Note you can leave the other configuration there if required to save re-setting up the plugin.

The SQL to do this:

delete from mdl_config_plugins 
where plugin = 'plugin_name' 
and name = 'version';

For schema changes, you may need to drop the table and have it rebuild it just to ensure that the changes are made correctly.

For production deployments you should use an upgrade.php script, but this is a handy shortcut if you are developing a plugin as you really just want to iterate to the final schema version which you would then put into upgrade.php.

Moodle and Timezones

Moodle timezone information in the first instance is loaded from:

http://download.moodle.org/timezone/

This returns a text file which is stored in the lib directory as timezone.txt

The data is stored in the table mdl_timezone (adjust to suit your db prefix).