Drupal

Drupal is a free and open source content management system (CMS) written in PHP and distributed under the GNU General Public License. It is used as a back-end system for at least 1% of all websites worldwide ranging from personal blogs to larger corporate and political sites including whitehouse.gov and data.gov.uk. It is also used for knowledge management and business collaboration.

The standard release of Drupal, known as Drupal core, contains basic features common to most CMSs. These include user account registration and maintenance, menu management, RSS-feeds, page layout customization, and system administration. The Drupal core installation can be used as a brochureware website, a single- or multi-user blog, an Internet forum, or a community website providing for user-generated content.

Over 6000 (as of October 2010) free community-contributed addons, known as contrib modules, are available to alter and extend Drupal's core capabilities and add new features or customize Drupal's behavior and appearance. Because of this plug-in extensibility and modular design, Drupal is sometimes described as a content management framework. Drupal is also described as a web application framework, as it meets the generally accepted feature requirements for such frameworks.

Although Drupal offers a sophisticated API for developers, no programming skills are required for basic website installation and administration.

Installation

Installation was straight forward and allowed me to install it into the existing database alongside MediaWiki, Joomla and Wordpress using a table prefix.

Structure

From the Drupal overview: If you want to go deeper with Drupal, you should understand how information flows between the system's layers. There are five main layers to consider:

1. At the base of the system is the collection of nodes—the data pool. Before anything can be displayed on the site, it must be input as data.
2. The next layer up is where modules live. Modules are functional plugins that are either part of the Drupal core (they ship with Drupal) or they are contributed items that have been created by members of the Drupal community. Modules build on Drupal's core functionality, allowing you to customize the data items (fields) on your node types; set up e-commerce; programmatically sorting and display of content (custom output controlled by filters you define); and more. There are thousands of different options within the fast-growing repository of contributed Drupal modules. They represent the innovation and collaborative effort of everyone from individuals to large corporations.
3. At the next layer, we find blocks and menus. Blocks often provide the output from a module or can be created to display whatever you want, and then can be placed in various spots in your template (theme) layout. Blocks can be configured to output in various ways, as well as only showing on certain defined pages, or only for certain defined users.
4. Next are user permissions. This is where settings are configured to determine what different kinds of users are allow to do and see. Permissions are defined for various roles, and in turn, users are assigned to these roles in order to grant them the defined permissions.
5. On the top layer is the site theme (the "skin"). This is made up predominantly of XHTML and CSS, with some PHP variables intermixed, so Drupal-generated content can go in the appropriate spots. Also included with each theme is a set of functions that can be used to override standard functions in the modules in order to provide complete control over how the modules generate their markup at output time. Templates can also be assigned on-the-fly based on user permissions.

User, Permission, Role

Every visitor to your site, whether they have an account and log in or visit the site anonymously, is considered a user to Drupal. Each user has a numeric user ID, and non-anonymous users also have a user name and an email address. Other information can also be associated with users by modules; for instance, if you use the core Profile module, you can define user profile fields to be associated with each user.

Anonymous users have a user ID of zero (0). The user with user ID one (1), which is the user account you create when you install Drupal, is special: that user has permission to do absolutely everything on the site.

Other users on your site can be assigned permissions via roles. To do this, you first need to create a role, which you might call "Content editor" or "Member". Next, you will assign permissions to that role, to tell Drupal what that role can and can't do on the site. Finally, you will grant certain users on your site your new role, which will mean that when those users are logged in, Drupal will let them do the actions you gave that role permission to do.

You can also assign permissions for the special built-in roles of "anonymous user" (a user who is not logged in) and "authenticated user" (a user who is logged in, with no special role assignments). Drupal permissions are quite flexible -- you are allowed to assign permission for any task to any role, depending on the needs of your site.

Admin user and role

• Adding a second administrator user
• Admin role in Drupal 6
• Admin role in Drupal 7

Modules

Modules and themes are installed by downloading and unpacking them into sites/all/modules or sites/all/themes or to a domain-specific location such as /sites/foo.bar/modules. Note that modules and themes should always be installed under the sites structure, never directly to the modules/themes directories in the codebase root. To check if the module is installed properly, Go to the admin/build/modules page to enable/disable modules and ensure dependencies are satisfied, and also go to admin/reports/status to see if there are any problems or issues to attend to.

Drupal modules on OD

We have our modules for all Drupal sites shared in /var/www/drupal6_modules and then symlinked from each of the codebases. Here's a list of the main modules we're using.

• Views - a smart query builder. Details below
• CCK (Content Construction Kit) - allows you to add custom fields to nodes using a web browser. Most of CCK has moved to core in the Drupal 7 version.
• Organic Groups - users create and manage their own 'groups'. Details below
• Date & Calendar - display any Views date field in calendar formats, including CCK date fields, node created or updated dates, etc.
• Notifications & Messaging - a Messaging Framework to allow message sending in a channel independent way.
• Voting API & Decisions - a replacement for poll.module and provides advanced voting systems and decision-making tools.
• WYSIWYG - use client-side editors to edit content.
• JQuery UI - wrapper module around the jQuery UI effects library.
• Token - small bits of text that can be placed into larger documents via simple placeholders.
• Flag - a flexible flagging system. Details below
• Translation helpers - enables other modules to respond to changes in the "source translation" of a set of translated content.
• Token - allows for event-driven behaviour from the UI.

Some parts of these modules are incorporated in the core of Drupal 7, which is now released.

Modules to check out

• Extending user profiles
• WikiTools - makes Drupal behave more like a wiki e.g. redirects and red-link ability
• Diff - show diff links in revisions tab
• Location Module
• Pathauto module - automatically generates path aliases for various kinds of content (nodes, categories, users) without requiring the user to manually specify the path alias. This allows you to get aliases like /category/my-node-title.html instead of /node/123
• Views Gallery - Integration with CCK, organic groups
• MERCI - MERCI can extend any content type into a list of unique reservable items (like studios) or buckets of interchangeable items (like DV cameras)
• CiviCRM - CRM for Advocacy, Non-Profit and Non-governmental Groups
• rcmail - Integrates RoundCube mail with Drupal and CiviCRM
• Views Slideshow - The classic front page slideshow, the Drupal 7 tutorial is good though the dev 3.x version of the slideshow must be used.

Views

The Views module provides a flexible method for Drupal site designers to control how lists and tables of content (nodes in Views 1, almost anything in Views 2) are presented. Traditionally, Drupal has hard-coded most of this, particularly in how taxonomy and tracker lists are formatted.

This tool is essentially a smart query builder that, given enough information, can build the proper query, execute it, and display the results. It has four modes, plus a special mode, and provides an impressive amount of functionality from these modes.

Among other things, Views can be used to generate reports, create summaries, and display collections of images and other content.

CKEditor

You can replace textarea fields in the page editor with a visual editor to make it easier for users. CK is an improved version of the old FCK. This is a two stage process:

• Install the CKEditor module
• Install the javascript directory

Instructions are in in the module's documentation.

You have to remember to switch off CK if you want to include HTML or PHP in your page.

Organic Groups

Organic Groups (OG) enables users to create and manage their own 'groups'. Each group can have subscribers, and maintains a group home page where subscribers communicate amongst themselves.

They do so by posting the usual node types: blog, story, page, etc. A block is shown on the group home page which facilitates these posts. The block also provides summary information about the group.

Groups may be selective or not. Selective groups require approval in order to become a member, or even invitation-only groups. There are lots of preferences to configure groups as you need.

Groups get their own theme, language, taxonomy, and so on. Integrates well and depends upon Views module

• Our Organic Group requirements
• Setting up Organic Groups
• Basic Organic Groups documentation
• Organic Groups Basics - video tutorial at drupalove.com
• Organic Groups handbook
• Alternative solution to Organic Groups

Setting up Organic Groups

After initially enabling all the required modules, a message comes up saying You must designate at least one content type to act as a group node and another as a group post. Create new content type if needed. The group node is a content type and is one "page" but it is actually a page that gets turned into the "set up the organic group" page once the configuration is done. You can think of the group node as the front door to your group's pages. It is the form you fill out to create a group.

The other part of the warning message regarding a group post type is saying that none of the existing content types are able to be posted into a group, they are set by default to May not be posted into a group. New content types can be created that can be or existing types can be adjusted to be postable into groups. We intend to use Organic Groups as an overall ontology containing everything in the site, so we adjust the existing content types rather than creating new ones.

To make groups usable, group related blocks must be added to the site, go to admin/site building/blocks to see the available blocks and add them to the various areas of the site. Note that some of these blocks only show up in specific contexts.

Adding a forum to a group

First create the forums and tell them not to be promoted to the front page since they'll be in their respective groups instead. Next, the authenticated users must be given permission to manipulate forum topics, either all or just their own. And also permission in the node module to access content.

Note: Forms have an inherent hierarchy which means that the Organic Groups module is redundant if forums are the only type being used in the group. The hierarchy is Containers → Categories ("forums") → Topics → Comments.

Adding members to closed groups

To add members to a closed group, first go to the groups administration page from administration/groups and then the group specific block becomes visible, and you can see the number of current members as a link. Clicking this link takes you to the groups users page which has an option to add members.

Automatically Create a Forum when you create a Group

Download and install OG Forums.

Go to admin/og/og_forum and click on "Update old groups" to have forum containers and default forums created for existing groups. Leave all the options on default unless you want to change them for a particular reason, then save. For further info go to the documentation.

Don't forget to update the permissions for each group to create new forum discussions or it will not be allowed.

Security

Like MediaWiki, Drupal doesn't come with the ability to restrict read access out of the box. The most popular method for achieving this appears to be the Taxonomy Access Control module (TAC) for permitting access based on tagging and roles, or the TAC lite module to restrict by individual user. You can also use Node Access User Reference and Node Access Node Reference to create fine-grained hierarchical access control, see the alternative solution to Organic Groups blog entry for more detail.

However, the Organic Groups module, discussed above, is more appropriate for sites that are already using it, since it has its own access control mechanism based on the group structure.

Flag

Flag (previously known as Views Bookmark) is a flexible flagging system that is completely customizable by the administrator. Using this module, the site administrator can provide any number of flags for nodes, comments, or users. Some possibilities include bookmarks, marking important, friends, or flag as offensive. With extensive views integration, you can create custom lists of popular content or keep tabs on important content.

Flags may be per-user, meaning that each user can mark an item individually, or global, meaning that the item is either marked or it is not marked, and any user who changes that changes it for everyone.

In this way, additional flags (similar to published and sticky) can be put on nodes, or other items, and dealt with by the system however the administration likes.

Rules

Install Rules module if you want to have event-driven behaviour. Make sure rule=making users have permission to view the Rules Admin module.

To make a Rule

• go Structure - Rules - Add New Rule
• give it a Label
• add Events, Conditions and Actions to this rule using the menu provided.
• if the trigger, condition or action that you want does not appear in the menu, you can write your own, using PHP and the hooks provided.
• use Rulesets to setup complex rules that can be reused
• pass data around rules using Tokens
• see Rules screencasts below.

Example: Assign Roles to Groups Using Rules

• Make sure you have your Users and Roles set up first.
• Go Admin - Rules - Triggered Rules - Add a new rule
• Give it a Label suitable for that particular rule, you may have several rules
• Add Event 'User has logged in'
• Add a Condition 'User Has Roles' and then whatever role you have chosen
• Add Action 'Subscribe user to the specified groups', there is a multiselect of groups
• If you now assign a user the role selected then the next time they log in they can see that private group and use its contents.
• You can test by logging in as that user, setting up a forum etc
• You have to write a condition for every role, but a role can autojoin multiple groups.
• It is possible to handwrite a hook so the action only occurs every once when a role is assigned.
• You can unjoin users from groups by making a new Undo rule, Negating a group membership condition then Unsubscribing from the group as the action.

Spamicide

The spamicide module is designed to prevent bots spamming forms without requiring the user to fill in a CAPTCHA. It doesn't seem to be under active development and has a few problems such as ignoring which forms you've elected to put the protection on and putting it on all of them regardless. It works by adding a new hidden input to the form and then treating submissions as spam if that input is filled in.

Unfortunately it seems that many bots in the field recognise this input, probably by the recognisable surrounding div. So I modified the module slightly by making it require the hidden input to be populated with "foobaz" and making the forms onSubmit event populate set the input's value to this. I did this with the following modfications:

I added the following snippet after line 288, the first line of the spamicide_form_alter function:

$form['#attributes']['onsubmit']='$("#edit-feed-me").val("foobaz")';

Next I modified line 341 to check for the "foobaz" value instead of ensuring it's empty:

elseif ( $form_state['values'][$spamicide_field] == 'foobaz' ) {

Project management

• Project Management / Ticket Tracking Systems - Greg Knaddison compares project management and ticket tracking systems
• Storm - offers a hierarchy of content types to help with project management
  • Storm drupal discussion group
• How to make your own BaseCamp in Drupal - a video tutorial
• Usable demo of BaseCamp in Drupal

Upgrading

The general upgrade procedure is to first put the site down for maintenance in admin/settings/site-maintenance. Then backup the file structure, replace with the latest download, then go to the /update.php script. After this has run, the original sites directory can be put back, and then go to admin/build/modules to see if any modules also require updating.

Skinning

Zen is the ultimate starting theme for Drupal 6 and 7. If you are building your own standards-compliant theme, you will find it much easier to start with Zen than to start with Garland or Bluemarine. This theme has fantastic online documentation and tons of code comments for both the PHP (template.php) and HTML (page.tpl.php, node.tpl.php).

The idea behind the Zen theme is to have a very flexible standards-compliant and semantically correct XHTML theme that can be highly modified through CSS and an enhanced version of Drupal’s template system.

Out of the box, Zen is clean and simple with either a one, two, or three column layout of fixed or liquid width. In addition, the HTML source order has content placed before sidebars or the navbar for increased accessibility and SEO.

The name is an homage to the CSS Zen Garden site where designers can redesign the page purely through the use of CSS.

Breadcrumbs

Drupal 6 and 7 both put breadcrumbs in the header, this might not be convenient. You can remove or relocate this feature by making .breadcrumb display none in CSS, or by removing or relocating the position of this line in page.tpl.php:

<?php print $breadcrumb; ?>

Links

• The Zen project page
• Free Drupal Themes - from SiteGround
• ThemeGarden

Farming

Drupal is already set up with the farm model in mind. in addition to the global settings, modules and themes in the codebase root, there is also a directory called sites which is used to add additional global or site specific settings, modules and themes. The sites directory contains a directory called "all" and any number of other directories named by domain or domain segment.

Clean URLs

Drupal supports clean URLs, but to have them working within our Wikia configuration requires a slightly different configuration than that recommended in the Drupal documentation on the subject. In the global Apache virtual-host container, use the following format. This maps the domain www.example-drupal.com to the drupal code-base in /var/www/domains/example-drupal.

RewriteCond %{HTTP_HOST} www\.example-drupal\.com
RewriteCond %{REQUEST_URI} \.
RewriteRule (.*) /example-drupal$1 [L,QSA]
RewriteCond %{HTTP_HOST} www\.example-drupal\.com
RewriteRule (.*) /example-drupal/index.php?q=$1 [L,QSA]

In addition to this, the following modification must be added to the sites/default/settings.php file in the base-URL section:

$base_url = preg_replace( '|(?<=\w)/.*|', '', $_SERVER['SCRIPT_URI'] );
if( $_GET['q'] == '/' ) $_GET['q'] = '';

Don't forget to enable clean URLs in the site administration section after making these modifications.

Scalability

In Lynda.com's Drupal Essential Training video series, Tom Geller said:

...you shouldn't use Drupal if it's going to be an extremely high-traffic or mission-critical site. If it's necessary to have this site online to save lives, you should probably use something else or at least have something else as a backup. If you're going to run something that's going to have millions and millions and millions of page views, probably Drupal is not the right solution -- although I should mention, Drupal does run very popular sites.

— "Choosing Drupal" video, timecode approx. 5:00

This has opened up a bit of a can of worms for us as we're involved in a project for which Drupal would have been a very appropriate starting point, but will need to be capable of serving very high traffic. Due to this comment we'd started looking in to using Plone on ZEO to scale, but upon further investigation it seems that Drupal may be ok after all as a number of very high traffic sites such as theonion.com are Drupal based.

We need to get hold of some high-traffic Drupal statistics that specifically relate to the number of edits per second, not just the number of concurrent visitors. This is because any CMS can be made to serve up any amount of static content without too much difficulty simply by adding a caching layer that diverts requests for unchanged content to an array of proxy servers. The difficulty comes when the CMS is required to respond to many requests per second which occurs if users need to make a change to the content such as by posting a comment.

• Sites using Drupal, ordered by popularity
• Tom Geller backtracks on the comment
• Scaling Drupal - an open-source infrastructure for high-traffic drupal sites
• Can a Drupal site handle a million page views per day?
• Presentation about 2.8 million page views per day in Drupal - very good presentation by Drupal scalability guru Khalid from 2bit.com

NoSQL for Drupal

• Integration of a No-SQL Database - proposal by a student with some community feedback
• noSQL and SQL - video guide to NoSQL by Drupal guy
• MongoDB

Our Settings

When setting up Organic Groups, a decision must be made about whether to create duplicate node types for posts and pages etc that are allowed to be posted into groups (it seems that the two options are mutually exclusive). Our plan for Organic Groups is for it to be our main ontology structure and so all content should be in a group. So for our purposes we can set all node types to be standard or wiki group posts.

In Drupal 7 standard vs wiki group posts no longer exists.

Things that could be improved

• Adding users to groups is pretty lame, done by comma-separated text, should be a multi-select
• There is no table of contents on the module page in Drupal 7, if a lot of modules are installed you have to use the browser's Find functionality.

Drupal 7

OG for Drupal 7

First set up a Drupal 7.

Using the explanations in the article http://drupal.org/documentation/install/basic :

• Download and untar the Drupal into the /var/www/domains directory in its own folder.
• Set up a new database. Use root as your mysql db user then you don't need to grant or add.
• Make the sites/default directory writable.
• Note you will have to add your new domain to /etc/apache2/sites-available/default to be able to browse to your new directory, which will need to bypass the wikia structure. Example config in http://organicdesign.co.nz/Set_up_a_new_domain_name
• Navigate to the new url and set up the drupal, it is very interactive.
• Set up modules by untarring into the modules dir:
  • ctools
  • entity
  • views
  • og
• You also need to install prerequisite modules:
  • messaging
  • notifications
  • token

Organic groups should now be working. If not, obey the UI's instructions to fix things, they are perfectly clear.

(Note at this stage we are using nightly builds of the modules. Some urls will be provided later.)

Next add new a content type called Group by navigating: Administer - Structure - Content Types.

Make sure under the group tab that it is:

• Group Type
• Not a Group Content Type

You can then make, say, group Posts by making them (group tab):

• Not a Group Type
• Group Content Type

Git for Drupal

To get access to Drupal development and contribute or patch modules you need to:

• register
• get a Git account with Drupal.
• identify yourself to Git.

Then follow the Git Instructions tab on the relevant Drupal project page, e.g. this example.

Migrating MediaWiki content to Drupal

• Creating nodes programmatically

See also

• Our Drupal test installation
• Official Drupal site
• Getting started with Drupal 6
• Examples for developers
• Code snippets
• Drupal Best Practices
• Structure Guide
• Drupal tutorial from SiteGround
• P2P Foundation's Drupal page
• Diaspora distributed social network vs Drupal: re-inventing the wheel?
• Drupal7 and the Semantic Web
• Drupal 7.x Taxonomy page, daily post by all comers
• Skinning Drupal 7 using Zen and CSS Video. Uses Mac CSS editor, we use Firebug, does the same thing.
• Drupal conference OG 7 links
• Drupal 7 conference development section links
• Create New Content Type in Drupal 7
• Organic Groups Module
• Series of screencasts about Rules
• Changes in URLs from Drupals 6 to 7
• CCK Module Basics
• A quick tutorial on calendar setup in Organic Groups for Drupal 6.
• Interesting article about Drupal contribution and sponsorship statistics