fromJune 2014
Feature:

Something Borrowed, Something Drupal

Outsourcing Open Source
3


Drupal 8 represents a radical shift, both technically and culturally, from previous versions. Perusing through the Drupal 8 code base, many parts may be unfamiliar. One bit in particular, though, is especially unusual: A new directory named /core/vendor. What is this mysterious place, and who is vending?

The "vendor" directory represents Drupal's largest cultural shift. It is where Drupal's 3rd party dependencies are stored. The structure of that directory is a product of Composer, the PHP-standard mechanism for declaring dependencies on other packages and downloading them as needed. We won't go into detail about how Composer works; for that, see my article in the September 2013 issue of Drupal Watchdog, Composer: Sharing Wider.

But what 3rd party code are we actually using, and why?

Crack open your IDE if you want, or just follow along at home, as we embark on a tour of Drupal 8's 3rd party dependencies. (We won't be going in alphabetical order.)

Guzzle

Perhaps the easiest to discuss is Guzzle. Guzzle is an HTTP client for PHP; that is, it allows you to make outbound HTTP requests with far more flexibility (and a far, far nicer API) than using curl or some other very low-level library.

Drupal had its own HTTP client for a long time... sort of. The drupal_http_request() function has been around longer than I have, and served as Drupal's sole outbound HTTP utility. Unfortunately, it was never very good. In fact, it sucked. HTTP is not a simple spec, especially HTTP 1.1, and supporting it properly is difficult. drupal_http_request() was always an after-thought, and lacked many features that some users needed.

What's more, it was a single function – one single 304 line function with a cyclomatic complexity of 41 and an N-Path complexity of over 25 billion. That's a fancy way of saying "completely and utterly impossible to unit test before the heat death of the universe." For a modern web platform, that's simply not good enough. (For more on cyclomatic complexity and N-Path complexity, see Anthony Ferrara's talk from DrupalCon Portland, “Development by the Numbers.”)

As we said, though, writing a good HTTP client is quite hard, and we already had plenty of hard tasks to do in Drupal 8. So instead, we outsourced it. After conducting a comparison survey of over a half-dozen different HTTP clients for PHP, we settled on Guzzle as the most feature-rich. The developer's decision to refactor Guzzle itself – to make it easier for Drupal to use just the portions we wanted – helped, too.

Guzzle actually has a lot of other capabilities that we're not using in core, but can be downloaded quite easily. One of the most interesting is the ability to auto-map RESTful services to PHP classes. (See the Guzzle documentation for more information.)

That same thought process applies to much of Drupal 8: “This is going to be really hard to do, but someone already did it. Let’s just save time and use theirs. Open Source is cool like that.”

Doctrine Annotations

Doctrine is a large project. It's most known for its database abstraction layer (DBAL) – for which Drupal already has "DBTNG" – and for the Doctrine ORM object-relational-mapper – for which the Entity and Field system already serves Drupal well, even if it doesn't have as cool-sounding a name. So what's Doctrine doing in Drupal?

The new plugin system in Drupal 8 makes use of "annotations". Annotations are a way to define special Docblock tags that can be parsed at runtime to provide metadata for a class or method. Essentially, they serve a similar purpose to "info hooks" in previous versions of Drupal, but keep that metadata right next to the class they describe. Of course, parsing those annotations into useful information takes work. (Some languages have native annotation support, but PHP does not; we have to rely on the docblock.)

As with Guzzle, why do that work when it's already been done? Doctrine's flavor of annotations is one of the more commonly used in PHP, so we adopted that and its annotations library. We even managed to submit some work back upstream to improve the efficiency of its parsing engine.

Easy RDF

The story with Easy RDF is much the same. Managing RDF graphs is complicated: It's better for there to be a few really good libraries for it than lots of mediocre ones. So we just adopted an existing one that worked. (Notice a pattern emerging?)

Zend Feed

“Zend? I thought Drupal was using Symfony!”
Open source isn't partisan. As with many other libraries, Drupal had an old and half-implemented RSS parser in the Aggregator module that left much to be desired. (Actually we had two; Views has an RSS generating routine.)

The most robust RSS and Atom parser right now in PHP is the Feed library out of Zend Framework 2. After some discussion with the Zend Framework maintainers, they were able to remove a few dependencies from it, which made it small enough for Drupal to leverage. Out with the Aggregator RSS parser, in with Zend Feed. As a bonus, although core isn't using it, we now have a full-featured Atom parser ready to go for any module that wants to use it. As of this writing Views isn't leveraging it yet, but there's an open issue to do so. (Volunteers welcome.)

Twig

The Twig template is worthy of its own article (see Morten DK’s article in this issue). Its genesis was – you guessed it – much the same.

By the time of DrupalCon Denver, in early 2012, most core developers had concluded that PHPTemplate was no longer viable and needed to be put out to pasture. We needed something to replace it and, as we were already leveraging Symfony by that point, “let's look elsewhere” was a viable strategy. What really sold core developers on Twig was simple: Front-end developers demanded it. Twig offers them a far nicer experience than PHPtemplate ever did, so core developers went “Okay, you want it, you got it!” That said, moving Drupal to Twig has taken a sizeable army of front-end developers, many working in core for the first time.

Assetic

Assetic is an asset management library; that is, it helps manage CSS and Javascript files. As is the trend, it is replacing much of Drupal's home-grown CSS/JS compression and aggregation logic. Work is still happening in this area, and it's highly unlikely that module developers will ever deal with it directly, but it's there.

PHPUnit

PHPUnit is the industry standard testing framework for PHP. Although it has not fully replaced Drupal's home-grown testing framework yet, it is slowly supplanting it. For most code written for Drupal 8, if it cannot be tested with PHPUnit, then your code is flawed. (See Sam Boyer’s article in this issue, PHPUnit and Drupal.)

PSR Logger

The tiny Psr\Log library is just a collection of interfaces released by the Framework Interoperability Group (PHP-FIG) to standardize logging. As of this writing, it is only included because it's a dependency for some Symfony components, but there is active work to replace the watchdog() function with a new logger that uses the same standard interface. (With due apologies to the editors.)

Gliph

Gliph is an interesting case. It was written by Drupal developer Sam Boyer to solve a Drupal problem, but there is nothing Drupal-specific about it. Sam simply decided to build it outside of Drupal as an MIT-licensed library that Drupal, or anyone else, could then import. That's an approach that I expect will become increasingly popular in coming years, both for core and contrib. Gliph is a graph management library, that is, mathematical graphs such as dependency trees. It will be used to complement Assetic, and again it's unlikely that module developers will ever use it directly. (But if you need dependency resolution logic, it's there – go for it!)

Symfony

Last but not least, there's Symfony. Drupal 8 is not using all of Symfony by any means; in fact, we're using less than a third of the component libraries it offers and none of the fullstack framework. Nonetheless, it shares the same core pipeline with many other projects in the Symfony family. Many of these can and do have their own articles, so for now we'll just give a cursory review of them.

HttpFoundation / HttpKernel

These are the libraries that started it all. HttpFoundation was the first significant 3rd party library added to Drupal 8, followed soon after by HttpKernel. HttpFoundation abstracts the HTTP Request and Response concepts, replacing PHP's native superglobals and "print, but hope you don't have cookies" mess. HttpKernel essentially provides an interface for mapping a request to a response; a simple concept, it's actually fundamental to what any web application does. It also includes many powerful standard implementations that Drupal is leveraging, including the default HttpKernel itself.

Once again, the need was to replace Drupal's page-only routing system with a pipeline that could handle the full power of HTTP, and do so with a more self-documenting API. After designing one in the abstract, the Web Services Initiative found that Symfony had already implemented essentially what we had concluded we needed.
Open Source – For The Win!

Routing / CMF Routing

Routing is the process of mapping an incoming request to the code that will handle that particular request. In Drupal 7, it was hook_menu, menu_get_item(), and page callbacks. In Drupal 8, it's Symfony's Routing component with enhancements from the Symfony CMF project.

The Symfony CMF Routing component was actually a close collaboration between Drupal and Symfony CMF; despite nominally being competitors, both projects saw the value in working together to build one really solid routing framework.

EventDispatcher

The HttpKernel library makes use of the Symfony EventDispatcher library as well. Events are, essentially, object-oriented testable hooks. The low-level parts of Drupal 8 are using events, while many older systems are still using hooks. For Drupal 8 contributed modules, it's a good idea to focus on using events (as well as plugins) over hooks in most cases. There is serious talk of removing hooks as redundant in Drupal 9, so get a head start on that transition (and make your code more testable to boot).

DependencyInjection

The last big library is the Dependency Injection component. It provides the Dependency Injection Container, or simply "container", that ties all of Drupal's loosely coupled libraries (both 3rd party and home grown) together into a cohesive system. Almost all developers will be interacting with it, but mostly through a services.yml file rather than dealing with its low-level APIs directly.

Serializer

The Serializer component is a simple framework for managing the serialization and deserialization of objects to various string formats. Drupal 8 is using it as part of the REST framework, as it provides a common way to convert Entities to and from different formats like JSON-HAL (our default), XML, etc. If you want to support a new format for Entities (say, JSON-LD or Collection or some XML format), you'd write new services for the Serializer, and the rest would wire itself up automatically.

Validator

This little library helps structure validation of data rules. It is used deep within the Plugin and Entity systems, and most module developers won't be interacting with it directly.

YAML

Finally there is YAML. YAML is a text file format that Drupal 8 is using for many configuration files. Symfony has a YAML parser, we needed one, you know the drill by now – Open Source FTW.

Reuse All the Things!

All of that is found in one simple directory. That's the advantage of decoupled libraries and easy sharing between projects: Do less work, reuse more code, get more done faster.

That's the power of /vendor.

Image: ©profotokris/123RF.COM

Comments

The "guzzle documentation" link is dead. I'm still very curious to what this feature is about, but I can't find anything in any documentation. Where can I read more about it?

When you say that Guzzle has "the ability to auto-map RESTful services to PHP classes", are you referring to service description responseClass operations? http://guzzle3.readthedocs.org/webservice-client/guzzle-service-descript...