Drupal as a "fat" front-end to big datastores

In this talk we will be exploring how HighWire press at Stanford University uses Drupal as a "fat" front-end for a large pre-existing NoSQL datastore.

HighWire Press at Stanford University hosts over 6.5 million scholarly and scientific papers. At the core of this platform is a NoSQL XML datastore that contains over 20 million unique records in XML format.

This presentation will talk you through how we use Drupal to allow site-builders to quickly and easily create public facing websites, using standard Drupal tools like panels and views that make use of a large external datastore as it’s canonical content store.

Some topics include:

1. Modeling unstructured data in Drupal.

Drupal's field system is great for modeling data that have well-defined schemas and map to the object->key->value paradigm. So what are the strategies for working with unstructured, hierarchical data like JSON and XML?

2. Importing data into Drupal

Very early on in this process we realized that using Feeds wasn't going to provide us with enough power for mapping our non-relational, hierarchical data to Drupal's field system. We will talk you through how we manage the translation and how we keep constantly changing data up-to-date.

3. Connecting Drupal to external RESTful services

In addition to our XML datastore, HighWire makes extensive use of a constellation of RESTful services that provides things like access-control and search. We'll show you how we interact with these services and how panels and ctools help.