How often have you heard the cry in your group “we need a database”?
We all struggle with the increasing problem of “68 databases“, a phrase that arose from the challenges faced by our own department. Our department head was bemoaning the fact that we had 68 databases in our mid-size department, most of which were completely isolated. “What we need is a single database to pull all this information together”.
The problem with this is that no single system is going to fit all situations. Of course, large technology corporations benefit hugely from the idea of a single coherent ecosystem. Apple (or Microsoft or …) makes huge profits off making it easy for consumers to stay within the Apple vision.
“Yes, of course, all such systems should use a common format,” they say. “And since we are the important [system], everybody should use ours.”
This has been the large driver behind open-data formats and open-source code. Avoiding the problem of being locked into any single vendor’s clutches. Such vendors are aware of this and profess openness in their standards, yet few actually adhere to these. For anyone who doubts this, just look at the most glaring nonsense over the last decade with Microsoft Internet Explorer which, while being the most commonly used web-browser, is commonly held to be the least compatible with established web standards.
Back in 2011, our department set about tackling this challenge with the LACES Project: Linking Academic Clinical & Education Systems.
Rather than trying to shoehorn all the needed functions and services into a single monolithic database or application, LACES attempted to find ways to couple the major systems that we worked with, and to provide something that smaller applications could link into.
The project was successful in some ways, delineating the challenges faced by even a mid-size department, and making some useful suggestions about how a loosely coupled approach, based on APIs, was more likely to be successful, than the older approach of tightly coupled data sharing (where small changes in any one system tend to break the linkages). The report, which has some usefully generalizable findings, is available here.
However, the project ran into a number of challenges. The biggest was the sheer scope of our problem. What was thought to be fixable with a reasonable cost and effort, quickly ballooned because of the tendency for people to want everything included and for this to solve all problems (again trending towards the attitude of “a single database”).
This has been a recurring and ironic theme for many of the groups we have worked with over the past decade, such that it prompted us to illustrate these ironies in a presentation at the AMEE Fringe in 2011 (Vienna, Austria), poking fun at some of assumptions made. The themes arising very much struck a chord with the audiences there.
The recommendations in the report, which describe the importance of loosely coupled systems, using approaches such as APIs, have been echoed in the general trends in information management. We are moving above from static shared objects (such as in Open Education Resources, OERs) towards being able to examine what our learners/customers actually do, with activity streams.