Enterprise Abstraction

Open Library Environment (OLE) Project - Regional Design Workshop

There will be a regional design workshop for the Open Library Environment (OLE) Project held in Ottawa January 14/15 at Library and Archives Canada.  I've been asked to give a short presentation on SOA - and I encourage anybody in range to come and participate.  Or, to participate in a workshop closer to your area if Ottawa isn't viable for you.  It's an exciting project:

"With support from The Andrew W. Mellon Foundation, a multinational group of libraries is developing the design for an Open Library Environment (OLE), an alternative to the current model of an Integrated Library System. The goal is to produce a design document to inform open source library system development efforts, to guide future library system implementations, and to influence current Integrated Library System vendor products."  -- http://oleproject.org

The project is based on a firm SOA footing - and has large multinational support.

The workshop is aimed primarily at research libraries, and will be dealing with "back-end" ILS functionality (not discovery).  Participation is encouraged by people who work with an ILS day-to-day, and also by those with theoretical, technical or administrative interests.  Registration is available for the Ottawa workshop here.

CISTI SRU API

As part of our ongoing effort to provide new experimental tools on CISTI Lab, I've recently made available an SRU based  API to CISTI that was developed by Chris Fenwick - who has since left CISTI for new pastures.

The API is built upon our base SOA webservices used to - amongst other things - to power CISTI Discover.  Essentially, the API provides REST-based access to CISTI's article-level metadata databases.  Being CISTI - these databases mostly contain data on Scientific, Technical and Medical articles primarily.

Chris made an awesome effort in a short period of time to make the service compliant with the SRU standard - but we're not making any guarantees as to its accuracy or completeness.

Give it a try if you're so inclined.  Note that it's experimental - and could move/change in future.

My First Paper - Implementing SOA at CISTI

I'm pleased to announce the publication of my first paper.  Essentially it's a discussion of my experience with the EA and SOA implementation process at CISTI over these past few years.  From the introduction:

As they seek new roles in the digital realm, libraries are finding it increasingly difficult to manage the complexity of technology implementation while continuing to cost-effectively meet their mandates as information providers. Many organizations find themselves dealing with legacy, isolated, duplicated and ineffective information systems. The practices of enterprise architecture and service oriented architecture hold much promise as methodologies to reduce complexity, to encourage and enable collaborations, and finally to rein in the beast of technology. Even libraries under budgetary constraints can benefit from knowledge of enterprise architecture and service oriented architecture best practices.

Anthony, Stephen K. "Implementing Service Oriented Architecture at the Canada Institute for Scientific and Technical Information." The Serials Librarian, 55(01-02), pp. 235 - 253.
DOI: 10.1080/03615260801970907

I invite you to view the pre-print version (PDF)

There was a much more involved than I first imagined, and it took much more time than I expected.  However, it's done, and it's in print - and I'm doing the happy dance.  I hope it will be of interest to members of the library community who are starting service oriented related initiatives, are interested in CISTI's activities in this area, or who just want to know more about the area of SOA in libraries generally.

I want to convey my eternal thanks to Karen Morgenroth, Jim Cole the Editor-in-Chief, Daping Tan, Richard Akerman, Jane Carbone, and my wife Tammy for helping me with this project, and for putting up with me generally.

NPArC - NRC Publications Archive

As noted by Richard, and amplified by MichaelGeist and Peter Suber - The National Research Council (NRC) of Canada's Senior Executive Committee (SEC) has mandated that effective January 2009, all deposit copies of all peer-reviewed publications (articles, proceedings, books, book chapters) and technical reports produced by NRC will require deposit in the NRC Publication Archive (known as NPArC). 

CISTI has produced a press release providing additional details including some areas of potential exemption: 

Wherever possible, NPArC will provide access to the full text of these publications. NRC's License to Publish (Crown Copyright) will be updated to declare its intent to deposit the full-text of NRC-authored publications in NPArC. However, the nature, timing and extent of access to individual publications depends on a variety of factors, including agreements with publishers, or in the case of technical reports the sensitivity or confidentiality of content.

As the architect for the NPArC project, I'm proud to see some movement forward by NRC on the difficult legal and policy issues for this initiative.  The technology is one thing, but as has been demonstrated time and again, the true hurdles with institutional repositories are less technical, and more human in origin.

That said... just a bit of the technology/architecture:  The NPArC project is intending to piggy-back on our ongoing Trusted Digital Repository (TDR) project that CISTI has been working on for the past while.  The TDR is, among other things, CISTI's solution to moving forward with SOA-based article-level content and metadata management.  The TDR - based broadly on the OAIS reference model - is intended to handle tens of millions of bibliographic records and articles - and is planned to be CISTI's primary article-level storage and management infrastructure.  It's much more than NPArC itself needs - but it's planned that TDR will be supporting a number of other CISTI offerings and services as well.

 

SOA - IIT Colloquium - Ottawa

The NRC's Institute for Information Technology is sponsoring a colloquium 10:20-12:00 November 15, 2007 entitled:  Migrating to Service-Oriented Architectures by Patricia Oberndorf of the Software Engineering Institute.  It's free, but registration is required.

Coarse grain of SOA

Terry Coatta explains in his article From Here to There, The SOA Way how SOA isn't a silver bullet when it comes to resolving the issues of distributed systems.

I often come across the myth that web services (or as we often say, SOA services) are slower than more traditional architectures.  Terry explains that (properly implemented) SOA resolves this issue through having coarse grained services.  Essentially, you have to consider the data, bandwidth and latency of a particular issue first.  For example instead of making multiple service calls (say, to retrieve each data component from an object independently) you should probably return all the data in a single call. 

Objects are still a very good way to model systems and they function reasonably efficiently in the local context. But they don't distribute well, particularly if one tries to use them in a naive way. A service-oriented architecture solves this problem by dealing with the latency issues up front. It does this by looking at the patterns of data access in a system and designing the service-layer interfaces to aggregate data in such a way as to optimize bandwidth, usage, and latency.

I've seen several novice attempts to "serviceify" objects directly -- essentially making API's directly to object interfaces -- and I can confirm that they don't work well at all.  This is one of the primary reasons that I feel that web services are best governed by SOA, and not haphazardly created.  That's not to say that such governance need be bureaucratic... but better to spend a little time scoping your services than to create poorly performing or unsustainable ones.

I agree with Terry's assertion that there's nothing fundamentally new about service-oriented architectures, and that "distributed computing has always been about the same set of problems".  Really, success of SOA is dependent following a good set of best practices about service creation.  It really is a different way of thinking about problems than most people are used to.

OCLC Developer Network

OCLC plans to launch a new developer network and WorldCat Grid early in the new year.  Essentially it seems that it's geared toward providing and developing new library web services/API's, toolkits, resolvers and registries and a network of library technology developers.

Here's hoping for success!  And hopefully for some SOA basis behind the API's. 

- Via Richard Akerman, via Bess

CISTI Lab

Many months back we'd started CISTI Lab, a website for CISTI developers to expose some beta applications and to illustrate some of the more experimental work that's not yet ready for prime time, but that could use some exposure and a few more eyes.

The Lab has recently been revamped, and now includes some of the work being done by the CISTI Research group, as well as a wiki that explains some of the work.  As time goes on we hope to be adding some new additions, hopefully including some SOA-based architecture and related services, tools and applications.  For one, I'm hoping to revamp or replace the CISTI Toolbar application I wrote with something a bit better... most likely LibX based.

From the CISTI Lab site:

CISTI Lab visitors and collaborators will be able to test and evaluate Web-based experimental applications for science libraries. It is a place for CISTI to demonstrate prototypes, collaborate with researchers within NRC as well as Universities, libraries and the private sector and to obtain feedback from early adopters.

CISTI Lab has at its disposal a significant collection of electronic documents and meta-data about these documents as well as a collection of software tools and APIs for building Web applications and Web Services.

From an architecture perspective, it's a place that we hope we can use to help prove architecture and technology concepts, expose some experimental web services, and to encourage innovation in the area of libraries and technology.  More generally, CISTI is hoping to encourage collaboration and interest from like minded individuals and organizations.

Libraries as (web) service providers

Richard pointed out a recent article in the Globe entitled Unlocking the Potential of SOA that got me thinking.  The article points out that that "about 40 percent of Canadian companies have started investing in SOA", at least in some fashion, but that SOA has yet failed to catch on strongly because "vendors have found it difficult to get across to business managers why SOA was important".

Now, depending on how you read the words Vendors and SOA, I suspect the issue may actually be that SOA vendors have had difficultly selling their 'solutions' to companies... and perhaps with good reason.  That said, I feel that there remains a huge gap between technologists and business management with regard to the potential of services as valuable business products in themselves.

In my own experience, it has been reasonably easy (or at least possible) to convince people that SOA is good, and that it has advantages as a technology implementation strategy.  What is more difficult is explaining the ways that SOA and services can change how we think about doing business, and the ways that business is done.

I'm not saying that businesses should neglect their web interfaces, but I do advocate considering the possibility that simply exposing services has potential as products of their own.  These services could be for free use, pay-per-use, or through a subscription for access model (just as with many websites).

Too often I've seen business managers confused by the difference between a web site and a (web) service.  Essentially, the difference boils down to:  A web site is an appropriate interface for humans, whereas a (web) service is an appropriate interface for computers/applications.

If you build an infrastructure based on Service Oriented Architecture and the build your web sites on top of those services you've worked towards improving the agility, maintenance and sustainability of your technical infrastructure.  If you also expose those underlying services, then you have an additional (essentially free) new business opportunity.  Sell the services you've already built, unmediated by a web site!

Libraries in particular have a great opportunity to share service access to provide better services for their patrons.  If every library offered an SRU service, and select libraries offered speciality services (things like metadata conversion, XISBN, Book Cover Images for example) larger institutions could benefit from increased revenues for charging for service access, while smaller libraries could benefit from less dependency in large vendor 'solutions' or infrastructure in order to provide simple abilities for
their patrons.

Another aspect is the fact that patrons don't consider the library (nor the libraries website) the first (nor second, nor third) place to go when seeking information.  To continue to provide value for patrons, libraries must interact with them wherever they are on the Internet... library services must be accessible in the medium of the users choice. Web portals can never achieve this goal because they still assume that patrons will (virtually) come to the library.  Karen at Library Web Chic makes this point clearly:

meeting your users where they are isn’t about making them come to the library website. In considering our long term virtual presence plans, the library website is a given. People who come to the site know we exist and want to use our services. To truly be successful we have to get our content into the path of the people who wouldn’t walk through our door (physical or virtual).

We couldn’t possibly begin to do this with our old site because of its static architecture. Long term I’d like a site which has a series of web services that can be exploited by my developers but also my the university web developers and who knows who else.

Who knows who else indeed.

Of course, the vision of a service-exposed world is perhaps naive or idealistic... but I think it's an essential potential value of SOA that is generally less understood and less emphasized.  And for the record, I do lean towards the attitude that libraries should expose such services freely where possible... but I do also recognize that financial sustainability is likely an important aspect and incentive.

Greg the Architect - Off the Grid

Greg the Architect has a new episode entitled Off the Grid.  I love this thing, though I imagine there's a fairly limited audience for the material.  I guess it fits with my fondness for the South Park and Robot Chicken genre, but it's even a little closer to home due to the subject matter.

Those darn business analysts... I knew it... I knew it all along!