Enterprise Abstraction

NPArC - NRC Publications Archive

As noted by Richard, and amplified by MichaelGeist and Peter Suber - The National Research Council (NRC) of Canada's Senior Executive Committee (SEC) has mandated that effective January 2009, all deposit copies of all peer-reviewed publications (articles, proceedings, books, book chapters) and technical reports produced by NRC will require deposit in the NRC Publication Archive (known as NPArC). 

CISTI has produced a press release providing additional details including some areas of potential exemption: 

Wherever possible, NPArC will provide access to the full text of these publications. NRC's License to Publish (Crown Copyright) will be updated to declare its intent to deposit the full-text of NRC-authored publications in NPArC. However, the nature, timing and extent of access to individual publications depends on a variety of factors, including agreements with publishers, or in the case of technical reports the sensitivity or confidentiality of content.

As the architect for the NPArC project, I'm proud to see some movement forward by NRC on the difficult legal and policy issues for this initiative.  The technology is one thing, but as has been demonstrated time and again, the true hurdles with institutional repositories are less technical, and more human in origin.

That said... just a bit of the technology/architecture:  The NPArC project is intending to piggy-back on our ongoing Trusted Digital Repository (TDR) project that CISTI has been working on for the past while.  The TDR is, among other things, CISTI's solution to moving forward with SOA-based article-level content and metadata management.  The TDR - based broadly on the OAIS reference model - is intended to handle tens of millions of bibliographic records and articles - and is planned to be CISTI's primary article-level storage and management infrastructure.  It's much more than NPArC itself needs - but it's planned that TDR will be supporting a number of other CISTI offerings and services as well.

 

SOA - IIT Colloquium - Ottawa

The NRC's Institute for Information Technology is sponsoring a colloquium 10:20-12:00 November 15, 2007 entitled:  Migrating to Service-Oriented Architectures by Patricia Oberndorf of the Software Engineering Institute.  It's free, but registration is required.

Coarse grain of SOA

Terry Coatta explains in his article From Here to There, The SOA Way how SOA isn't a silver bullet when it comes to resolving the issues of distributed systems.

I often come across the myth that web services (or as we often say, SOA services) are slower than more traditional architectures.  Terry explains that (properly implemented) SOA resolves this issue through having coarse grained services.  Essentially, you have to consider the data, bandwidth and latency of a particular issue first.  For example instead of making multiple service calls (say, to retrieve each data component from an object independently) you should probably return all the data in a single call. 

Objects are still a very good way to model systems and they function reasonably efficiently in the local context. But they don't distribute well, particularly if one tries to use them in a naive way. A service-oriented architecture solves this problem by dealing with the latency issues up front. It does this by looking at the patterns of data access in a system and designing the service-layer interfaces to aggregate data in such a way as to optimize bandwidth, usage, and latency.

I've seen several novice attempts to "serviceify" objects directly -- essentially making API's directly to object interfaces -- and I can confirm that they don't work well at all.  This is one of the primary reasons that I feel that web services are best governed by SOA, and not haphazardly created.  That's not to say that such governance need be bureaucratic... but better to spend a little time scoping your services than to create poorly performing or unsustainable ones.

I agree with Terry's assertion that there's nothing fundamentally new about service-oriented architectures, and that "distributed computing has always been about the same set of problems".  Really, success of SOA is dependent following a good set of best practices about service creation.  It really is a different way of thinking about problems than most people are used to.

OCLC Developer Network

OCLC plans to launch a new developer network and WorldCat Grid early in the new year.  Essentially it seems that it's geared toward providing and developing new library web services/API's, toolkits, resolvers and registries and a network of library technology developers.

Here's hoping for success!  And hopefully for some SOA basis behind the API's. 

- Via Richard Akerman, via Bess

CISTI Lab

Many months back we'd started CISTI Lab, a website for CISTI developers to expose some beta applications and to illustrate some of the more experimental work that's not yet ready for prime time, but that could use some exposure and a few more eyes.

The Lab has recently been revamped, and now includes some of the work being done by the CISTI Research group, as well as a wiki that explains some of the work.  As time goes on we hope to be adding some new additions, hopefully including some SOA-based architecture and related services, tools and applications.  For one, I'm hoping to revamp or replace the CISTI Toolbar application I wrote with something a bit better... most likely LibX based.

From the CISTI Lab site:

CISTI Lab visitors and collaborators will be able to test and evaluate Web-based experimental applications for science libraries. It is a place for CISTI to demonstrate prototypes, collaborate with researchers within NRC as well as Universities, libraries and the private sector and to obtain feedback from early adopters.

CISTI Lab has at its disposal a significant collection of electronic documents and meta-data about these documents as well as a collection of software tools and APIs for building Web applications and Web Services.

From an architecture perspective, it's a place that we hope we can use to help prove architecture and technology concepts, expose some experimental web services, and to encourage innovation in the area of libraries and technology.  More generally, CISTI is hoping to encourage collaboration and interest from like minded individuals and organizations.

Libraries as (web) service providers

Richard pointed out a recent article in the Globe entitled Unlocking the Potential of SOA that got me thinking.  The article points out that that "about 40 percent of Canadian companies have started investing in SOA", at least in some fashion, but that SOA has yet failed to catch on strongly because "vendors have found it difficult to get across to business managers why SOA was important".

Now, depending on how you read the words Vendors and SOA, I suspect the issue may actually be that SOA vendors have had difficultly selling their 'solutions' to companies... and perhaps with good reason.  That said, I feel that there remains a huge gap between technologists and business management with regard to the potential of services as valuable business products in themselves.

In my own experience, it has been reasonably easy (or at least possible) to convince people that SOA is good, and that it has advantages as a technology implementation strategy.  What is more difficult is explaining the ways that SOA and services can change how we think about doing business, and the ways that business is done.

I'm not saying that businesses should neglect their web interfaces, but I do advocate considering the possibility that simply exposing services has potential as products of their own.  These services could be for free use, pay-per-use, or through a subscription for access model (just as with many websites).

Too often I've seen business managers confused by the difference between a web site and a (web) service.  Essentially, the difference boils down to:  A web site is an appropriate interface for humans, whereas a (web) service is an appropriate interface for computers/applications.

If you build an infrastructure based on Service Oriented Architecture and the build your web sites on top of those services you've worked towards improving the agility, maintenance and sustainability of your technical infrastructure.  If you also expose those underlying services, then you have an additional (essentially free) new business opportunity.  Sell the services you've already built, unmediated by a web site!

Libraries in particular have a great opportunity to share service access to provide better services for their patrons.  If every library offered an SRU service, and select libraries offered speciality services (things like metadata conversion, XISBN, Book Cover Images for example) larger institutions could benefit from increased revenues for charging for service access, while smaller libraries could benefit from less dependency in large vendor 'solutions' or infrastructure in order to provide simple abilities for
their patrons.

Another aspect is the fact that patrons don't consider the library (nor the libraries website) the first (nor second, nor third) place to go when seeking information.  To continue to provide value for patrons, libraries must interact with them wherever they are on the Internet... library services must be accessible in the medium of the users choice. Web portals can never achieve this goal because they still assume that patrons will (virtually) come to the library.  Karen at Library Web Chic makes this point clearly:

meeting your users where they are isn’t about making them come to the library website. In considering our long term virtual presence plans, the library website is a given. People who come to the site know we exist and want to use our services. To truly be successful we have to get our content into the path of the people who wouldn’t walk through our door (physical or virtual).

We couldn’t possibly begin to do this with our old site because of its static architecture. Long term I’d like a site which has a series of web services that can be exploited by my developers but also my the university web developers and who knows who else.

Who knows who else indeed.

Of course, the vision of a service-exposed world is perhaps naive or idealistic... but I think it's an essential potential value of SOA that is generally less understood and less emphasized.  And for the record, I do lean towards the attitude that libraries should expose such services freely where possible... but I do also recognize that financial sustainability is likely an important aspect and incentive.

Greg the Architect - Off the Grid

Greg the Architect has a new episode entitled Off the Grid.  I love this thing, though I imagine there's a fairly limited audience for the material.  I guess it fits with my fondness for the South Park and Robot Chicken genre, but it's even a little closer to home due to the subject matter.

Those darn business analysts... I knew it... I knew it all along!

Planet Library SOA

Peter Murray (aka "The Jester") has been kind enough to setup a new library SOA aggregation at Planet Library SOA.  So, you  growing masses of library architecture enthusiasts, please feel free to converge there and slurp from the fountain of library SOAness.  (Note that library in this context means that place with books and journals and stuff).

Job Posting - Technology Architect - IT

Interested in libraries and architecture?  Perhaps you'd be interested in the recent job posting for an architecture position at CISTI?  Despite the job title - Technology Architect - the position is really about enterprise architecture, SOA and the like. 

Yahoo Pipes

Recently I'd been asked to enable an RSS feed for my SOA category in TypePad.  TypePad allows for the provision of custom feeds, but only though the Advanced Template Set functionality that comes with a Pro account. 

Richard had done some thinking on this awhile back, and suggested using a Blogdigger hack to accomplish it.  While Blogdigger can get the job done, I was a little reticent to use that solution.  Firstly, who's Blogdigger, and when will they change their *ahem* API.  Secondly, the Blogdigger hack doesn't allow much control over how the results are presented, particularly the title of the resulting feed.

Since I'd been looking for an excuse to try out Yahoo Pipes, this seemed the opportune time.  In about 5 minutes I'd created a custom feed from my site that allows an SOA-category-only RSS feed for this site.  Unfortunately, there's a known bug in Pipes that prevents this from being used universally, but hopefully that issue will be fixed before too long.

I also made a more generic one that allows you to configure the category and feed URL as parameters (either via the Pipes site, or on the feed URL directly), so you can try it on your own site.  It should work on most TypePad sites (keeping the aforementioned bug issue in mind).  No warranties are expressed or implied. 

 

All of that to say, Pipes is and awesome example of what SOA and XML should be all about.  You should be able to mix, match and connect these things nearly effortlessly.