Enterprise Abstraction

CISTI SRU API

As part of our ongoing effort to provide new experimental tools on CISTI Lab, I've recently made available an SRU based  API to CISTI that was developed by Chris Fenwick - who has since left CISTI for new pastures.

The API is built upon our base SOA webservices used to - amongst other things - to power CISTI Discover.  Essentially, the API provides REST-based access to CISTI's article-level metadata databases.  Being CISTI - these databases mostly contain data on Scientific, Technical and Medical articles primarily.

Chris made an awesome effort in a short period of time to make the service compliant with the SRU standard - but we're not making any guarantees as to its accuracy or completeness.

Give it a try if you're so inclined.  Note that it's experimental - and could move/change in future.

My First Paper - Implementing SOA at CISTI

I'm pleased to announce the publication of my first paper.  Essentially it's a discussion of my experience with the EA and SOA implementation process at CISTI over these past few years.  From the introduction:

As they seek new roles in the digital realm, libraries are finding it increasingly difficult to manage the complexity of technology implementation while continuing to cost-effectively meet their mandates as information providers. Many organizations find themselves dealing with legacy, isolated, duplicated and ineffective information systems. The practices of enterprise architecture and service oriented architecture hold much promise as methodologies to reduce complexity, to encourage and enable collaborations, and finally to rein in the beast of technology. Even libraries under budgetary constraints can benefit from knowledge of enterprise architecture and service oriented architecture best practices.

Anthony, Stephen K. "Implementing Service Oriented Architecture at the Canada Institute for Scientific and Technical Information." The Serials Librarian, 55(01-02), pp. 235 - 253.
DOI: 10.1080/03615260801970907

I invite you to view the pre-print version (PDF)

There was a much more involved than I first imagined, and it took much more time than I expected.  However, it's done, and it's in print - and I'm doing the happy dance.  I hope it will be of interest to members of the library community who are starting service oriented related initiatives, are interested in CISTI's activities in this area, or who just want to know more about the area of SOA in libraries generally.

I want to convey my eternal thanks to Karen Morgenroth, Jim Cole the Editor-in-Chief, Daping Tan, Richard Akerman, Jane Carbone, and my wife Tammy for helping me with this project, and for putting up with me generally.

OCLC Developer Network

OCLC plans to launch a new developer network and WorldCat Grid early in the new year.  Essentially it seems that it's geared toward providing and developing new library web services/API's, toolkits, resolvers and registries and a network of library technology developers.

Here's hoping for success!  And hopefully for some SOA basis behind the API's. 

- Via Richard Akerman, via Bess

CISTI Lab

Many months back we'd started CISTI Lab, a website for CISTI developers to expose some beta applications and to illustrate some of the more experimental work that's not yet ready for prime time, but that could use some exposure and a few more eyes.

The Lab has recently been revamped, and now includes some of the work being done by the CISTI Research group, as well as a wiki that explains some of the work.  As time goes on we hope to be adding some new additions, hopefully including some SOA-based architecture and related services, tools and applications.  For one, I'm hoping to revamp or replace the CISTI Toolbar application I wrote with something a bit better... most likely LibX based.

From the CISTI Lab site:

CISTI Lab visitors and collaborators will be able to test and evaluate Web-based experimental applications for science libraries. It is a place for CISTI to demonstrate prototypes, collaborate with researchers within NRC as well as Universities, libraries and the private sector and to obtain feedback from early adopters.

CISTI Lab has at its disposal a significant collection of electronic documents and meta-data about these documents as well as a collection of software tools and APIs for building Web applications and Web Services.

From an architecture perspective, it's a place that we hope we can use to help prove architecture and technology concepts, expose some experimental web services, and to encourage innovation in the area of libraries and technology.  More generally, CISTI is hoping to encourage collaboration and interest from like minded individuals and organizations.

Libraries as (web) service providers

Richard pointed out a recent article in the Globe entitled Unlocking the Potential of SOA that got me thinking.  The article points out that that "about 40 percent of Canadian companies have started investing in SOA", at least in some fashion, but that SOA has yet failed to catch on strongly because "vendors have found it difficult to get across to business managers why SOA was important".

Now, depending on how you read the words Vendors and SOA, I suspect the issue may actually be that SOA vendors have had difficultly selling their 'solutions' to companies... and perhaps with good reason.  That said, I feel that there remains a huge gap between technologists and business management with regard to the potential of services as valuable business products in themselves.

In my own experience, it has been reasonably easy (or at least possible) to convince people that SOA is good, and that it has advantages as a technology implementation strategy.  What is more difficult is explaining the ways that SOA and services can change how we think about doing business, and the ways that business is done.

I'm not saying that businesses should neglect their web interfaces, but I do advocate considering the possibility that simply exposing services has potential as products of their own.  These services could be for free use, pay-per-use, or through a subscription for access model (just as with many websites).

Too often I've seen business managers confused by the difference between a web site and a (web) service.  Essentially, the difference boils down to:  A web site is an appropriate interface for humans, whereas a (web) service is an appropriate interface for computers/applications.

If you build an infrastructure based on Service Oriented Architecture and the build your web sites on top of those services you've worked towards improving the agility, maintenance and sustainability of your technical infrastructure.  If you also expose those underlying services, then you have an additional (essentially free) new business opportunity.  Sell the services you've already built, unmediated by a web site!

Libraries in particular have a great opportunity to share service access to provide better services for their patrons.  If every library offered an SRU service, and select libraries offered speciality services (things like metadata conversion, XISBN, Book Cover Images for example) larger institutions could benefit from increased revenues for charging for service access, while smaller libraries could benefit from less dependency in large vendor 'solutions' or infrastructure in order to provide simple abilities for
their patrons.

Another aspect is the fact that patrons don't consider the library (nor the libraries website) the first (nor second, nor third) place to go when seeking information.  To continue to provide value for patrons, libraries must interact with them wherever they are on the Internet... library services must be accessible in the medium of the users choice. Web portals can never achieve this goal because they still assume that patrons will (virtually) come to the library.  Karen at Library Web Chic makes this point clearly:

meeting your users where they are isn’t about making them come to the library website. In considering our long term virtual presence plans, the library website is a given. People who come to the site know we exist and want to use our services. To truly be successful we have to get our content into the path of the people who wouldn’t walk through our door (physical or virtual).

We couldn’t possibly begin to do this with our old site because of its static architecture. Long term I’d like a site which has a series of web services that can be exploited by my developers but also my the university web developers and who knows who else.

Who knows who else indeed.

Of course, the vision of a service-exposed world is perhaps naive or idealistic... but I think it's an essential potential value of SOA that is generally less understood and less emphasized.  And for the record, I do lean towards the attitude that libraries should expose such services freely where possible... but I do also recognize that financial sustainability is likely an important aspect and incentive.

Yahoo Pipes

Recently I'd been asked to enable an RSS feed for my SOA category in TypePad.  TypePad allows for the provision of custom feeds, but only though the Advanced Template Set functionality that comes with a Pro account. 

Richard had done some thinking on this awhile back, and suggested using a Blogdigger hack to accomplish it.  While Blogdigger can get the job done, I was a little reticent to use that solution.  Firstly, who's Blogdigger, and when will they change their *ahem* API.  Secondly, the Blogdigger hack doesn't allow much control over how the results are presented, particularly the title of the resulting feed.

Since I'd been looking for an excuse to try out Yahoo Pipes, this seemed the opportune time.  In about 5 minutes I'd created a custom feed from my site that allows an SOA-category-only RSS feed for this site.  Unfortunately, there's a known bug in Pipes that prevents this from being used universally, but hopefully that issue will be fixed before too long.

I also made a more generic one that allows you to configure the category and feed URL as parameters (either via the Pipes site, or on the feed URL directly), so you can try it on your own site.  It should work on most TypePad sites (keeping the aforementioned bug issue in mind).  No warranties are expressed or implied. 

 

All of that to say, Pipes is and awesome example of what SOA and XML should be all about.  You should be able to mix, match and connect these things nearly effortlessly.   

CISTI Pay Per Article

CISTI's Pay Per Article has just been released on the CISTI website.  Pay Per Article (get it?  Paper Article - even though, the product is really very electronic from the search to payment to the delivery of items) is a new offering from CISTI, that allows article level searching and ordering of documents from CISTI.  It is based on the idea that many clients, rather than being pre-registered and requiring lots of information, could simply order documents with a credit card. 

From an architecture perspective Pay Per Artcle is CISTI's first truly SOA-based application release.  The project has produced a lot of new web services (based on our architecture) that we hope to use, reuse and expose in the near future.

More information is available at the Pay Per Article site, or the Pay Per Article FAQ.

SOA Ideas for 2007

Dion Hinchcliffe has a very interesting post entitled Eleven Emerging Ideas for SOA Architects in 2007.  I really like Dion's list, particularly the idea of SOA being enabled more directly to the user.  He suggests that making services consumable in the browser (via REST or similar), making AJAX the face of your SOA, providing access to SOA services via widgets.  In other words, he's recommending starting to externalize SOA in different ways more directly to the end user.  This is an idea for which I'm very much in favour.

He also mentions the statistic that 48% of CIOs will be looking to actually start using their SOAs to connect to external partners this year.

This gives me hope.  We've spent the last 2-3 years building architecture and communicating SOA, followed by our first real SOA/EA based projects.  We're just now reaching a place where we'll have services available internally, and ready for the acid tests of reuse (let's see).  In my opinion, it's taken about 3 years for us to be poised to be seriously considering extending/exposing portions of our SOA in various ways.  The 48% stat makes me think we're in a good place on the curve, particularly for a small organization.

CISTI Widget

Some shameless self promotion.  In the recently released Fall 2006 issue of CISTI News there's a story about the CISTI Widget that I knocked up a few months back.  I've been meaning to get back to the thing to update it, and to do some more interesting work, but time is precious these days.

That said, feel free to check it out, steal it, fix it, improve upon it.  My real purpose was just to learn about the widget language, and produce something simple.  It is indeed simple. 

In version 2 I had done some work on moving the display of the results within the widget.  Whilst I did manage that, I never really returned to fix and polish it... so I've not made it available yet.  It was using David Walker's Shrew (SRU) service, and was (in a terribly hackish way) doing it's own parsing.  Perhaps time will move more slowly next year, or they will add another day in the week.

Second Life

I'm not sure how I could possibly have a Second Life when I have not a first.  Nonetheless I've been exploring recently just to see what this thing is all about.  My first reaction was to look for the things to shoot, or the puzzle to solve.  Before too long I came the the realization that such things didn't really exist here.

So, I'm poor, homeless... but I can fly and I change my appearance at a whim.  At least from those respects my second life is better than my first.  Six Linden dollars doesn't go far in world... perhaps I'll have to develop my building/scripting skills and start a business.

So, if it's not a game, then what is it?  Well, the best I can gather, it's a different way to interact with people.  Closer and more intimate than text chat/instant messaging, but distant enough to be comfortable, even for a (closet) introvert such as myself.  So going in expecting a game, my initial impression was disappointment, or perhaps more accurately boredom.  However, as a way to interact, meet, demonstrate as compared with text messaging, yea, I can see the appeal.

Colleagues of mine were informing me that with the new generation of kids today everything is IM or text messaging, as compared to the dinosaur email that I still tend to consider rather nouveau.  Looking on it as an improvement to text chat enlightened me a bit.  Yes, all of these people milling around chatting and not "doing anything" is exactly the point.  It's a meeting place, that happens to be 3D where you can explore and build stuff.  OK.  It's not a game.

So, why would a real life company/organization want a presence in Second Life?  Well, if you're a service oriented organization -- and I don't mean SOA here -- it may be a way to interact on a different, if initially weird, level.  This is why I think it appeals to libraries and librarians so much.  You can have the perfect, people-oriented perfect vision of a library where people stroll into your building and actually ask you questions.  It could also be a good place to have virtual meetings/conferences with staff and partners.  I think the main reason, currently, is just to be there.

The John has just posted a decent list of links for SL newbies.  And both Richard and John  -- the CISTI Second Life cabal if you will -- have posted on the recent news and happenings on SL.  The CBC news program The National did quite a long story on it yesterday.  Check out the links above for good links to other news stories.  Second Life has certainly made it to the mainstream media.  To me that's always an indicator that it's probably either boon or doom in the near future.

If you see me floating around SL, feel free to chat.  I'm homeless, and I have nothing to do anyway.  You'll find me under the moniker Gandalf Gumsing.

Yes, I look exactly like my avatar in real life.  I swear.

Update:  Peter Binkley has just posted some of his experiences and thoughts about Second Life and libraries.  He posits that SL may have positive implications as another arena for mashups based on open standards.  Along those lines I would suggest that it's an opportunity to expose SOA-based library web services. 

With regard to it being a timesink.  Probably.  Worth it?  Not sure yet, but I'm willing to explore a bit more.  Perhaps if we concentrate on making our library services available via open standards as suggested, the library services will find their way into Second Life, and other channels... and we will not have to worry as much about the actual end-use/implementation, as that can be achieved by those most interested in their respective channels.

UPDATE #2:  Richard supplied a link to the CBC new story text which includes links to the video.