For Good APIs, Legacy Systems Need an Attitude Adjustment

Why is opening up legacy systems using APIs still so hard?

As IT architects, we tend to focus on the technical reasons: data formats, protocols, messaging capabilities and so on.

I believe that we should also focus on another reason: our legacy systems have an attitude problem.

Do systems really have an attitude? Most of them can’t talk (yet), but imagine if they could. The batch, COBOL, mainframe systems I wrote in my first coding job might have sounded something like this:

"I don’t talk to other programs. I’m not even sure whether they really exist. All that I know is that once I day I wake up, I take that pile of data over there, I do some stuff to it, and I move it over there."

This simple, lonely life of a batch programme from thirty years ago might not seem relevant today. But those batch programmes grew up to be our big legacy systems, and now they run the world:

"I still take the data from that pile over there and move it to this pile over here. I’ve been doing that for thirty years, except now I do it once an hour rather than once a day. For some reason. I seem to do a lot of other things as well. I’ve got much bigger and much more complicated. I pick up and process lots of other piles of data: I’m good at that. If that was all I had to do, life might be easy. But I also get bombarded by all these messages that I’m expected to respond to immediately. I was never designed to do that. I do my best to cope. I make sure that I know exactly what every message means, and exactly who it comes from. I haven’t got any time for your ‘coarse grained business services’. Just give me a message that I know what to do with. And I know that lots of you want me to deal with lots more messages. Well, I’m very busy, so you’ll have to wait. You want a new message? Join the queue. You want to change an old message? Join the queue. You want to get rid of a message? Good luck with that: I just don’t have time."

Now imagine what a simple web service application built at the beginning of the e-commerce era might sound like:

"Hi! Pleased to see you! Have you come to connect with me? I don’t do very much right now. If I’m honest, I have to admit that I’m probably more interface than internals, but I’m ready to connect with anyone who’s interested. Here’s my contract. It’s a little bit hard to understand, I’m afraid. Lots of angle brackets and probably a bit more structure than we actually need. But we’ll simplify that over time. The important thing is to start working together. Oh, are you going? If you see anyone else, send them this way. It’s a bit quiet around here at the moment . . ."

And we know what happened when those simple web services grew up:

"Hi. Bear with me a minute. I’m a bit busy right now. There was a commercial break and another million customers came on line. Let me just . . . clone myself. There we go. Now, what did you need? Yes, I do a bit more these days than when I first started out, but I’ve tried to stay slim. The important thing was learning this cloning trick: now there’s a few of me when we only need a few, and thousands of me when things get frantic. Is that my old contract? Wow: I haven’t seen that in years. Here’s my new one: it’s a lot easier and simpler to understand, but it still does the same job: open access to feature, function and data. Haven’t had to change it much recently: people seem to like it."

This is fanciful, but it is also familiar. It’s not hard to imagine legacy systems as big beasts, chained in the dark, scary but dutiful. We know how difficult it is to manage those big beasts, and how much respect the teams that do so deserve. It’s also easy to imagine a web service starting as small, plucky, eager to connect (and a bit lonely in the days following the dotcom bubble) and growing to the digital scale we know today, having learnt the tricks of scaling and resilience along the way.

The differences are also familiar. Legacy systems are hard to connect to because they were born in the days before connectivity was common. Every connection is an imposition. Web services and digital systems are easier to connect to, and have better APIs, because when they were born they were nothing but connectivity. Every connection is a further reason to exist.

Of course systems aren’t really people. However, they are managed by teams made out of people, and those teams are changing the way they think about the world. The owners of legacy systems should continue to feel proud of the rich set of features, functions and data that they own, but can also embrace the idea that their main job is to make these features, functions and data available to the rest of the organisation, and ultimately to the rest of the world. New connections are not an imposition: they are a reason to exist.

That’s what we’re trying to do in HSBC Technology. A while ago we adopted the motto, ‘there’s no such thing as legacy systems; there’s only legacy thinking’. The teams that manage the big legacy beasts aren’t just trying to keep them under control, they’re getting them to open up. They’re measuring themselves by the APIs they’re making available to the world.

That's why I was pleased that the authors of the winning white paper at our recent HSBC Architecture Symposium in Pune (Sushanto Mukherjee, Sam Varghese, Bapurao Kshirsagar and Manish Shriwas) came from the Core Banking team, and described their approach to opening up legacy systems using APIs. They’ve found a technique to convert existing MQ interfaces into RESTful APIs, but they’re not stopping there: the important thing about their approach is that it gives them more time to work on truly opening up their systems.

Those of us who have been in the IT industry for a long time have had to reinvent our skills and attitudes many times: it’s important to remember that we have to help our systems do the same.