integrations
Why is it that integrating into another system can be so tedious? Sometimes, the integration endpoints are backed by vast financial support and professional care. Too often, though, they’re a mess that’s just been pieced together in a hurry.
Larger companies like Google, Twitter, etc., have their shit together concerning this. A substandard API would be a sign of either low effort or immaturity. Also, it would cause a mess within customer service. Public APIs are generally OK, and getting better. The problem lies within those private ones where the integration is implemented ad-hoc. Many of these APIs (that I’ve seen) are more like a nightmare of arcane old techs, crackling communications channels and pompous colleagues on the other end.
estimating #
Estimating the time required to implement an integration, even to a well documented system, can be difficult. There’s bound to be surprises and mistaken assumptions.
When the target system has no documentation, but merely a promise of “we’ll have technical readiness for an XML transfer by the end of October”. An optimist might think something along the lines “It’s not JSON but at least I can curl and parse that XML, and most likely it’s SOAP or some other standardised format. Ah, It’s done in a week.”
Come January, after a few months of emails back-and-forth, with three day latency, the integration is “ready”. The connection to the end-point needs some weird proprietary authentication setup, which takes a week to install. The XML file lies at some server and needs to be downloaded via FTP. The file looks standardised, but has another layer of non-standard XML encoded inside it. And the documentation “will be written once everything else is complete”.
In addition, there’s a communication lag, as the important of the integration project with you and the 3rd party is unlikely equal.
The horror stories are many, and all of them near impossible to estimate correctly.
customer service #
The lack of documentation is sometimes a kind of a elephant in the room. It’s easier to instruct the colleague sitting next to you, than a one several time zones away.
Examples are good for everything. Just describing something won’t do that much good, as it requires the reader to imagine something, and that’s where things go wrong. They’ll have to make assumptions, which can lead to incorrect estimations and general misery.
Proper communication with the users is important in here too. The developer generally doesn’t want to wait hours or days for a solution to some weird or unclear problem, as that also costs money. The speed of customer service is essential, but so is correctness. Fast and useless response is still useless.
An outsourced person following a predetermined script, with no proper technical understanding or even a sense of context, will only induce frustration. A developer (or some technical person) with total understanding but with a pompous attitude towards consumers themselves, or the tools they’re using, is even more infuriating.
repeating mistakes #
Working integrations can be achieved with ease, with developer’s singing each other’s praises and clients gaping in awe. When an API is sensible, the documentation is usually adequate, and no customer service is needed.
I’m afraid after experiencing both the good and the crazy scenarios, I still keep hoping for best and underestimate the possible problems. Estimation in general is known to be problematic, and the difficulty is emphasised when dealing with a third party. Writing the actual code often takes a fraction of the time required by communication, investigation and deciphering.
Though sometimes, a minimum viable integration needs to happen immediately. Data needs to be transferred as soon possible. Corner cases and errors are dealt with when the need arises. That’s OK. The madness begins when this mess is suddenly declared “done”, made public or sold as a product or feature. That was never supposed to happen.