Code vs Low-code 2023-05-05

2023-05-05 R, python, git, low-code

I was forwarded a link recently to some marketing for a Microsoft low-code platform. I thought I'd explore my thoughts on it here, in the context of analytical work.

TL;DR

I believe open-soure code (R, python, git, and SQL) is the best way to deliver vibrant, fast, collaborative multi-organisaton analytics. The NHS in particular is moving towards inter-organisation collaboration, and I believe that the NHS's need to standardise on open-source methods is particularly important if analysts are to make a significant positive difference for patients.

Low-code is an easy sell

In many ways low-code sounds attractive. A way to deliver output without needing to learn to code. I understand that. My main programming languages over time have been fortran, matlab, php, JavaScript, and R. All of them in their own ways are horribly ugly to look at and at times pretty unfriendly to use. The bar to entry is as low as it can be, and lowering all the time, but it's still not low. You have to REALLY WANT to code, and all programmers depend heavily on lifelines like google, stackoverflow, and nowadays chatGPT. So a shortcut to the results without needing to know all that stuff sounds attractive.

I don't have a neutral view here. I'm going to explore the hidden costs of low-code. These opinions are formed from years of dealing with both code and low-code platforms. If you have a different viewpoint, let's chat on twitter or mastodon. My mind is open, but my experience is lived.

What is the alternative interface

Giving a computer instructions on what you want done is traditionally done with code. If you want to remove or minimise that method, the instructions need to be transferred by some other method. Short of punch-cards, or thinking your work into existence, the only real alternative is to replace keyboard keystrokes with mouse clicks. You'll be selecting menu items, dragging blocks onto the screen and moving arrows around to connect them up.

While it's true that you're no longer coding, you're still in the chair and heavily involved. But the mouse clicks you are making are not reproducible. How do you describe what has been built? How do you communicate or transfer the work to a colleague?

Written in code, your work documents itself. It describes line by line how to achieve the result you want. But when clicked into existence, your clicks are momentary. Your work is stored as a cumulative "state" based on the sequence of clicks, as well as where on the screen and in what order you made them. Sharing that state is limited to the methods your platform gives, because the state is almost certainly stored in a proprietary format. Even if you can access or download it (probably as JSON, or another similar object), the only place you can load it is into your vendor's platform. Can you share your work with others to collaborate, or is the only option to hand over ownership of the work? I'm looking at you, QlikSense.

Sharing and collaboration

When choosing a low-code solution, if you want to be able to collaborate with others outside your organisation you'll need to make sure they're using the same platform. Do you know what platform they use? Have they even made the decision yet? Can you influence the decision? Do you even know who your future collaborators are going to be? One thing is certain - if they choose a different platform your collabration options will be seriously limited. Even if they are using the same platform, your systems may not be configured to communicate or inter-operate. Often there is a system admin who could theoretically unlock a collaboration with a single mouse-click, but they are not allowed to, because the platform doesn't differentiate between application state and data. In these cases sharing analytical method without also sharing the precious data is impossible.

That box where you put the code

The most powerful part of low-code is that box in the UI where you type the code. It's so true.

There is always a box in low code platforms where you can type code. You lose some screen real-estate because of the vendor's logo in the top left, navigation in the top right, and the big "update" or "save" button at the bottom. So it's a little like working through a letterbox, but hey, better than nothing right?

At this point the platform will flatter you by referring to you as a "power user", but it's important not to be fooled. You have reached the limit of the low-code platform, but to avoid you having to tell your boss that the work you need to do is impossible in the platform, they will allow you to code something up. From this point on, you're coding, but now you're doing it in a vendor-specific and non-transferable language. You're now writing your own handcuffs.

Some systems even have multiple internal languages to learn. If you want to get beyond intermediate level with PowerBI, you're going to have to learn both "DAX" and "M", which are different but confusingly inter-related. In Qliksense you'll need to know the proprietary script language (and some JavaScript once you hit the limitations of the out of the box visualisations).

What's more, as you become more familiar with the locked-in language you're now using, you will spot opportunities to reduce some of the earlier mouse-clicks you made. That data load you did by dragging a file into the UI? You can script that too, and now you've learned to code you'll probably find it faster and more convenient than doing it with the mouse.

You have come full circle. You know how to code. But the language you have ended up learning is only useful in your specific organisation, for as long as the platform in question is maintained and paid for by the powers that be. A tiny proportion of future employers in your business domain use the same platform that you are now expert in. Your expertise is not valuable to most other employers out there, unless you broaden your search by changing domain (eg. from healthcare to website sales analytics, or a/b testing), which of course has an experience cost too.

Similarly, the organisation you work for struggles to hire these niche skills, so the onboarding time is long, and productivity is low because of the time spent training (or worse, not training, and allowing new hires to teach themselves).

Positives?

Well, I guess you have to write less code. If you're working in a domain which is fairly standardised (financial accounting?), and which needs little inter-organisation collaboration, then low code can fit. When you can define and control the whole ecosystem, then you can choose the one that best fits and standardise it everywhere. But when you don't have control of the whole ecosystem (for example in NHS healthcare analytics), "future you" may not be happy with the constraints they find themselves working with.

If

If you're going to go down the route of low-code for analytics:

  • Know about what you are trading off in return for a gentle on-ramp.
  • Know what the ceiling is, and when you're likely to reach it. How long before you're doing most of your work in the "power user" box?
  • Have an exit plan. Whether that is the closure of the temporary project, or some active re-engineering and (whisper it) coding, later.
  • Make a plan for training and keeping your people. You are about to ask them to specialise in some niche tools, and some may not be comfortable making that personal trade.

Wrap up

My advice to a healthcare analyst still deciding on their tools is:

  • If you're working with text analysis, or big(ish) data, say 100 million rows or more, python possibly through anaconda may be a good place to start.
  • If you're working with smaller data, and doing a wide variety of general-purpose light (or heavy) statistics work (eg. process automation and pipelines, combining many different data sources, and reporting which includes the need for flexible and creative visualisations), R probably with RStudio may be the workhorse you need.
  • Either way, you'll need git and a github, gitlab, or bitbucket account to collaborate through.
  • You're also going to need SQL, but assuming you're working in R or python, you will only need the simplest subset of the language. You will be able to lean on the power of your non-SQL code to avoid needing to learn the niggly bits.

So there we have it, my fairly opinionated, but hard-earned views on code vs. low-code. I'd really encourage any conversations about this on twitter / mastodon. Anything you think I have wrong?

Enjoy!