Coding by Magic

Most of my programming ventures begin with a single line of code I’m curious about. Who is calling the code, and how does this code fit into the larger jigsaw puzzle?

As a result, “find usages”, “go to declaration”, and “go to implementations” are three of my most often used IDE features. I use them to jump between caller and callee, interface and implementation, functions and the tests that verify them. I develop a mind map, tracing the possible logical routes through the codebase.

There are, unfortunately, ways to obfuscate these connections:

• Invoking functions or retrieving variables indirectly by name (e.g. reflective programming) makes it difficult to find the caller.
• Coding by convention uses implicit connections between components (e.g., “foo.xml uses the logic in foo.ts because they are named the same.”)
• App behavior controlled by global state that changes at mysterious times (one part of the app flips a global boolean that is used by an entirely different part of the app much later).

I colloquially call these obfuscations “coding by magic.” With magic tricks, seemingly impossible things happen before your eyes. You know magic isn’t real, so there’s got to be a logical explanation for what just happened - but it’s hard to figure it out!

It can be a nightmare when you have to interact with magic code:

• My tools (like “find usages”) no longer work, so I often have to fire up the debugger just to find out how a function is being invoked.
• It makes refactoring dangerous - you never know if that function you just changed might be invoked from a place you weren’t expecting (resulting in magical runtime exceptions).
• Constraints make otherwise complex codebases manageable. Magic code often breaks these constraints and you’re more likely you are to end up in a spaghetti code situation.

Sometimes the only answer is to code by magic (e.g., a plugin system requires dynamic code invocation). But, generally, I avoid coding by magic because it makes understanding codebases more difficult.

Usually we use code by magic because it adds a convenience: e.g., instead of having to write out a list of API endpoint modules, why not just look for all modules with the name *_api.py? But in the long run, I find that these short-term wins turn into long-term losses, where the added complexity of magic code bogs down your ability to maintain the codebase.

The most effective way to avoid coding by magic is to embrace explicit connections in coding, even if it is a bit more laborious (at first). The more a codebase grows, the harder it is to understand, and at that point you’ll take whatever blessings you can get.