Introduction
A lot of people don’t realize that a fair amount of the code you see on the Internet is written by amateurs.
Professionals are often too busy to be posting code, so a lot of what you see is amateurs asking questions of other amateurs, and responding to those questions. Or amateurs writing blogs about how they think things work, without having a clue about how they actually should be done. Or software packages or even programming languages written by amateurs.
For purposes of this article, we’re not going to consider a professional in terms of certification, or taking tests, or even receiving a salary.
For our purposes, professional means a person has learned how to ‘do it right’. They code reasonably quickly, rarely make mistakes, can write a surprising amount of code in a relatively short time (when that is appropriate), and can read/comprehend code written by other professionals in a surprisingly short time. They write code that is very readable to other professionals – and well documented. Their code can be understood at a high level from the contents of each file (including comments), without needing to understand the entire system to understand each file. They consider documentation to be poor if one must look at the source code to understand what a piece of code does, at least at a high level.
At some point in their career a professional coder has likely worked in a top notch software group, and they have learned from that experience about the differences between people who just ‘know how to code’ and people who ‘know how to code well’. In the vast majority of cases, moving from amateur status to professional status requires a mentoring experience where somebody that knows how to ‘do it right’ gives feedback, and an organization that systematically ensures that people learn from their mistakes – and we all make mistakes.
In effect, the difference between professional and amateur is often the difference between a ‘punk kid’ that thinks they know what they are doing, and somebody who really does.
But also recognize that many older employees who have coded for many years are still doing so on an amateur level: when comparing adults, the distinction between professional and amateur is not primarily based on age.
Code Style and Reading Efficiency
Why is understanding this important? One reason has to do with something called ‘coding standards’. There are people who are going to come up with ‘coding standards’ that they will try to enforce on others. Sometimes they will base these standards on things they see on the Internet.
But the problem that often occurs is that all too often well-meaning-but-misguided people are copying standards developed by amateurs, and trying to impose them on everybody else on their team. We have tools such Rubocop – a Ruby code checker – that can be used to impose code standards on people who are writing Ruby code. Similar tools exist for other languages. Often people will set these tools up to run when a commit or push is being made to a revision control system such as ‘git’.
But if the standards have elements that are not consistent with professional work, then these folks are actually requiring their teammates to write inferior code – and to work with poor efficiency. Worse, for the ‘new guys’ (and gals) they are being forced to learn bad habits. Down the road this will create all kinds of problems and ultimately hurts the bottom line of the organization.
One typical problem that we find when amateurs create coding standards involves having rules that force code to be tightly compressed on the left side of a page. A professional will prefer code that is aligned into columns and takes more width on a page.
You might wonder: Why is this a problem? Isn’t it purely arbitrary, simply a matter of using two different styles that are equal in all respects. The answer is NO!
The reason has to do with how human beings read. We want programmers to be good readers – this is extremely important for being efficient at maintaining and reviewing code, and thus a fundamental characteristic of the professional coder. An organization might be able to tolerate a few people in coding jobs that are poor readers, in special cases (perhaps as a form of ‘charity’), but no commercial organization can afford to have the majority of it’s programmers to be poor readers.
It is well known among folks that have studied how humans read – a subject within the broader field of ‘cognitive psychology’ that bad readers tend to have a dominant eye movement that is horizontal when they read, and they also spend a lot of time ‘sounding things out’ in their heads as they slowly proceed from left to right along each line, with lots of pauses.
Good readers, on the other hand, extensively use vertical scanning in their reading, and rarely ‘sound things out’. In the context of coding, a good reader will try to quickly scan a page vertically to find critical information that needs more focus, then they might read left-to-right on a few lines to ensure they fully understand what is happening on those lines.
Often kids learn this on their own – the kids that read a lot of books tend to pick up good reading habits unconsciously. But good reading technique can also be explicitly taught to kids – and it can be learned through disciplined practice even by adults.
Efficient vertical scanning requires information to be organized more in columns, and not heavily compressed into the left side of a page.
If we look at one well known ‘recommended style guide’ for Ruby, we find things like this:
# good (normal indent) <--- This comment is WRONG! This isn't good code!
def send_mail(source)
Mailer.deliver(
to: 'richard_roe@somewhere.com',
from: 'john_doe@somewhere.com',
subject: 'Important message',
body: source.text
)
endOnly an amateur programmer would consider this ‘good’, because it’s not aligned to support vertical scanning with the eyes. Think back to the old fashioned printed newspapers and magazines – they don’t have columns mixed together like this, because that makes reading efficiently really hard.
If a poet was equally careless in writing a poem, we might see something like this:
’Twas brillig, and the slithy toves "Beware the Jabberwock, my son! Did gyre and gimble in the wabe: The jaws that bite, the claws that catch! All mimsy were the borogoves, Beware the Jubjub bird, and shun And the mome raths outgrabe. The frumious Bandersnatch!”
Not that easy to read, even if you know this poem by heart!
Look at the difference in readability when the columns of the poem are lined up in a sensible way, like one might find in a newspaper article:
’Twas brillig, and the slithy toves "Beware the Jabberwock, my son! Did gyre and gimble in the wabe: The jaws that bite, the claws that catch! All mimsy were the borogoves, Beware the Jubjub bird, and shun And the mome raths outgrabe. The frumious Bandersnatch!”
We can think about code in much the same fashion, but the lesson here is even more applicable for code than it is for poetry because code is usually far more dense and complex than poetry – and the price of making mistakes is far higher. We all make mistakes, but the professional has work habits that help them make far fewer mistakes than the amateur. Writing code so it can be read efficiently ultimately leads to making fewer mistakes in addition to letting people work faster and more efficiently.
Keeping this in mind, a professional might write the above code as follows, in order to facilitate vertical scanning with the eyes:
# <comments not shown - that's a topic for another day>
def send_mail(source)
Mailer.deliver(
to: 'richard_roe@somewhere.com' ,
from: 'john_doe@somewhere.com' ,
subject: 'Important message' ,
body: source.text
)
endThe exact amount of whitespace used will differ from person to person, and some of the details of how this example is formatted might differ from one professional or one organization to another, and that’s fine. But anybody who codes at a professional level as defined here will organize information in multiple columns to support efficient reading of the code with vertical scanning by the eyes.
Some languages allow placement of whitespace before colons in situations like this, and that’s really nice. Ruby doesn’t – at present – but if it did the above example might look like this:
# <comments not shown - that's a topic for another day>
def send_mail(source)
Mailer.deliver(
to : 'richard_roe@somewhere.com' ,
from : 'john_doe@somewhere.com' ,
subject : 'Important message' ,
body : source.text
)
endArguably, this is even better – hence easier to read – than the previous example. The columns are more clearly delineated. Human beings that are good readers can easily separate out the text to the left and right of the colon from the colon itself as they scan vertically down the page. I’ll say more later about the surprisingly powerful capabilities human beings have for ‘pattern recognition’. On the whole, Ruby is a well designed language – better than most – but it does fall short in a few places.
Using two spaces of indent from line to line is problematic, because it tends to encourage people to ‘scrunch’ things together on the left, which is not desirable. When people already have bad habits, one of the last things we want to do is encourage them … When all is said and done, you can get away with two spaces – four is better but two is not terrible – but you definitely don’t want interleaving of columns as the initial example had.
Some of the guidelines in the previously referenced ‘Ruby Style Guide’ are good – but some of them are very bad and these items were clearly written by amateurs: they reflect the fact that amateurs don’t really understand the critical elements that go into being good at programming.
Again, this is not merely a matter choosing between two equivalent styles, a personal choice like picking a religion, it’s a question of supporting how the human eye-brain system can most efficiently work. If this isn’t clear, put your mouse on the floor and try to use your computer with your foot moving the mouse and your toes clicking the buttons for a few weeks. After that experience, would you try to force all the engineers on your team to only use the mouse with their foot?
Professional programmers are going to make the time to learn to be good readers, because this enormously improves their productivity. Amateur programmers simply won’t bother – and in most cases won’t understand the issues discussed here because nobody has ever told them any of this. Most universities require some classes outside of their computer science / technology / physical science / engineering curriculum, but most people in those fields tend to view those courses as ‘useless fluff’, avoid them whenever possible, and come out of those programs with very little knowledge of topics like cognitive psychology that are outside their primary field. This means they need to learn how to do it right ‘on the job’ because they didn’t learn how to do it right in school. The ‘publish-or-perish’ system makes a bad problem worse, because the vast majority of PhDs are themselves amateur programmers who have never learned how to ‘do it right’ and aren’t willing to spend the time to learn because the rewards in the system predominantly go to people who put their time into publication and not teaching. Law is far from the only area of modern society badly corrupted by systemic ethics problems.
All of that is unfortunate, but it’s common, and so organizations need to be prepared to handle this problem.
Code Style and Professional Thinking
Another difference we find between professionals and amateurs is that the amateurs don’t use information protecting mechanisms such as the ‘protected’ and ‘private’ keywords in Ruby.
Professional programmers understand that complexity is what makes writing and maintaining code difficult. They understand that one of the most important aspects of being a professional is knowing how to manage complexity.
When a professional programmer starts working on a program, they don’t just start coding.
Instead, they are going to think about ‘Where is the complexity in this problem?’ and ‘How do I manage it?’ They think in terms of data structures and algorithms and creating well documented APIs (Application Programming Interfaces) – and they use language keywords to separate interface from implementation and to hide complexity as part of managing complexity. Order of operations is critical: think first, code second. Any program more than a page or so in length – and many shorter programs – should be approached in these terms.
Language keywords such as ‘protected’ and ‘private’ support information hiding, and thus are a very powerful tool that can be used to help manage complexity. These keywords allow the designer of an API to keep the user of an API from having to understand all the details of how the API is implemented – and to a professional, almost everything they do will be done in terms of APIs because each API can be used to hide some of the complexity of the underlying code. If the API doesn’t exist, the professional will create it – developing it as they develop the program, and writing a lot of the documentation before they write the implementation. Thus, the user is shielded from complexity that in most cases they don’t need to understand – and when and if they need to understand it, they can peek ‘beneath the hood’.
Even an enormously complex system can be managed by the good API. For example, one might have a ‘developer mode’ that makes more advanced system features available – the average user doesn’t turn it on. The average user does not have to deal with the extra complexity found in the developer API.
Designing a good API means providing for novice users, for expert users, for legacy users, for new users, and everybody else.
Again, order of operations is critical: think about where the complexity lies in the problem and how you will manage it before you do anything else. Sometimes you’ll need to run some experiments before you can figure that out – it’s ok to code those experiments. These principles apply to both agile and non-agile programming, to a spiral model as well as a waterfall model. Think, anticipate, run though use cases and corners, do needed experiments, decide on an approach that will scale to handle the complexity that the problem will eventually present us, document the ideas – maybe not in full right now, but certainly in part – then, finally, we start the real coding.
This is a very different approach to programming than the amateur has: the amateur hacks things together until they get it to work, and if there is any documentation it’s written after the fact. There is little or no information hiding, very complex data structures are fully exposed, and often the entire design must be understood before one can understand any part of it. The amateur isn’t thinking about managing complexity – or if they are thinking about it, they don’t even come close to getting it right. In some cases, maybe they do ok in handling these issues with some relatively minor aspect or portion of the design, while leaving enormous amounts of complexity unmanaged in other aspects of the design – because getting the complexity management right wasn’t a priority and (unlike the professional), the amateur didn’t start by thinking about this aspect of the problem – or they completely misjudged the real source of the complexity in the problem.
Note that making a misjudgement of this sort shouldn’t be a fatal error, so long as the person learns the right lessons from their mistake. We all make mistakes. Sometimes the lessons that stick best start from a mistake.
The use of ‘protected’ and ‘private’ in code does not mean the author is a professional – but the absence of these keywords – in a language that has them – in any non-trivial program is almost certainly an indicator that the person writing the program was/is an amateur.
Some languages, such as Python, don’t even have these keywords – which makes Python, despite it’s current popularity, a language written by amateurs for amateurs. This doesn’t mean professionals can’t use it – it’s like a machine tool without a guard mechanism, you can still use it, but you have to be extra careful, both in your own work, and in how your organization uses the tool (assuming you can influence that – a topic called ‘Influence Tactics’ in Organizational and Industrial Psychology). Sometimes there are good reasons to use a language that has major problems, such as when the language provides access to good libraries. Even with an inferior language, the professional can and will still think about the problem in terms of managing complexity, and perhaps will hope that the language they use will eventually be fixed to better support professional work. If the language provides access to good libraries for solving a particular problem, then those libraries reduce the complexity of the programming task, so the limitations of the language become less significant though still problematic.
These are not new ideas – some programming languages started supporting information hiding decades ago. For any language today to not support information hiding mechanisms represents a major failure on the part of the language designers to understand the difference between amateur programming and professional programming, as defined here – and to competently research programming language design.
Re-Inventing the Wheel
Another characteristic of amateur programmers is they over-use third party libraries in situations where they should be writing their own code.
They like to tell people (the amateurs LOVE to tell other people what to do) that one should never ‘re-invent the wheel’ – find code from somebody else that has solved the problem and use that code.
Given that a lot of the code on the Internet is written by amateurs, one should expect that a lot of code from the Internet has serious problems. To use an analogy, what if the inventor of the wheel has only tested their wheels on concrete? A professional might wonder how that wheel performs on other surfaces, such as asphalt, or dirt, or sand, or in the presence of garden hoses or tree roots or rocks. A lot of amateur coding designs are the wheel that only works on concrete – the people writing the code don’t have enough vision and experience to realize that a wheel needs to do a lot more.
For example, you can find XML and YAML parsers that don’t easily give the user the line and column of errors – information that is critically important for debugging problems. Many compilers and parsers for languages with include constructs (or extensions to a language that have include constructs) can’t produce a ‘include stack’ that tells the user how they got where they are when something goes wrong. Think of this as being similar to a ‘call stack’ that one can get when one raises an exception in a programming language. The absence of the ‘include stack’ can make it very difficult to debug problems, especially in languages with ifdef-type constructs and macros and that’s especially true when macros can have scope that extends through a whole ‘compilation unit’ and not just ‘within’ a file.
Similarly, you find tools for languages with preprocessors or ‘compiler directives’ that don’t support showing the user a file after the preprocessor has run or the ‘compiler directives’ have changed the source file. Versions of ‘gcc’ and ‘cc’ have such an option (the -E option), most Verilog/SystemVerilog compiler do not have such an option – despite the fact that all these languages have very similar macros and include mechanisms.
In general, professional programmer’s recognize the existence of something called ‘ifdef hell’ – where excessive use of ifdef-type constructs makes code very hard to read efficiently. It is far better in such cases to write a code generator designed in such a way that it keeps the code from becoming yet another source of complexity that – like underbrush – can hide bugs (in much the same fashion as overly complex law hides loopholes that benefit special interest groups at the expense of society).
These kinds of oversights make it very difficult to debug some types of problems, an amateur doesn’t think of that, a professional does. That means the professional will carefully examine a piece of third party Internet code that is a candidate for use, to see whether or not it actually handles likely ‘use cases’ and that it works correctly in relevant ‘corner cases’. If it doesn’t they will seach further, or write their own version – they may very well invent a ‘better wheel’ because they need to build a design based on professional code, not amateur code. Often, in fact, the time spent examining a whole bunch of amateur designs will vastly exceed the time it would have taken to just do it yourself – and do it right.
Often vendors think they are saving money by hiring relatively inexperienced programmers – but then fail to provide for the mentoring and feedback processes needed to turn these people from amateurs into professionals. They and their customers will pay for that mistake for a long time to come, because of oversights like the ones described here.
All too often ‘bad code’ exists for a long time in commercial products because the cost of some third party competing with better code is too high, due to startup costs and artificial monopolies created by the legal system allegedly in societies interest, such as copyright and patent. This ‘bad code’ creates all sorts of problems for society. Unfortunately, a lot of things in legal systems don’t actually work to societies net benefit. But that’s a topic for another day.
Verbosity
Often, amateur programmers will assert that ‘shorter, less verbose, code has fewer bugs’ and will further claim that this is ‘supported by studies’. This is not entirely false, but it’s often false.
There are two fundamental issues here. One is that most programmers / engineers / physical scientists have ZERO training in evaluating studies. They look at a study, and they choose to believe the results if they fit their pre-conceived notions about how the world works, and they choose to dis-believe the results if they don’t fit those preconceived notions.
There is a third choice, and it’s the correct choice: to look at the study and see what was actually measured and how good the measurement is likely to be. In a lot of ‘studies’ – and this is even more true in surveys or polls – the measurement is actually worthless for any practical purpose or at least worthless for the purpose at hand. Most people have no idea how to even begin thinking about this. It’s really unfortunate that we don’t require everybody to take some courses in social science, particularly in critical reading and evaluation of research, at the grade school level. The basis of science is measurement – science ultimately derives from the decision of enlightened Middle-Ages thinkers to require SOME arguments to proceed from measurement instead of religious truth. Having experimental and control groups is NOT the basis of science, that’s just one technique of MANY to try to improve the quality (accuracy, precision) of the measurements that are the true basis of science. When fields such as astronomy and economics are based on measurement, they are sciences.
There is a term for what we are doing when we take a measurement in some context and assume it applies in a different context: this is called ‘pseudo-science’. This technique is disturbingly often used in marketing, law, and politics, but people in most walks of life (including engineering and programming, where we would like to think people are smarter and more ethical than this) will use it surprisingly often.
In the case of coding, we need to consider that a study may not be looking at code written by a professional, as defined here, and if it isn’t than trying to apply the conclusions of the study to professional code is psuedo-science: it’s taking a result measured in one context and assuming it holds true in a second, very-different context. A study that claims amateur programs that are ‘shorter’ are ‘better’ in some sense then ‘longer’ programs could have all kinds of issues that make the result not generalizable to professional work. For example, professionals often make use of code generators in very clever ways to produce large pieces of code that are still very maintainable – far more so than most hand-written code to do the same thing.
In practice, many studies about programming turn out to be worthless and even highly misleading when examined critically. Thank the publish-or-perish system.
Sometimes longer is better. It allows the programmer more opportunity to create patterns (both within files and across files) that can be easily understood by a reader that spends a reasonable (but not excessive) amount of time looking at the code, in place of shorter but much more cryptic code that has to be painstakingly reverse engineered. The Obfuscated C Contest has many examples of code that is a lot shorter than it could have been – and almost impossible to read. This is an extreme example, but the principle is generally applicable.
The human brain has a remarkable capability for pattern recognition and matching – and really good programmers understand this and build on this capability in writing code. Aligning things in vertical columns as described above is a very simple application of this principle: much more sophisticated things are possible.
Sometimes more lines of code is better than fewer lines of code when it facilitates the reader taking advantage of those wonderful and extremely powerful pattern matching and pattern recognition capabilities the human brain has.
And even in situations where the code could be made smaller, it’s not always desirable to spend the time required to do so. Optimization of both language and code can be very time consuming – and it should always be a professional judgement call as to when that optimization should or should not take place.
There is another fundamental issue to consider with respect to verbosity. A lot of engineers are both bad writers and bad readers. It takes them a long time to do either – they are not efficient at either. Often, people that are bad readers will delude themselves into believing that ‘shorter is better’ because they hate reading and want it to be over as soon as possible. They blame the author of the code/comments for their own short-coming. The same applies to bad writers with respect to writing code/comments.
Somebody with more personal integrity would instead consider that maybe the problem is not the code/comments they are complaining about, maybe the problem is found between the back of their chair and their keyboard – namely themselves. There are specific techniques that people can practice to get better at reading, in which case their ideas about what constitutes ‘overly long’ code will likely change dramatically. An amateur programmer will often choose to do nothing about being bad at reading: a professional knows they can not afford to be bad at reading. A huge percentage of the total time spent on a piece of software will be spent on maintenance – and that usually means reading a lot of code. Even if it’s your own code this is true – nobody remembers all the details of their code six months after they wrote.
Similar considerations apply to writing. Shorter is not always better. Professional writers of books, newspaper articles, magazine journals seldom try to get their topic down to the smallest number of words. They pay attention to word count, in the same way that a good programmer pays attention to speed, but they also recognize that shorter is not always better. Different people learn in different ways, and sometimes reaching a broad audience means having to accept longer writing, perhaps even some redundancy. The same applies to writing code and especially to writing comments. Managers and engineers that are themselves bad writers and/or bad readers will often complain that something is overly long when in fact it is exactly the right length, because they don’t have the reading or writing skill needed to understand why things were done the way they were done. Their views will often change significantly should they choose or be forced to develop the skills they should have learned in school.
This is one possible complication that must be addressed when moving people from engineering to management – and all too often experienced managers from an engineering background have never figured out or been told that they need to fix their skills in this area. Finding a tactful but still effective way to communicate this can be a challenge.
A key element in learning to be a good writer is the editing process. A good editor is worth their weight in gold. All too often organizations have NO review process, and only ONE person – the author – looks at the comments or the other documentation they are responsible for. This is often ‘justified’ by overly tight schedules imposed by upper management – which tend to occur when upper management people are attempting to make themselves look good at the expense of everybody else – a sociopathic mindset. This is a recipe for long term disaster, though it sometimes seems to work in the short term when people aren’t examining events with care and a rational mindset. Don’t let this happen in your organization – and if it is happening, find some way to deal with the sociopaths.
Ironically, the same social science and liberal arts courses that are often despised and degraded by many programmers and engineers are also (sometimes) courses that could have taught them to be much better readers (and writers, and public speakers), not to mention better at interpersonal relationships, including relationships with their co-workers. A better grade-school curriculum would help enormously in making sure people already had these skills before they started college, let alone started working as a programmer or engineer. Also, small class sizes are critical to teaching these skills effectively.
Misunderstanding of Functional Programming
Amateurs sometimes behave like lemmings, they love to jump on the ‘latest new thing’, even if it isn’t really new, and they are very susceptible to marketing that misrepresents ideas or techniques. Often they remind me of people desperately looking to join a cult. Or maybe another way of looking at it is that amateurs, like children, love to play with shiny new toys.
One current area where this is happening is the topic known as functional programming. The vast majority of material that I’ve seen currently posted on this topic is amateur work. All sorts of claims about the ‘superiority’ of the ‘new technique’ are made, but most of what is being said is either not new, or simply doesn’t survive critical scrutiny.
Hardware systems are made up of two types of design elements: combinational logic, and sequential logic. We can think of functional programming as the software equivalent of combinational logic. The key characteristic of combinational logic is that it doesn’t have state. A function takes in arguments, processes those arguments, and returns an answer, just like combinational logic. Sequential logic, on the other hand, has state.
It has been known for a very long time that pure combinational logic or pure functions – those with no state – have some real advantages in some situations over state based design. There are some fairly complex designs involving pure combinational logic, such as the different ways to implement math operations in hardware. Functional programming is not a new idea.
But no real, non-trivial system consists of purely combinational logic.
Part of the amateur propaganda created for functional programming is that it is somehow superior to other types of programming, such as object-oriented programming. The propaganda represents things as if people had to make and ‘either-or’ decision: pick one or pick the other.
This is completely nonsense: real systems need to process state. As soon as we introduce a need for state, we can no longer do pure functional programming.
On other words, in practice, to be useful, real systems need state – the final result of a functional operation, and sometimes intermediate results, need to be stored somewhere.
For ‘intermediate results’, think about systems where we calculate multiple possible outputs at the same time, only later deciding which one to use, such as prediction-correction systems. This might be required in situations where we need to keep up with an input data rate – but don’t yet have the information that tells us when we can stop (a common problem in test and measurement systems).
If we throw out the object oriented approach, then the functional programmer is left with storing their state in global variables, with no information hiding. This is not at all desirable, and this becomes easy to think about in the context of non-trivial programs: it’s much better to store some of our state locally inside objects then it is to make the entire state global, because in doing so we reduce the complexity of the global state. This works especially well when you have a well designed language with keywords such as ‘private’ or ‘protected’. It’s all about managing complexity.
Anything that can happen with combinational logic has parallels with functional programming. In one form or another, the same considerations that we find in the design of combinatoric systems apply to functional programming. Remember that everything in a computer comes down to logic operations (combinatoric) on state (sequential). You can’t get away from that.
When we look at complex combinational logic designs, we often find there is a need for state within the combinational logic, or in other cases there might not be a need, but there can be a strong benefit from having state. A single flip flop (1 bit state element) can substitute for an enormous amount of combinational logic in some designs.
In these cases, a fanatical adherence to functional programming would actually make the design more complex! This is often the case with pipelining, where sequential elements allow an algorithm to be done in small bites, rather than having to do the whole thing in a giant combinatoric system. We also have the issue that real systems often require pipelining to meet timing – thus not just design complexity, but real world constraints can prevent a pure combinatoric system from working, or at least from working efficiently.
Often with functional programming, we find code that is very inefficient because people are so focused on ‘being functional’ that they don’t realize they are introducing enormous inefficiency into their code. For example, a function that produces a waveform might call another function to add wave shaping – but then you might end up making some very expensive calls to trigonometric functions every time you generate a period of the wave. It’s very unlikely your optimizer will pick up on this. It’s far better, if the wave shaping is based on the same calculations, to precompute as much as you can, store that state in a sensible way, and pass it in so you only have to do it once!
Another claimed advantage for functional programming is that it’s ‘inherently parallel’. True, we can say the same thing about combinatoric logic, and in fact large arrays of parallel combinatoric logic are common in hardware system so that calculations run faster or sensors are working in parallel. For example, think about the enormous parallel arrays used to implement sensor systems in digital cameras.
But the propaganda for functional programming is that this somehow ‘does away’ with the notorious complexity of doing parallel programming with other techniques. Again, this is nonsense: the functional programming itself may be parallel, but to be useful you need state, and managing that state is where we find parallel programming becoming complex (e.g. the Readers and Writers problem)! Functional programming doesn’t magically let you avoid this. It doesn’t do any good to have all sorts of fancy processing on the bits of your camera image if you can’t actually transfer those bits to the user – and to do that you need to store and transfer state, and for a large array that’s not trivial. The sensors and all kinds of logic associated with the sensors (such as filters) may be working in parallel, but sooner or later that information needs to be put into a serial data stream that can be stored on a disk or other medium.
Professional programmers will sometimes use functional programming, but it isn’t done in isolation from OOD, it’s not a question of choosing one or the other, it’s done as part of a complete system with both functional and state-based elements such as objects. They’ll spend a LOT of time looking at the complexity of the state in their system and figuring out how to manage that complexity.
Amateurs misuse, misunderstand, and mis-use functional programming, and end up with overly complex and/or inefficient designs – or just wasting lots of other people’s time in pointless arguments about the alleged superiority of the functional programming cult.
Summary
A lot of the material related to programming that we find on the Internet, including posts to discussion forums, code libraries, code examples, tools, coding styles, and other things, is written by amateurs.
To become a professional at writing code – to be far better than the untrained, inexperienced, and often careless people who write a lot of the code one finds on the internet – one must learn to differentiate how amateurs do things from how professionals do them. This typically requires time and practice of skills, including mental skills such as how to think about problems. It helps a lot to have mentoring and feedback from others who already know how to ‘do it right’. The mentors should have good people skills, including good communications and listening skills.
Many senior engineers and most academics will not be able to mentor others in this way, either because they themselves don’t program at a professional level (and perhaps don’t know that) or because they lack the relevant people skills. Everybody should have a class in how to communicate effectively with other human beings in Grade School – and perhaps another in College – it is astonishing how many academics and managers and senior engineers routinely make mistakes that are discussed early and in great detail in books on good communication.
Organizations that want their people to be professionals will set up appropriate mechanisms to ensure this process happens. It’s not easy to change an existing organization, where typically there will be experienced engineers – hopefully not software engineers – that are amateurs in their outlook and perspective and don’t want to admit it or learn a different way of doing things. Many senior electrical/mechanical/chemical/system engineers have a lot of coding experience, and have written lots of code associated with valuable products – without ever learning how to code at a professional level. Many of these folks have a long history of hacking together poor solutions to problems – and they will always do things that way until somebody steps in and helps them change.
These people need to be handled carefully to ensure that they don’t create roadblocks to any change process, and any mentoring they do for younger engineers may need to be monitored and supplemented with training in coding at a professional level, for otherwise they might end up just passing their bad habits on to another generation. The result, of course, is that companies miss market windows, miss schedule, take longer to write code, and have more bugs in the code that their customers interact with – and that ultimately hurts their reputation and their bottom line.
The key to handling the senior engineers or programmers is to respect the contributions and domain knowledge of these individuals, while convincing them (or helping them convince themselves) that an ‘old dog’ can still be taught ‘new tricks’ (you might want to avoid the phrase ‘old dog’ to keep yourself out of trouble). Doing this requires thoughtful management and good people skills – and it is easy to screw up here. Be careful! In the vast majority of cases, telling somebody that they are coding at an Amateur level is the wrong thing to do – no matter how true it may be! Some people can handle that level of bluntness, and may even prefer it, but that’s rare. Typically, you must find a different way to word things or manage things as part of facilitating change in the organization.
With respect to the younger engineers, what we often find is that in attempting to improve the work done by their organization, they have actually achieved the exact opposite of what they set out to achieve. Using a tool such as Rubocop to impose bad rules on people is not helping people learn to write better, more professional code. Instead, the people in their organization are writing inferior and less professional code – and that may not be realized for a while – especially when these people start getting their bad habits set in stone and they too become part of the problem instead of being part of the solution.
Often these younger engineers who propose style enforcement are very bright people, with good intentions, who consistently get good performance reviews – but they don’t yet have the experience to realize the pit trap they are stepping into when they try to impose problematic coding standards on others; standards based on their own limited experience and erroneous material they have stumbled across on the internet. Often amateur material on the Internet presents itself as if it was professional – being able to see through this often requires more experience than the younger engineers have. Good intentions alone are not enough.
It’s up to managers and senior engineers to keep an eye on the young engineers and help them not get themselves and the organization in trouble. The good news is that the younger engineers tend to be less prickly and sensitive than the older engineers, and their bad habits are not as deeply entrenched, but you still have to careful – and respectful – in dealing with them. A person that has been told time-after-time that they are very smart will often have trouble in recognizing situations where their performance is sub-optimal and accepting the need to change.
For individuals not working in an organization, the good news is that even without mentoring and feedback, one can still improve oneself by appropriate thought and effort. It’s a lot harder, and it takes a lot longer – don’t delude yourself into thinking that isn’t true – but it can be done.
As before, I’ll note that professionalism in coding is not largely a function of age when looking at adults. It’s easy for both younger and older engineers to make these kinds of mistakes. Some people in both groups can learn from their mistakes pretty quickly, others will not learn without more effort on the part of their mentors and management, and some – very few – will never learn. For the last group, you can either get rid of them or treat them as charity cases. In the latter case, do it politely and respectfully if possible – it might not ever be appropriate to tell pr even suggest to somebody you consider them a charity case, but you still have to find ways of wording things and handling them. Sometimes you can find a way to slot them into an appropriate position and role that lets you balance the good of the individual against the good of the organization, where their strengths can still provide value without their weaknesses causing problems.
In both the workplace and outside the workplace, a key element in learning from one’s mistakes is to avoid hubris and/or arrogance. The refusal to learn from one’s mistakes often takes the form of hubris or arrogance. If you carefully re-examine your words and actions from time to time, and can detect circumstances when you are exhibiting either characteristic, then that will help avoid making that mistake in the future – and will ultimately both speed up the learning process and help you change from being an amateur programmer to being a professional programmer.
There are a LOT of people who are considered ‘smart’ or even ‘extremely smart’ by themselves or others, who nevertheless still code at an amateur level, even after many years of ‘experience’. This usually happens because they are unwilling to recognize their mistakes – even in private or to themselves – and thus they can not learn from those mistakes. Self-honesty is critical.
We can think of this in terms of a difference between Intelligence and Wisdom as characteristics of individuals – a point that will be familiar to many players of well designed Role Playing Games. Note that we’re not talking about IQ tests when we talk about Intelligence – we’re talking about the real thing.
A person who can learn from their mistakes, and does so quickly, can change from an Amateur coding mindset to a Professional mindset in a reasonable amount of time. That person has both high Intelligence and Wisdom, at least with respect to coding.
There are many people who have one but not the other – Intelligence but not Wisdom – and those people will always be a liability to a coding group and to an employer. This doesn’t mean that an employer shouldn’t hire such people – sometimes employers will choose to hire people who have a learning disability such as being dyslexic (perhaps as a form of charity) – and that’s ok, business is not always about maximizing profits, the managers and senior engineers involved just need to make sure that the organization can work efficiently around the limitations of those people.
On the other hand, when it comes to the prima-donnas or the arrogant – the people that refuse to learn – an organization is almost always better off without them.