One of the most common questions about software development is how to become more productive as a programmer. It is well known that there is a big difference in productivity between mediocre programmers and great programmers, although this is still a little understood phenomenon. For example, there has been anecdotal evidence that the amount of work a great programmer can produce in a week cannot be matched by a mediocre one even in a year’s worth of effort.

So, what is the source of the difference? In general, it includes everything from intellectual capacity to focus, but there is usually a small number of common elements that can be observed in the work of every good developer. It is helpful to try to incorporate such elements to our normal workflow as much as possible. Here I talk about three practices that I have commonly observed on highly skilled developers.

Doing the right thing first

A great foundation for creating correct software involves the avoidance of cargo-cult: that is, the practice of doing things just because they seem to work on the surface. Smart people don’t do something because somebody else told them to do, or because it seems to work most of the time.
Excellent programmers go the extra mile to understand how something works.

This practice can be generalized to any aspect of life, but it is absolutely essential in software development. For example, the cause for most software bugs usually lies hidden in something that was not well understood by the programmer when the code was initially written.

People who don’t look for the root causes of what they are doing are prone to development that uses cookie-cut strategies, including the following: using copy-and-paste from online forums as their primary way of solving problems; incorporating code generation tools that create templates that seem correct, without understanding why and how they work; using libraries and frameworks that are unnecessary just because they are employed by existing code.

The best way to avoid this issue is to make sure you understand what you are doing at every step of software creation. Given the complexity of today’s architecture it is true that complete knowledge is very hard or maybe impossible to achieve. However, good programmers strive to learn at least the essential facts about everything they are doing.

Being proactive in improving the system organization

Another issue that is very important is the way your organize software. Great professionals take care not only of how the current work is performed, but also how past work is organized. They continuously look for methods that can improve the existing code base in a way that will make future decisions easier to make.

There are several approaches that one can take in this direction. One is to use refactoring tools, which are quickly becoming so common on most IDEs. Modern refactoring tools can help turn spaghetti code into reasonably readable code with little work. And using automated testing tools can make the whole process even simpler.

Another suggestion is to allocate time to fix small issues, including things like formatting, proper commenting, and related problems. Although such small things don’t seem to be a high priority issue, they have the potential to turn a code base into a unmanageable mess if left without proper attention. Good programmers know that this kind of effort can reap dividends almost immediately, due to the improved maintainability of the resulting code.

Development Speed

Another overlooked factor in programmer productivity is speed. Usually people equate quick effort with solutions that were not well designed. While it takes time to do the right thing, it is also important to proceed at full speed when a good solution has been decided upon.

There are several factors that affect programming speed. Most of them have to do with badly managed programming environments. For example, some projects require approval even for small changes in a code base. This kind of bureaucracy can yield bad results for the general quality of a software project.

Another issue that may impair development speed is the use of wrong or outdated tools. For example, there is no excuse for failing to use a modern source control application. We currently have so many high quality options such as subversion, git, and mercurial, and they are all free. A good developer should be able to just pick the best tools available and use them as needed.

To progress quickly it is also essential to have fast feedback. This is usually achieved by the use of testing tools, such as unit testing frameworks and libraries. These days, there are unit testing frameworks for every major programming language, so improving in this aspect should be a matter of simply designing tests, and applying them in a way that exercises the functionality you’re implementing or modifying.

Conclusion

Great developers are hard to find, but everyone can learn important lessons from their practices. I have observed throughout my career that good developers are extremely concerned with doing things for the right reasons, by understanding as much as they can about the problem and the language of implementation. They are also very proactive about organizing the system for optimum work performance. Finally, they are able to produce high quality work in a speedy manner, and they use whatever tools are necessary to achieve such results.

Image credit: commons.wikimedia.org

Software development is a very specialized area. It often enables the creation of strict specialities, where a programmer spends his whole time learning the intricacies of a particular language and computer environment and employing that knowledge to particular applications.
While it is true that most of us go to school and learn a common introductory computer language, it is very unlikely that jobs will be available for the exact language you learned during those years.

For example, Pascal was the language used to teach programmers for a long time. I was one of the students that had his first programming experience with Pascal. Despite this, as one of the consequence of the increasing specialization in software development, nowadays hardly anyone writes code in Pascal anymore. All the “real” programming is performed in languages that are reasonably good at exploring the existing computational environment and supported by major manufacturers of PCs, Macs, and mobile devices.

When we look at job descriptions, the specialization of the profession becomes even more impressive. Each job requires a different subset of skills that can only be acquired with years of job experience. All of this leads to a scenario where people feel compelled to spend a long time working with the same tools and libraries.

Specialization versus Exploration

One of the main consequence of high specialization in software engineering is that it is nowadays possible that you spend 99% of your working time writing code in a single language, for a single computer system.

While this may lead to an increase in productivity, in the long term it may also be harmful to your personal development as a engineer. It is not only useful, but also desirable that we have more varied experiences. The current trend in specialization may lead to more productivity in the near future, but it may also impair the development of equality important skills, such as an appreciation of system architectures in general, the a wide knowledge of different systems and a familiarity with alternative solutions that might just as well be useful to the problems you’re solving.

Boredom and burn out are frequently an adverse result of overspecialization. Under some circumstances it becomes increasingly hard to focus on a single aspect of the system. Good software engineers understand that this means they need to look at something different to improve their careers.

How to Embrace Diversity

There are no rules as to what you can do to break your common patterns. Just look for things that entice your imagination and jump at the opportunity of increasing your development possibilities. Here are a few suggestions that may help you, though.

• Learn a language that provides a different programming paradigm. For example, if you are good at object oriented programming, try to learn a functional language. This will help you see problems in a different way. Certain programming languages are adequate for a particular application domain, so making this change can also help you to understand these different problem domains as well.
• Investigate new programming environments. This is a good way to learn how things can be done differently from what you’re used to. For example, if you are used to graphical IDEs that can do everything with a mouse click, try to get your hand into a command line based environment. You will certainly feel you’re unproductive at the beginning, but the idea is to learn different tricks from command line environments that can be translated into your own programming workflow.
• Experiment with different application areas. Sometimes we’re so used to our particular domain that considering a different area may provide a lot of insights into the way we work. For example, if your area is web programing, try to create something that requires number crunching with floating-point libraries. It is certainly a nice challenge that may help you in learning new concepts. On the other hand, someone currently working with scientific computing could look into creating a basic GUI-based application, or even a simple game.

As you see, the possibilities are limitless. Spending a few hours in such an exercise will not transform you in a specialist in a new area of programming, but will give you a few insights on how programming works in other domains. Then, if one day you have a similar problem in your hands, the solution will be much easier to figure out based on the concepts that you grasped during the exploration period.

Conclusion

Software developers don’t need to spend their whole time solving a narrow class of problems. In fact, it is sometimes just as helpful to look into different directions, which can provide a renewed focus and fresh ideas to be applied at your main goal.

Using exploratory programming exercises can be a great tool to improve your programming skills, even if you’re only using these concepts for small time periods. The general ideas that you will absorb can be applied in many areas and benefit current projects as well as future work.

Image credit: commons.wikimedia.org

Literate programming is the methodology for software development proposed by Don Knuth and used by himself to build the TeX system for document preparation. TeX is the system used as the basis for LaTeX, which is itself used by everyone in science and mathematics to write technical documents.

The original version of literate programming was developed in a dialect of the Pascal language and the system was called web (notice that at the time there was no such thing as the World Wide Web). Since then, Knuth has moved on to use C as his main language. The resulting literate programming system was improved and renamed to cweb.

Literate Programming

The main advantage of literate programming is that it allows the smooth integration of programming and documentation. Cweb programs are composed of documents that mix a description of the program with C code that implement what is being explained. This document is later on processed to create a source file as well as a documentation file.

Literate programming provides a paradigm shift in software development because code becomes secondary to the main description of a concept. The explanation of what a program does is what drives the programmer through a literate programming document. In a sense, source code in C (or any other language used by the system) exists only to illustrate what is is being discussed in the documentation part.

On the other hand, literate programming provides all tools necessary to put the coding sections together in the right order to create full-featured programs. Instead of programming according to the rigid sequence required by the C syntax, one can add new instructions to the code in a style that is more consistent with the general concepts, rather than having to worry about exact placement and with particular file locations.

Using cweb on Standard Projects

One of the critiques of literate programming is that one needs to apply it from the start to benefit from the possible advantages of this coding style. However, most existing code has been developed in a non-literate way, that is, it is code that follows the strict sequence dictated by language compilers. All established companies have a huge investment in non-literate code that is being used in production, and it is not realistic to believe that such code would be rewritten for the sake of a new programming methodology.

Consequently, if one were to convert from normal programming conventions to literate programming than they would need to rewrite a huge number of C, Java and other language-specific files. And notice that all this would be necessary just to start programming with literate programming tools such as Cweb.

One of the things that people don’t know, however, is that it is not strictly necessary to use separate literate programming files to create literate programs. Although tools such as cweb expect to read a file that is a mix of documentation and code, it is possible to trick the cweb tools to use C source code just as well, after a few modifications have been applied. That is the goal of the script called “CwebNoTangle” that I wrote some time ago.

CWebNoTangle script

The main idea behind the tool is that, if we intend to start using literate programming without rewriting the existing software, the first step is to avoid using ctangle altogether. CTangle is the part of cweb that reads literate programming files and convert them into C source files. Most of its work is concerned with putting together C fragments in the order that the C compiler expect them to appear.

For example, in C the header files for a source file are generally included in the top of the file. However, a literate program doesn’t need to define such header files at a particular point. Instead, it is possible to add header files at any location when they are first needed.

However, we just need to notice that when using a straight C file as source for the literate tool, it is not necessary to perform an independent reordering step. Instead, the source file is already in the order expected by the compiler. This means that the ctangle tool is not strictly needed to create a syntactically correct source file.

That’s why I called the set of scripts CWebNoTangle: they allow the use of cweb to create full documentation for software, without the need to use ctangle for creating C source files.

You can take a look at how the system works on this github project [1]. The code is written as normal C with extended comments, which store documentation that can be easily translated into TeX commands.

Conclusion

Literate Programming is a great tool for creating software that is easy to understand and modify. Traditional literate programming tools are hard to integrate with existing environments, however, because they require code to be written in a different format.

It is possible, however, to use normal C files as input for both source and documentation. A simple example of how to do this is provided in my CwebNoTangle script. You can use a similar system or create your own, but it is nice to see that there is a simple transition between traditional programming methods and literate programming.

References

[1] https://github.com/coliveira/Hyperbolic/blob/master/cwebnotangle
[2] Image credit: commons.wikimedia.org