One of the big concerns of companies that employ software engineers has been how to measure the productivity of these people. From a business standpoint, the idea is that one should find ways to maximize the work done by their employees. However, it is difficult to evaluate how this maximization process would happen if no one knows how to measure it.

Performance measurement in the software world is complicated by several issues. For starters, the metrics that have been used by the industry are hardly useful. Here are some examples:

• KLOC (thousands of lines of code): is virtually useless, because it can be easily inflated, and correlates inversely with readability. If you hired a civil engineer, would you pay him by the amount of concrete he used in a building?
• number of functions: similarly to the previous one, it can be easily gambled, because one can create very small functions instead of longer ones.
• number of bugs closed: some developers are evaluated by the number of bugs the were able to close. This can however generate a culture of bug proliferation.
• number of features: probably a more useful way to measure productivity. However, it is difficult to quantify how hard to implement a feature is (for example, generating a report versus writing a compiler optimization step). Also, quality of the feature must be considered as part of the equation. While bug free software is rare, but # of bugs should be as low as possible through careful testing.

In general, the problem of measuring software engineers based on any of the measures described above is that they are smart people. If you give incentives, for example, for writing more lines of code, developers will find creative ways to create more lines than necessary. It is just a fact of life: people align themselves with the rewards.

The conclusion is that measuring productivity is as complicated in software as in any other area of engineering. For example, how can you measure if a mechanical engineer is productive? Maybe by the quality of projects that he finished, but that is not a quickly applicable metric. Sometimes, companies want to look at the job of a software developer as if he or she were a writer — who is payed by the number of words written in a day. That model doesn’t scale, because different pieces of software have different difficulties. Some parts of a program have to be rewritten dozens of times until they are correct. Other parts are pure repetition.

If a comparison needs to be made someone that writes, probably the best model would be a mathematician. A mathematician writes papers, but productivity is not just a matter of typing speed. Mathematicians need to create “bug free” theorems, which take lots of time and craft to write.

The best solution I have seen to this kind of problem is: get smart people to work together, and provide them with a direction. This is how most of the good software we know was created. In fact, having smart and goal oriented people is probably just what you need, because sometimes the directions proposed by business people are not right. It is more interesting to have the right result than a monster that was created according to the view of a clueless manager.

The good news is that some companies already know this lesson. This is the way Google “manages” its developers, and other companies like Facebook, and other startups are following the lead. In the future, if we are lucky, other mainstream companies will adopt this view.

Software entrepreneurs constantly make the mistake to equate number of features in software with a better chance of dominance in the market. And, if one look at how the market works, there is some justification for this thinking: if a software has a large number of features, then it can attract a larger user base. And, of course, more users means more money. Having a big app, therefore, means that we can get more business streams from the same niche or from related user niches.

However, a factor that is often overlooked is that increasing the number of features in an application may also reduce maintainability. Which means that the company will necessarily take more time to react to changes in the market. Consider classic example of Netscape: they had a web browser with more features than any other, and exactly because of this they were not able to maintain their leadership in the browser arena.

Creating a Proper Foundation

The only way to avoid getting jammed in complexity is to create a strong framework in which to build an application. There are practical examples of systems that were able to grow from very small to very large in a short time thanks to the superiority of the underlying architecture. A case in point is the UNIX operating system, whose basic ideas allowed it to grow immensely without sacrificing the qualities of its core. On the other hand, the Windows OS, which lacks such a strong foundation, has not been so successful in maintaining quality.

A typical response from companies that need to cope with this phenomenon is to throw more developers at the problem. While more developers could give some contribution, it is really hard to fix fundamental problems with an architecture after it is in use for a long time. Also, more developers end up adding complexity to a system, and the end result can be even worse in terms of maintainability.

The Opportunity

While it is a sad thing that applications are generally so badly designed, this state of affairs also present great opportunities for new companies. The goal is to create software that can improve on the legacy system by providing a better foundation for extensibility. If you have a good design for a piece of existing software, that would make it much easier to write and extend. Such a framework can provide a technical advantage that would turn the table to your favor. The framework could be created and deployed in the following way:

• Create a small app that has the core functionality and is still attractive to a segment of users. It helps a lot if these users are not happy with the existing offers by big vendor. Make sure that your software present users with a good, extensible architecture. This is the factor that will make it easier to add more functionality later.
• Get even more users interested in your app by adding functionality beyond the core infrastructure.
• Avoid the costly mistake of adding too many features that would complicate your basic product.
• Once you get established, make sure you continue to maintain the basic framework. Remember, the core technology of your program is what will make it hard for your competitors to replicate what you did.

Hi, this is the 11th part of a series of posts on 30 tips to becoming a better developer. If you would like to keep up to date with the topics that I am covering, just check the main post.

As software developers, we enjoy thinking of a system in as higher level as possible. This has been an organizing principle in software development: people tend to create higher level abstractions that make it easier to reason about a more complex system.

However, working at higher levels of abstraction has its cost. Inefficiencies become harder to see, and at some point performance becomes a problem that needs to be handled.

A layered approach to software

Computers are organized in layers. The lowest level is hardware. In a higher level is application code. It is nice to work on higher levels, but if you don’t understand the most basic parts of the system it becomes difficult to make competent decisions. This is even truer as code gets more abstract.

This is not say that one shouldn’t be working at higher level. In fact, even though we shouldn’t understand about concepts such as machine language, it would be counter-productive to work with machine-dependent code.

The idea is that you should know (or at least have a good idea of) what is happening at the lower levels of your code. This way, you can understand what trade offs are being made in your code.

Another reason why it is important to understand the lower levels is that you can “break free” of the higher level language if necessary. People working in time sensitive code (such as game programmers) already know when to use C or assembly for critical sections of their programs. A smart software architect knows when to reach for help on the lower levels of the software architecture.

This is specially true when higher level abstractions don’t work. For example, suppose there is a bug in a compiler or interpreter. How can you diagnose it? The only way is understanding what they do and how it should be done correctly (in this particular case it involves checking the machine code generated by the compiler). Otherwise, one is at the mercy of whoever is supplying the programming software used.

Conclusion

Thinking with his/her own head is very important for a software developer. Understanding the lower layers of the system you are working with will make wonders for you ability to think by yourself.

In this respect, one of the great advantages of open source software is to allow people to poke the innards of applications and even the operating system. Despite what you think of software freedom, working with open source software may improve your awareness of lower levels of the system.

Another possible help is to read books and articles that can shed some light on the lower layers of your system. Here are a few suggestions:

• Advanced Windows is a great book on how the Windows OS works;
• Understanding the Linux Kernel is a great introduction to the kernel and its main structures;
• Computer Systems: A Programmer’s Perspective gives an overview of computer systems, including assembly language, hardware, databases, networks, and related topics.

If you work with programming, from time to time you will see articles talking about the inevitable downfall of C++ as a main programming language.  Some people go as far as to say that C++ is even harmful to your career and that you should avoid using it whenever possible.

Despite the reasons why some people dislike C/C++, I believe that the backlash against C and C++ is rooted more on lack of understanding than on real fact.

The Case for C

C was created to be a language for professional programmers, not for people that are still learning the trade. The designers of C created it as a means to efficiently develop operating systems. One of the consequences of this design is that C is as close to the hardware as one can get without using assembly itself.

This may be bad if you are just starting, but it is liberating if you are a seasoned programmer that really wants to get the most out of the machine.

If you look at the criticisms against C, the most important arguments are that it makes it easy for programmers to shoot themselves on the foot. No wonder, C was created exactly to make such things possible.

Writing Simple Programs Quickly

If your goal is to write small programs easily, you should stop your criticism of C. The fact is that for writing small, simple programs C doesn’t count. Such programs should be written in Perl, Python, or whatever flavor of scripting language is the most popular at the time. This is what UNIX programmers have done forever. 

For small programs that impose no bottleneck on the system any language is OK, as long as it makes it easier to solve the problem at hand. Lisp and Prolog are excellent to solve symbolic problems, for example. Quick file processing can be solved with Perl.

Large Scale Programs

Now, let us talk about the programs that really matter in each platform, i.e., large scale software that imposes a bottleneck on the machine.

In higher level languages, such as Pascal, Java, or Lisp, one is working with protective gear that allows them to be sloppy without sacrifice of safety. This is not to say that one cannot write efficient programs in Lisp, say, but doing this is harder than usual.

In a language like Lisp, if you want to write efficient code you need to understand all sources of bottlenecks. This usually means you have to understand how your particular system is implemented and how the underlying architecture works. Then, you need to find ways to avoid such bottlenecks, and doing this usually means going to a lower level of programming such as (guess what) rewrite you program  in C or assembly (or maybe using typed expression in Lisp, which is just another way of thinking in assembly).

In other words, you have to be really expert in the language, in the implementation of the language, and in the underlying machine in order to write something out-of-ordinary. And if you have the knowledge to do this, it might be easier just to write everything in C. On the other hand, while you may need to do this just in a few cases on the whole program, the bigger your program is the higher the chances you will need to go lower level more frequently. This is why writing a web browser or an operating system in anything else other than C/C++ is such a difficult proposition.

Now compare the situation with C. Sure, you cannot expect that a novice programmer will know how to handle pointers or null terminated strings. That is fine, though, because C was not made for them. The C language was created as a vehicle to write efficient software for people that care enough to use such a tool.

Notice, however, that once you master a few concepts you can treat C as just another high level language, but one that matches closely to hardware concepts like real memory addresses. Maybe you will lose the illusion of objects, for example, but you will gain a real ground on what happens in the computer.

What about C++

I talked a lot about C, but what about C++?

C++ is the child of C that was adopted by the industry. C++ has everything that the industry likes about a technology: it has buzzwords, it is constantly changing and requiring new tools and compilers, it has strong support from big companies like Microsoft, and it  has a lot of legacy code.

The legacy issue with C++ is so important that C++ has essentially killed C as a commercial product. You don’t buy nowadays a C compiler, you by a C++ compiler that can also compile C code. In this sense, C++ is really important, because it is the only way C programmers can continue using C.

Also, despite the many problems with the extensions created by C++, many of them are really useful, such as namespaces, a standard library of containers, and basic support for objects. So, basically all commercial installations of C++ can use C and pick-and-choose the features from C++ that they want (see for example Google’s style guide for C++ [2]).

In conclusion, although a lot of people don’t love C/C++, it has an important role in the industry that is not been filled by any other language. This probably means that we will continue to see good C++ programmers making money and working on interesting projects in several areas.

Further Reading

Many of the advantages and disadvantages of C++ for large scale programs are listed on  Large Scale C++ Software Design, by Lakos.

An even deeper description of C++ model is given in Inside the C++ Object Model, by Lippman.

[1] http://www.ittybittycomputers.com/IttyBitty/CppHarm.htm

[2] http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml