Developing commercial software is an activity that many companies have to do. Sadly, the results of such efforts are usually misplaced, as a lot of software that is created by companies is either inadequate, or takes an order of magnitude more time and resources to build than it should take.

Assembly Line Thinking

One of the big mistakes of commercial software development is not realizing that building software is very different from building a traditional product such as a car. There is no industrial line that can be used to create a complex software.

Companies that are used to the assembly line methodology believe that you can just assign different people to do small pieces of software, and then combine everything. While it is true that people have different skills in the software world, programming is best done when someone is able to reason about the whole functionality of the program, before going to the details.

Ideally, you want people to be assigned and responsible for creating large pieces of code, and let them think hard about it. If you don’t have people that is completely involved in the creation of software in this way, what you end up is with a few pieces of software here and there that, while solving the original problem, have no clear relation to each other. Therefore, the product will miss cohesion and an inner structure that would allow an easier development strategy for future changes.

Lack of Proper Design

Another reason why commercial software is so frequently bad and expensive is that the design process is not done properly, or when existing, puts emphasis on aspects of the software that are not essential for a high quality product.

One of the ways in which this problem occurs is in the lack of a general strategy for software design, with decisions made by people that really don’t understand the underlying problems that are being solved.

Maybe a simple rule that could fix this issue is the following: don’t let business people decide what goes into software. Listen to their opinions, but leave the options open so that the designers of the software have a real decision on what features will be included and how.

One of the big issues faced by software writers is that business people that approve the software think they understand something about how to write it. This is akin to patients thinking that they can give orders to a doctor about what treatment to follow. Many of these power users like to give a lot of recommendations about what the software should look like, mostly based on a superficial knowledge of the problem. They lack the knowledge of software design or user interface design that would be necessary to make the recommendations valid. However, due to the realities of business organizations, these ill conceived suggestions are frequently taken as requirements for the software.

If we really want a software system to be successful, it needs to be designed by people that really understand not only about the problem domain, but about proper software design techniques.

The other problem is having a design done by committee. This happens when, while defining the requirements of a product, analysts go to a large number of people in order to define the features of the application. In doing this they, frequently for political reasons, adopt the disparate suggestions of several people. The suggestions may even be individually sound, but if you let software to be designed this way it will completely lack a unifying approach, and will certainly be more complicated than it really needs to be.

The best way to avoid this kind of problem is to have one person, or a small group of people, responsible to define the main architecture and features of the software. Whenever there are conflicting requirements, it would be up to this group to determine how the situation should be handled, in a way that it doesn’t damage the main functionality of the application.

It is also interesting to make sure that owners of the application are informed of design decisions, so that there will be no surprises on the customer side. Every design decision should be careful explained, so when something is not included, there is a reasoning behind it. Most of the time, users will be satisfied when the application provide what they want, even if it is not in the same way that they initially imagined. It is just a matter of proper communicating with users about how the application is supposed to behave.

If you are like me, you probably think that the advertisements on Google are mostly annoying. In rare cases they show something interesting, but most of the advertisement on Google searches are not useful for the things I am looking for — unless I am trying to buy something.

Now, for a limited period of time, you can have access to Google search without the annoying ads. The trick is using the test search engine that Google announced recently.

The idea behind this is that Google wants to test its new infrastructure for search. So, they will allow anyone to search on it, as a way of checking that results are correct.

The interface is identical to the current system, but the results may be a little different.

So here is the tip: if you want to help test this new infrastructure and, more importantly, avoid seeing the ads showing everywhere on Google, just use the address http://www2.sandbox.google.com/ instead of google.com.

This is a piece of information I am posting for my own convenience, but that may be of use to other people using Git.

I use git to keep my local files. Not only programming files, but also all types of information, including blog entries and other miscellaneous data.

Sometimes I delete or move a whole directory, and wish to recover it later.

The recipe is simple, but easy to forget because I rarely use it:

git ls-files -d | xargs git checkout --

To explain this, notice that git ls-files -d just list deleted files. Then, xargs will feed these names to git checkout --, which is responsible for restoring the file.

This weekend I spent some time to test clsql on sbcl. I tried to do this before on Windows, but I couldn’t get it to work. So I used my trusted Amazon ec2 linux machine to do the job.

Initially, I had to use yum to install sbcl. It is as simple as

yum install sbcl

and watch the packages being downloaded. I also needed to install gcc and mysql-devel (this depends on the distribution).

yum install gcc
yum install mysql-devel

Then, I used asdf to install the required packages, including clsql. These are some of the commands I used:

(require :asdf)
(require :asdf-install)
(asdf-install:install :uffi)
(asdf-install:install :clsql)

There was a lot of complaining from asdf-install, especially related to a missing gpg library. Also, most backends other than mysql failed to compile, which is ok, and I just ignored it selecting one of the options given by the debugger.

Once installed, you can use clsql to make calls to mysql. I created a simple database called carlosdb with the commands (typed on mysql prompt):

create database carlosdb;
use carlosdb;
create table users (
     `id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
     `name` VARCHAR(100) NOT NULL);

Then, we can use clsql in the following way:

(require :asdf)
(asdf:operate 'asdf:load-op 'clsql)
(use-package :clsql-user)
(connect `("" "carlosdb" "" "") :database-type :mysql)
(execute-command "use carlosdb");
(query "select * from users")

The two results of the last operation are lists like the following:

((1 "john nash") (2 "james jim") (3 "jon williams"))
("id" "name")

where the first list is composed of sublists, each have as elements the contents of a record. The second list just stores the names of the fields (id and name, in this case).

Some Helpful Resources

• Practical Common Lisp: a book that explains what asdf is all about.
• An example of clsql use
• The pages for clsql and uffi on cliki.

In the last post about learning to use the debugger, I mentioned a few tricks that you can perform with a good debugger, like for example changing the values at certain memory locations. These are tricks that can speed up your debugging process, because you don’t need to recompile lots of code and restart your program just to test a small change.

Another related technique consists of using the compiler itself to perform some check while you make changes to the code

An example of how this works is presented in a post by Hovik Melikyan. He discusses what he calls brute force programming, where the idea is to use the compiler to perform temporary checking.

For example, one of the techniques is to use the compiler to figure out exactly where a variable might be used. The simple thing to do in this case is just to rename the definition of the variable temporarily. During compilation, you will see all locations where the variable was used and fix them accordingly.

A related technique that I use in C++ can be used to show where a type was defined. If you want to find the definition of a type (usually from a library, such as MFC, for example), what you can do is just writing a new typedef with the type you are searching for. If the type is, say, Widget, you would then write

typedef int Widget;

If you do this, the compiler will immediately give you an error saying something like: “type Widget was previously define on file X, line Y.” So, now you know where the type was originally defined.

Is there any other trick that you use to make your compiler work for you? Let me know.