Testing (Part 2)

In the previous post, we've looked at what to test. Now it's time to see how to test. There are several different aspects:

• Regression Testing - Here, we test the result of current test with its previous results. This ensures that the bugs we fixed today didn't break the things that were working yesterday.
• Test data - There are two types of test data: real world data and synthetic data. Both are important, because each of these tests different aspects and behaviour of the system.
• Exercising GUI Systems - Testing GUI required special testing tools. Some of them record the events and play whenever required, while some others are based on a script. Writing code that is decoupled from GUI helps us to write better tests, because we can test the backend without interacting with the GUI.
• Testing the Tests - How do we test the tests? The easiest way is to introduce bugs intentionally in the tests and see if they complain. 
• Testing Thoroughly - How do we know if we have tested our code thoroughtly? The answer is we don't, and we never will. But we can use coverage analysis tools to keep track of it.

- summary of Ruthless Testing, from The Pragmatic Programmer: from Journeyman to Master 

Testing (Part 1)

Generally, developers hate testing. They hate to see their code breaking.

But we don't. We test every bits and pieces in our code. Why? Because we know that this is one place where we can see Butterfly effect in action! One simple mistake can cause large (and very bad, of course) effects in the future.

Test Early, Test Often, Test Automatically

Many teams develop elaborate test plans for their project. But automated tests, which run with every build is far better and successful than test plans that sit on a shelf. The earlier we find the bug, the easier the fix. In fact, a good project may have more test code than production code.

Just writing tests is not enough. We have to make sure that we run that often. We cannot say that coding is done unless all the tests are passing. These are some major types of test we need to perform:

• Unit Testing - Tests a module. Because if the module can't work alone, it cannot work with others as well.
• Integration Testing - Tests how the subsystems play and interact with each other.
• Validation and Verification - Ensures that what is build is what the user needs. Checks the functional requirements.
• Resource Exhaustion, Errors and Recovery - Tests how the system behaves in real world conditions under varying factors like memory, CPU bandwidth etc.
• Performance Testing - Stress testing, in which we tests our system and how it behaves when there is heavy load.
• Usability Testing - Different from all the above tests. Here we test the system with real users. It helps us to know how easy it is for them to use it.

- summary of Ruthless Testing, from The Pragmatic Programmer: from Journeyman to Master

Automation

When the first Model-T car was released, the instructions for starting the car were two pages long! Creeppy!! We get irritated if the car takes more than 5 seconds to start when we turn the key. A great example to show how we realised the importance of automation over the course of time.

Why do we need automation? It ensures consistency and repeatability. Manual procedures ensure consistency up to an extend, but not always repeatability. Because the interpretation varies slightly for different people. People are not repeatable as computers are.

Many tools are available for handling repeated tasks. One such example is cron. It allows us to schedule tasks to run periodically. We use cron to automatically backup data everyday at midnight, or to generate reports at the end of every month. We use makefiles to compile projects. It runs all the tests before build and compiles the project automatically with a single command.

Another great example is build automation. We use Continuous Integration Servers to deploy code to production servers. We use API doc generators to generate documentation automatically from source code.

Automation is inevitable. Let the computer do all the repetitive, mundane tasks. We've got better things to do...


- summary of Ubiquitous Automation, from The Pragmatic Programmer: from Journeyman to Master

Pragmatic Teams (Part 2)

Orthogonality

Traditional team organization is based on the waterfall model. The team includes individuals who are assigned roles based on different job functions. This is more strict in some cases, where one set of people are not allowed to talk to another!

This is a great mistake. It is a misconception that different aspects of development such as analysis, design, coding and testing can happen in isolation. The decisions taken in such cases may not be accurate.

Organize Around Functionality, Not Job Functions

Dividing based on functionality favours in many ways. When there customer wants to change the database, only one team gets affected. The sense of ownership increases when the team is aware of the fact that only they are responsible for that functional aspect.

Automation

A great way to achieve consistency and accuracy. Why do all the work manually when your editor can do a lot of things for you? Automation is an essential part of every process. We'll discuss about that in detail in the coming section


- summary of Pragmatic Teams, from The Pragmatic Programmer: from Journeyman to Master

Pragmatic Teams (Part 1)

So far, we have seen so many techniques that can help you as a better individual. But the value and outcome of these techniques are multiplied manyfold if you are working in a pragmatic team. Here are some of those techniques we already discussed in terms of teams:

No Broken Windows

Remember the broken window which you were too lazy to fix? It can happen to teams also. It is difficult for a pragmatic programmer if she joins a team which doesn't care about the quality. Some teams apply a quality manager, to take care of the issues. This is absolutely ridiculous! Pragmatic teams know that quality is the result of contributions from each and every member in the team.

Boiled Frogs

You might as well remember the frog which doesn't notice the gradual change in the water and ends up cooked. Being concious and mindful about all the varying aspects is not always easy, even for a team. Sometimes occurs from the assumption that someone else is already looking into the issue.

Fight this. Make sure that everyone monitors the change. If not, appoint a chief water tester, who constantly checks for variations. It is not necessary that you have to reject the change. At least, you should be aware of it.

Communicate

Good teams communicate well! Customers love to talk to such teams because their meetings are well organized. Discussions are live and the team speaks with one voice.

Don't Repeat Yourself

Chances of duplication is more in a team. A good project librarian can help in such situations. If a team member is looking for something, she must know that she has to talk to this person first. What if the project is too large for one person to handle? Divide it based on functional aspects so that one person can handle one particular aspect of the entire project.


- summary of Pragmatic Teams, from The Pragmatic Programmer: from Journeyman to Master.

Don't be a Slave to Formal Methods

There are so many well-described software development methodologies available: from structured development, CASE tools and waterfall to UML and object orientation.

Remember this. never get addicted to any of them!

It doesn't mean that formal methods are bad. But considering one without putting in your effort to analyse based on the context and developmental practices is a waste.

Don't Be A Slave To Formal Methods

Formal methods have some serious shortcomings:

• Most of the formal methods use diagrams to capture requirements from the user. No matter how simple it is, most of these look like an 'alien image' to the user. Finally, what the user understands is the designer's interpretation of it. Use prototypes whenever possible and let the user play with it.
• Formal methods seem to encourage specialisation. One group of people work on one diagram, another group on another.. and it goes on. This leads to poor communication. We prefer to be together and love to understand the whole system we are working on.
• We like to write adaptable, dynamic systems that allows us to change the character of applications at run-time. Most of the existing methodologies are incapable of doing that.

Should we completely avoid formal methods? No, but analyse before you use. Look at the methodologies critically. Never underestimate the cost of adopting new tools and methods. And also, never think about the cost of the tool when you look at its output. Extract the best out of all.

- summary of Circles and Arrows, from The Pragmatic Programmer: from Journeyman to Master

Specification Trap

Program specification is an important part of software development. It acts as the bridge between requirements and program development. It is also an agreement with the user - a contract which ensures that the final system developed will be in line with the requirement.

The problem is, some designers find it difficult to stop. They will not be happy until they pin point every single detail of the problem. This is a big risk. Why? These are the reasons:

• No matter how well you try, different people will have different perspective of the requirements. The user may not get 100% of what he/she really asked. Changes at the last stage will always be there.
• Pen is mightier than a sword. Languages are powerful. But there are cases where a natural language cannot explain what we need. Don't you agree? Then try writing a set of instructions for this problem: How to tie bows in your shoelaces? You may stop before finishing. Even if you complete it, other people reading the instructions may not get what you meant. Here's a different saying: Sometimes, it is easier done than said!
• Programmer, who is implementing the requirement might have a better idea to do it. Writing down every detail is like restricting their ideas and creativity.

These reasons don't imply that specifications are bad. There are cases where clear and detailed specifications are required, depending on the working environment and the product you develop.

Rely not only on requirements. Always go for prototypes or tracer bullets. Because, sometimes, it is easier done than said!

- summary of Specification Trap, from The Pragmatic Programmer: from Journeyman to Master

Wait for the Right Moment

Imagine the picture of a lion hiding behind bushes to catch his prey. He is waiting.. waiting for the right moment to jump on to the prey for a perfect catch.

Well, that was a horrible example! It's just to convey this idea: There is always a right time to begin. Learn when to start and when to wait.

We are developers. Listen to the inner voice that whispers to us ‘wait’, before jumping on to the keyboard and start writing the code.You might have a nagging doubt in our mind. Listen to it before proceeding. This may not always be a big deal. Sometimes, it could be just a disturbing thought. Give it some time. It will crystallize into something more solid. Or it will solve the great mystery that prevents you from continuing.

There is always a starting trouble while beginning a new project. But, how do we know if reluctance to start is a good decision, or we are simply procrastinating?

There’s a technique answer the question and solve the problem. If you are not sure when to start or how to start, simply take a part of the problem which you think is difficult to solve. Start developing a prototype for it, which doubles as a proof of concept. This will result in either of the following:

• You’ll feel that you are just wasting our time. No need to think twice. Throw away the prototype and start working on the actual code.
• You’ll find that there’s something wrong with the basic premise. You'll also get to know the right way to start with. Your instincts were right. Now you can launch the project in the right direction.

Prototype always helps. This is much more acceptable than simply announcing, ‘I don’t feel like starting’.


- summary of Not Until You’re Ready, from The Pragmatic Programmer: from Journeyman to Master

Solving Impossible Puzzles

Many times, we have stumbled upon a problem which seems impossible to solve. They look like a hard nut to crack, but are they really? Sometimes, the solution to the problem lies elsewhere, in a different path. We make ourselves limited by imaginary constraints. Separate them out. Hand pick the ‘real’ ones from the pool of all limitations and constraints.

We use the term ‘think outside the box’ to recognize and eliminate constraints which are not applicable in our problem. But, if the box is the boundary of our constraints and conditions, the real challenge is to find the box! The box could be larger than what we think it is.

Don’t think outside the Box - Find the Box

Next time, while encountering a difficult problem, iterate through all possible solutions. Some solutions may seem stupid. Think twice before ruling them out. Remember the Trojan Horse story? How do you get the troops into a walled city without being discovered? It’s sure that they discarded the ‘through front door’ option as it was a crazy idea to die fast.

So, every time you get a solution, think once again. There must be an easier way.


- summary of Solving Impossible Puzzles, from The Pragmatic Programmer: from Journeyman to Master

Requirements (Part 2)

Documenting Requirements

Now, we have idea about the requirements of our user. As professionals, we would like to write them down so that we don’t miss anything from our valuable user. We have to publish a document that can be used as a basis for discussions between developers, the end users and the project sponsors. There are different ways for this.

Swedish computer scientist, Ivar Jacobson proposed the concept of use cases to capture the requirements. Use cases help us to describe a particular use of the system in an abstract fashion. But his book was a little vague, which created different opinions among different people.

One way of looking at use cases is to emphasize on their goal driven nature. Templates can be used as a starting place. Use of a formal template ensures that we include all the information needed in a use case.

Over Specifying

Never be too specific, at least in the case of requirements! Good requirement documents remain abstract. They consist of simplest statements which reflect the business need clearly. But also make sure that the requirements are not vague.

Requirements are not architecture, Requirements are not design, nor are they the user interface. Requirements are need.

- summary of Digging for Requirements, from The Pragmatic Programmer: from Journeyman to Master

Requirements (Part 1)

We use the term, 'Requirement Gathering' everyday. This usually implies a group of analysts collecting the requirements from the user. 'Gathering' forces us to think that requirements are already available, we just need to find them.

But in practice, it is not exactly true. Requirements are rarely seen on the surface. They are buried deep beneath the layers, hidden from the eyes of the analysts or even the user. Perhaps this is not the right one, but it's better to remember Steve Job saying:

"People don't know what they want until you show it to them"

Digging for Requirements

The hardest part is to recognize a true requirement while digging through the dirt. For that, we need to know what a requirement is. Let's start with a basic one:

"An employee record may be viewed only by a nominated group of people"

This is an example of a good requirement. Why? Because this implies something that the system requires in a general way. Let's dig in further. Consider the following:

"Only an employee's supervisors and the personnel department may view that employee's records"

Is this statement truly a requirement? Perhaps today. But what if the policy changes tomorrow and somebody else can also view an employee's record? Then we'll have to re-write the requirement. But first example applies even after the policy change. It is always important to know why the users are doing certain things rather than knowing how they are currently doing it.

What's the best way to get inside our user's requirements? The answer is simple: become a user. Sit with the user while she's doing her job. Think and understand from the perspective of the user. We'll see how the requirement collection changes from a mere boring task to something remarkable.

Work with a User to Think Like a User


- summary of Digging for Requirements, from The Pragmatic Programmer: from Journeyman to Master

Evil Wizards

Wizards are always great! From the time when we started using computers, they take care of many things like software installations and removals (‘The wizard will now install your software!!’).

Wizards also help us in coding, especially while working with IDEs. They write us initial code for setting up a project, or depending on the situation at hand. We can use wizards to create server components, implement Java beans and handle network interfaces - all areas where it’s better to have expert help.

But, never let the wizard to be evil! The code wizard generates may not be required for our program, or may be wrong depending on the circumstances. Be aware of what the wizard writes for us. If we don’t understand the code written, then the program control is not in our hand.

Don’t use wizard code you don’t understand.

This code will also eventually become a part of our application. Always understand what’s there in our program at any point of time.


- summary of Evil Wizards, from  The Pragmatic Programmer: from Journeyman to Master

Unit Testing (Part 3)

Writing Unit Tests

Unit tests for a module shouldn’t be located far away from the module. In small projects unit tests can be included in the module itself. But for larger projects, it’s better to move them into a separate directory. But why do we say that it should be closer? Because, anything that’s not easy to find will not be used!

There are two advantages of doing this:

• It gives some examples on how to use all the functionalities of the module.
• A mean to build regression tests for validating the future changes in the code.

But providing unit tests is not only enough. We have to run them often, every time when we make changes to our code. It helps us to confirm that the class passes its tests once in a while.

Test Harness

When we write a lot of test code, it’s always better to have a standard test harness to create a testing environment. This helps to make our life easier.

A test harness can be implemented in the same language used for coding, or using makefiles and scripts. It can handle common operations like logging status, analyzing output for expected results and selecting and running the tests. It can also be GUI driven.

No matter however it is implemented, a test harness should have the following capabilities:

• A standard way to specify setup and cleanup.
• A method to select individual tests or all the tests.
• A method to analyze output for expected or unexpected results.
• A standardized form of failure reporting.

Tests should be always composable. That means, a test can be composed of subtests of subcomponents to any depth. In this way, we can select the complete test, or individual parts of the test depending on the requirement at that time.


- summary of Writing Unit Tests & Test Harness, from The Pragmatic Programmer: from Journeyman to Master

Unit Testing (Part 2)

Testing Against Contract

We like to write test cases for a module to ensure that the modules honors its contract. This will tell us

• whether the code meets the contract.
• whether the contract means what we think it means.

We test the module to check whether it delivers the functionality it promises, over a wide range of test cases and boundary conditions.

How do we define contract? For example, let’s take the factorial of a number problem. It’s contract can be defined as follows:

• Pass in a negative argument and ensure that it is rejected.
• Pass in an argument zero and ensure that it is accepted (boundary condition).
• Pass in an argument between zero and upper limit and ensure that the correct factorial is returned.

We have to test all these scenarios to ensure that our factorial module is working.

Why do we go to all this trouble? We are investing time and effort to write test cases. All we want is to avoid ‘time bombs’ in our code that sits unnoticed and blows up at an unexpected moment later in the project.

Suppose our module uses a linked list and a sort algorithm, we have to test the linked list, sort and the module completely. If the linked list and sort pass and the module test cases fails, we’ll know that the problem is with the module. This helps to reduce the debugging time also.

While designing a module, always design both its contract and the code to test that contract.


- summary of Testing Against Contract, from The Pragmatic Programmer: from Journeyman to Master

Unit Testing (Part 1)

While writing code, we have to build testability into our software from the beginning itself. This helps us to test each piece of our code before integrating them together.

Unit testing helps us to do this. This is done on each module of our application. Unit testing is performed isolating each module from one another so that the functionality of each module can be tested separately.

Unit testing establishes an artificial environment and invokes the module under testing. It checks the result returned by the module and compares it against a list of known values or against the results from previous runs of the same test. Unit test ensures that in a complete system, individual parts work as expected.

- summary of Unit Testing, from The Pragmatic Programmer: from Journeyman to Master

Refactoring (Part 2)

How do we Refactor?

Refactoring can be triggered by many factors such as deeper understandings, changes in requirements and so on. But never try to break into large quantities of code and refactor. We’ll end up in a much worse position than we are.

Refactoring has to be done slowly, deliberately and carefully. Below are some guidelines for refactoring given by Martin Fowler:

• Don’t try to add functionality and refactor at the same time.
• Have good tests before refactoring. It’ll help us to detect if anything breaks.
• Take short steps.

While refactoring, fix the code and everything depends on the code. This will a pain. But it’s going to hurt more later.


- summary of Refactoring, from The Pragmatic Programmer: from Journeyman to Master

Refactoring (Part 1)

As the program develops, we will have to rethink earlier decisions and rework certain portions of the code. Code is not static. It needs to evolve.

Rewriting, reworking, and re-architecting code is collectively known as refactoring.

Refactoring.. When should we do that?

Following are some scenarios where we need to refactor our code:

• Duplication - when we detect a violation of DRY principle.
• Non-orthogonal design - when we discover that the design can be much more orthogonal.
• Outdated knowledge - when the knowledge about the problem domain increases.
• Performance - when we need a much better performance than existing.

Refactoring is not always easy. We have to go through the existing code and modify it without affecting the functionality. Many developers are reluctant to do this because their code is mostly fragile.

Time is another reason for not refactoring. But the reality is that, if we fail to refactor it now, we might have to spend much more time later for fixing bigger problems.

Refactor Early, Refactor Often

- summary of Refactoring, from The Pragmatic Programmer: from Journeyman to Master

Algorithm Speed (Part 3)

Algorithm Speed in Practice

We may not be dealing with sorting or searching algorithms more often in the real life. But there are situations where we need to think of estimation. While encountering a single loop, it’s easy to identify now that we are dealing with a O(n) algorithm. It is O(n×m) if it has a nested loop.

Estimate the Order of Your Algorithms

If we have an algorithm of order O(n2), we can try to bring it down to O(nlog(n)). If we don’t know how long it takes, the easiest way is to test with different set of inputs and plot a graph. With around 3 - 4 points in the graph, we’ll be able to estimate the order of the algorithm.

But this is not always the case. A simple O(n2) algorithm works better than O(nlogn) for smaller values of n. At the end of the day, what really matters is how long our code takes to execute with real data in the production environment. So, always

Test Your Estimates

In some other cases, the fastest is not always the best to do the job. We have to make sure that the algorithm is apt for our problem, before going any further.


- summary of Algorithm Speed, from The Pragmatic Programmer: from Journeyman to Master

Algorithm Speed (Part 2)

The Big O Notation

The O() notation is a mathematical way of dealing with approximations. We say that the worst case time complexity of an algorithm is O(n2). It means, for n records, the time taken for the algorithm to run is in the order of square of n. We consider only higher orders while estimating time complexity. For example:

O(n2 + 2n) = O(n2)

The higher order dominates other values while n varies. Since we are eliminating lower order terms, one O(n2) algorithm can be much faster than another O(n2) algorithm.

- summary of Algorithm Speed, from The Pragmatic Programmer: from Journeyman to Master

Algorithm Speed (Part 1)

We estimate the time taken to complete the project, or the time taken to complete a particular task. There is another kind of estimation: estimating the resources used by an algorithm. This includes time taken to complete the algorithm, processor and memory consumption etc.

This kind of estimation is always important. Resource estimation is used to know how long the program takes to run with a particular set inputs. This also helps us to understand how the program scales for large number of records, thereby letting us know which all parts of the code need optimization.

How do we estimate algorithms? 

That is where we get the help of big O notations.

Estimating Algorithms… What does that mean?

Algorithms work with variable inputs: sorting takes an n element array, matrix operations require an n×m matrix etc. The size of the input affects the running time and amount of memory it takes.

But why do we need to estimate algorithm speed? Because the rate at which the execution speed increases is not always linear. An algorithm which takes one minute to process 10 records may take a lifetime to process 1000!

Big O notations allow us to perform a more detailed analysis.

- summary of Algorithm Speed, from The Pragmatic Programmer: from Journeyman to Master