Ruby Under a Microscope

Recently, I started reading Ruby Under a Microscope, by Pat Shaughnessy. Being a Ruby programmer for the last three years, I felt this is a good time to start reading it. To give you a brief idea, this book explains what happens behind the scenes while executing a Ruby program. I started just out of curiosity and without even reading any reviews online.

I am currently reading the second chapter of the book. The amount of information the book has packed into each of the chapter is really astonishing. In fact, I was able to give a 45 mins talk to my fellow programmers at Multunus just with the first chapter!

The book starts from the moment when you execute a ruby script with the command:

ruby your_script.rb

From there, it is a walkthrough on how ruby reads, tokenises, parses and compiles the program into something the underlying Ruby Virtual Machine understands. For me, this was a pragmatic recap of compiler theory from my college days. What I learned then was just theory. Here, Pat shows the actual source code which does what is meant to be done! I got a new perspective on the basic concepts like tokenising and grammar parsing just by reading the first chapter. This book gave me courage to go through the actual ruby source code written in C!

If you are a Ruby programmer, and if you care about fundamentals of computer science like compiler theory, don’t think twice. You should read this book. As I told already, I am in the second chapter of the book as of now. So far so good. I will post a detailed review with my learnings once I complete the book.

Being Agile

This is a bit complicated than it sounds. Agile is not a set of processes that one can blindly follow. Nobody practices Agile method. It's a way of thinking about software development.

Let's start with a more formal description called Agile Manifesto. Agile Manifesto is a collection of 4 values and 12 principles to guide us through the process of software development.

Rather than dumping everything here, I'll take you through the right resources to read about it. Here is the Manifesto for Agile Development and the Principles behind Agile Manifesto. To be agile, you need to put these values and principles into practice.

Agile Methods

Agile methods are processes that follows agile philosophy. They consists of individual practices like setting coding standards, using version control etc. Of course, these practices have been around for years. But agile refines the set of existing principles by handpicking those which are aligned to agile philosophy, discarding the rest and also by bringing in some new ideas.

Every team, project and situation is unique. You might want to create your own agile method by mixing together various agile practices. But, creating a brand new agile practice from scratch is a bad idea unless you have used agile development for a prolonged period of time. Always start with an existing, proven method and refine it in lieu of starting one from scratch.

- learnings from the Art of Agile Development

Tower of Hanoi

Tower of Hanoi is a recurrent mathematical problem. In recurrence, the solution to each problem depends on the solutions to smaller instances of the same problem.

I'd solved this problem a few years ago. But now, since we talk about functional programming, I decided to solve the problem using Elixir.

Read more about the problem statement here: Tower of Hanoi.

If you want to try it interactively, go here.

Here is my solution:

defmodule TowerOfHanoi do

  def move(1, source, destination, intermediate) do
    IO.puts ["Move ", source, " to ", destination]
  end

  def move(no_of_disks, source, destination, intermediate) do
    move(no_of_disks - 1, source, intermediate, destination)
    IO.puts ["Move ", source, " to ", destination]
    move(no_of_disks - 1, intermediate, destination, source)
  end

end

I'm updating my GitHub repository (yedhukrishnan/mathematics) with more math problems. Feel free to check them out to suggest improvements or to give comments.

Functional Programming (Part 1)

I've been talking about Elixir for a while. It's never too late, but I believe it's high time to share what I learned about functional programming and why is it important.

Let's start with the fundamental difference. In an object oriented world, we solve the problem using classes and objects. A class depicts a behaviour while an object stores a state. While solving a problem, a lot of time is wasted modelling classes and hierarchies. But in a real world, we want to get things done than spending time maintaining states. This is what makes functional programming different. While the imperative style of programming concentrates on what and how to get things done, functional programming concentrates only on what. How is it being done, we don't need to worry about that.

In functional programming, we never mutate data. We just transform them. What's the big deal if we change some existing data? Here is the simplest explanation: We don't need to worry about data inconsistencies in a multi-threaded environment. It's always ensured that nobody is going to touch the data created by us.

Recursion (Part 1)

Unlike in imperative languages, functional programming languages (including Elixir) achieve looping through recursion. Why? To explain that, let's consider C:

for(i = 0; i < 5; i++) {
  sum = sum + i;
}

Here we are modifying (mutating) the value of i as well as sum in each interation. Mutation is considered as a bad practice and functional programming languages do not encourage it. Let's see some basic examples of recursion in Elixir.

Say, we want to print the message "hello" five times. How do we do that?

defmodule Recursion do
  def print_hello(n) when n <= 1 do
    IO.puts "hello"
  end

  def print_hello(n) do
    IO.puts "hello"
    print_hello(n - 1)
  end
end

Recursion.print_hello("Hello!", 3)
# Hello!
# Hello!
# Hello!

Here, defmodule is used to define a module (group of several functions). We can have multiple definitions of the same function. In the first function when n <= 1 is a guard. This function will get executed only when the guard condition is satisfied (no wonder why it is called a guard!). When there are multiple clauses (definitions) of the same function, Elixir looks for a match and executes it when it finds one. If no matches are found, it will throw error.

In this case, when n = 3 and n = 2, the second clause will be executed. When n = 1, the first clause will get executed. 

Read more:

• Modules
• Recursion

Elixir Language (Part 5)

Binaries

In the last section, we talked about representing strings as UTF-8 encoded binary. But we left a topic for this section: How a character having code point above 255 is represented in binary?

Well, we all know that a single byte can only hold a value between 0 and 255. Let's consider the string hełło. Here ł has a code point 322.

This is where UTF-8 comes into picture. It solves the problem by using 2 bytes to represent each ł, and one byte each to represent h, e and o.

iex> string = "hełło"
"hełło"
iex> byte_size string
7
iex> String.length string
5

Here, we can see that the byte size of the string is 7 while the string length is 5. The function byte_size/1 can be used to check how many bytes are actually used in the memory to represent the given string.

Read more: Binaries, strings and char lists

Testing (Part 2)

In the previous post, we've looked at what to test. Now it's time to see how to test. There are several different aspects:

• Regression Testing - Here, we test the result of current test with its previous results. This ensures that the bugs we fixed today didn't break the things that were working yesterday.
• Test data - There are two types of test data: real world data and synthetic data. Both are important, because each of these tests different aspects and behaviour of the system.
• Exercising GUI Systems - Testing GUI required special testing tools. Some of them record the events and play whenever required, while some others are based on a script. Writing code that is decoupled from GUI helps us to write better tests, because we can test the backend without interacting with the GUI.
• Testing the Tests - How do we test the tests? The easiest way is to introduce bugs intentionally in the tests and see if they complain. 
• Testing Thoroughly - How do we know if we have tested our code thoroughtly? The answer is we don't, and we never will. But we can use coverage analysis tools to keep track of it.

- summary of Ruthless Testing, from The Pragmatic Programmer: from Journeyman to Master 

Testing (Part 1)

Generally, developers hate testing. They hate to see their code breaking.

But we don't. We test every bits and pieces in our code. Why? Because we know that this is one place where we can see Butterfly effect in action! One simple mistake can cause large (and very bad, of course) effects in the future.

Test Early, Test Often, Test Automatically

Many teams develop elaborate test plans for their project. But automated tests, which run with every build is far better and successful than test plans that sit on a shelf. The earlier we find the bug, the easier the fix. In fact, a good project may have more test code than production code.

Just writing tests is not enough. We have to make sure that we run that often. We cannot say that coding is done unless all the tests are passing. These are some major types of test we need to perform:

• Unit Testing - Tests a module. Because if the module can't work alone, it cannot work with others as well.
• Integration Testing - Tests how the subsystems play and interact with each other.
• Validation and Verification - Ensures that what is build is what the user needs. Checks the functional requirements.
• Resource Exhaustion, Errors and Recovery - Tests how the system behaves in real world conditions under varying factors like memory, CPU bandwidth etc.
• Performance Testing - Stress testing, in which we tests our system and how it behaves when there is heavy load.
• Usability Testing - Different from all the above tests. Here we test the system with real users. It helps us to know how easy it is for them to use it.

- summary of Ruthless Testing, from The Pragmatic Programmer: from Journeyman to Master

Automation

When the first Model-T car was released, the instructions for starting the car were two pages long! Creeppy!! We get irritated if the car takes more than 5 seconds to start when we turn the key. A great example to show how we realised the importance of automation over the course of time.

Why do we need automation? It ensures consistency and repeatability. Manual procedures ensure consistency up to an extend, but not always repeatability. Because the interpretation varies slightly for different people. People are not repeatable as computers are.

Many tools are available for handling repeated tasks. One such example is cron. It allows us to schedule tasks to run periodically. We use cron to automatically backup data everyday at midnight, or to generate reports at the end of every month. We use makefiles to compile projects. It runs all the tests before build and compiles the project automatically with a single command.

Another great example is build automation. We use Continuous Integration Servers to deploy code to production servers. We use API doc generators to generate documentation automatically from source code.

Automation is inevitable. Let the computer do all the repetitive, mundane tasks. We've got better things to do...


- summary of Ubiquitous Automation, from The Pragmatic Programmer: from Journeyman to Master

Pragmatic Teams (Part 2)

Orthogonality

Traditional team organization is based on the waterfall model. The team includes individuals who are assigned roles based on different job functions. This is more strict in some cases, where one set of people are not allowed to talk to another!

This is a great mistake. It is a misconception that different aspects of development such as analysis, design, coding and testing can happen in isolation. The decisions taken in such cases may not be accurate.

Organize Around Functionality, Not Job Functions

Dividing based on functionality favours in many ways. When there customer wants to change the database, only one team gets affected. The sense of ownership increases when the team is aware of the fact that only they are responsible for that functional aspect.

Automation

A great way to achieve consistency and accuracy. Why do all the work manually when your editor can do a lot of things for you? Automation is an essential part of every process. We'll discuss about that in detail in the coming section


- summary of Pragmatic Teams, from The Pragmatic Programmer: from Journeyman to Master

Pragmatic Teams (Part 1)

So far, we have seen so many techniques that can help you as a better individual. But the value and outcome of these techniques are multiplied manyfold if you are working in a pragmatic team. Here are some of those techniques we already discussed in terms of teams:

No Broken Windows

Remember the broken window which you were too lazy to fix? It can happen to teams also. It is difficult for a pragmatic programmer if she joins a team which doesn't care about the quality. Some teams apply a quality manager, to take care of the issues. This is absolutely ridiculous! Pragmatic teams know that quality is the result of contributions from each and every member in the team.

Boiled Frogs

You might as well remember the frog which doesn't notice the gradual change in the water and ends up cooked. Being concious and mindful about all the varying aspects is not always easy, even for a team. Sometimes occurs from the assumption that someone else is already looking into the issue.

Fight this. Make sure that everyone monitors the change. If not, appoint a chief water tester, who constantly checks for variations. It is not necessary that you have to reject the change. At least, you should be aware of it.

Communicate

Good teams communicate well! Customers love to talk to such teams because their meetings are well organized. Discussions are live and the team speaks with one voice.

Don't Repeat Yourself

Chances of duplication is more in a team. A good project librarian can help in such situations. If a team member is looking for something, she must know that she has to talk to this person first. What if the project is too large for one person to handle? Divide it based on functional aspects so that one person can handle one particular aspect of the entire project.


- summary of Pragmatic Teams, from The Pragmatic Programmer: from Journeyman to Master.

Elixir Language (Part 4)

Binaries

In Elixir, strings are UTF-8 encoded binaries.

What is a binary?

A binary is just a sequence of bytes.  By default, each character in a string is internally represented in memory using 8 bits.

Let's take a simple example. The string "hello" in UTF-8 encoded binary form is 104, 101, 108, 108, 111. As you can see, each character uses a number (code point) between 0 and 255, which is represented with 8 bits or 1 byte.

You can define a binary using <<>> as shown below.

iex> <<0, 1, 2, 3>>
<<0, 1, 2, 3>>

We have the string concatenation operator <> in Elixir, which is actually a binary concatenation operator.

iex> "hello" <> "world"
"helloworld"
iex> <<0, 1, 2, 3>> <> <<4>>
<<0, 1, 2, 3, 4>>

Let's see "hello" in Elixir's binary form:

iex> "hello" <> <<0>>
<<104, 101, 108, 108, 111, 0>>

Here, I just used a concatenation operator to append a binary to a string. It converted the string to its internal binary representation.

The unicode standard assigns code points to many of the characters we know. For example, a has code point of 97. Here, as you can see, h has a code point of 104.

All commonly occuring characters have code points between 0 and 255. But there are characters whose code points are above 255, which cannot be represented in memory using a single byte. In the next post, I'll explain how they are represented and stored in memory.

Read more: Binaries, strings and char lists

Don't be a Slave to Formal Methods

There are so many well-described software development methodologies available: from structured development, CASE tools and waterfall to UML and object orientation.

Remember this. never get addicted to any of them!

It doesn't mean that formal methods are bad. But considering one without putting in your effort to analyse based on the context and developmental practices is a waste.

Don't Be A Slave To Formal Methods

Formal methods have some serious shortcomings:

• Most of the formal methods use diagrams to capture requirements from the user. No matter how simple it is, most of these look like an 'alien image' to the user. Finally, what the user understands is the designer's interpretation of it. Use prototypes whenever possible and let the user play with it.
• Formal methods seem to encourage specialisation. One group of people work on one diagram, another group on another.. and it goes on. This leads to poor communication. We prefer to be together and love to understand the whole system we are working on.
• We like to write adaptable, dynamic systems that allows us to change the character of applications at run-time. Most of the existing methodologies are incapable of doing that.

Should we completely avoid formal methods? No, but analyse before you use. Look at the methodologies critically. Never underestimate the cost of adopting new tools and methods. And also, never think about the cost of the tool when you look at its output. Extract the best out of all.

- summary of Circles and Arrows, from The Pragmatic Programmer: from Journeyman to Master

Specification Trap

Program specification is an important part of software development. It acts as the bridge between requirements and program development. It is also an agreement with the user - a contract which ensures that the final system developed will be in line with the requirement.

The problem is, some designers find it difficult to stop. They will not be happy until they pin point every single detail of the problem. This is a big risk. Why? These are the reasons:

• No matter how well you try, different people will have different perspective of the requirements. The user may not get 100% of what he/she really asked. Changes at the last stage will always be there.
• Pen is mightier than a sword. Languages are powerful. But there are cases where a natural language cannot explain what we need. Don't you agree? Then try writing a set of instructions for this problem: How to tie bows in your shoelaces? You may stop before finishing. Even if you complete it, other people reading the instructions may not get what you meant. Here's a different saying: Sometimes, it is easier done than said!
• Programmer, who is implementing the requirement might have a better idea to do it. Writing down every detail is like restricting their ideas and creativity.

These reasons don't imply that specifications are bad. There are cases where clear and detailed specifications are required, depending on the working environment and the product you develop.

Rely not only on requirements. Always go for prototypes or tracer bullets. Because, sometimes, it is easier done than said!

- summary of Specification Trap, from The Pragmatic Programmer: from Journeyman to Master

Elixir Language (Part 3)

The Match Operator

In Elixir, we use = operator to assign value to a variable:

iex> x = 1
1

What's the big deal? Well, in Elixir, = is not just an assignment operator. It's a match operator.

iex> x = 1
1
iex> 1 = x
1

As long as the left hand side and the right hand side are equal, Elixir will not complain. But when a mismatch occurs, it raises a MatchError:

iex> 1 = x
1
iex> 2 = x
** (MatchError) no match of right hand side value: 1

For assigning value to a new variable, the variable should always be on the left hand side of the operator.

The match operator can be used to destructure complex data structures into its components. Here is an example where Elixir uses pattern matching on a list:

iex> [a, b, c] = [1, 2, 3]
[1, 2, 3]
iex> a
1

Here is a common question that pops up in mind when we talk about the match operator. If we are using the expression x = 1 in Elixir. How does elixir know whether we want to do a match or assignment? 

By default, Elixir always performs assignment if the variable is on the left hand side. If you specifically want to perform match, you can use the pin operator (^).

iex> x = 1
1
iex> ^x = 2
** (MatchError) no match of right hand side value: 2

Here I explained the parts which I liked the most in Elixir. This is only meant for an introduction into the language. All the examples are taken from the Elixir's getting started guide. 

Read more and learn here: Elixir: Pattern Matching

Elixir Language (Part 2)

Here are the few things I liked about the language at the first glance:

Lists and Tuples

Square brackets are used in Elixir to specify a list of values. They can hold values of multiple data types. 

iex> [1, 2, true, :atom, 3]
[1, 2, true, :atom, 3]

They are represented in memory using linked lists. This means accessing the length of a list is a linear operation.

Tuples, unlike lists, store elements contigiuously in memory. They can also hold values of any type.

iex> {:ok, "hello"}
{:ok, "hello"}

Short-Circuit Operators: and, or

By default, and and or operators are short-circuit operators in Elixir. That means, the right hand side of these operators will be executed only if the left hand side is not enough to determine the result.

iex> false and error("This error will never be raised")
false

iex> true and error("This error will never be raised")
** (RuntimeError) undefined function: error/1

Here, the first expression returns false because 'false and anything' is always false. But in the second part, the left hand side is not enough to get the result. Hence, it tries to execute the right hand side, and it fails because there is no function called error/1.

In the next part, I'll share what I learned and liked about the match operator in Elixir.

Read more:

• (Linked) Lists
• Tuples
• Basic Operators
• Short-Circuit Evaluation (wikipedia)

Elixir Language (Part 1)

After listening to the Introduction into Elixir language, by Dave Thomas, in Bangalore Ruby Users Group meetup, I always wanted to learn the language, or at least functional programming part of it. I did a failed attempt the same day evening! Yesterday, I started again with the tutorials. I am sharing some of my learnings here.

Elixir Language

Elixir is a simple, easy to learn functional programming language build on top of Erlang VM. It is a dynamic language which allows metaprogramming and can be used for building concurrent, distributed and fault-tolerant applications with hot code upgrades.

Installing Elixir in Ubuntu/Debian

The only prerequisite for installing Elixir is Erlang, version 17 or later. Precompiled packages are available in Erlang Solutions website. Follow the instructions on their website to install Erlang. Then, install Elixir by cloning their repository and generating binaries:

$ git clone https://github.com/elixir-lang/elixir.git
$ cd elixir
$ make clean test

Make sure to add the bin directory to the PATH environment variable ( Add it to ~/.bashrc file for bash, ~/.zshrc file for ZSH ... )

$ export PATH="$PATH:/path/to/elixir/bin"

This is a distilled version of installation instructions for Debian/Ubuntu users (like me!). Refer the original version for detailed instructions.

In the next part, I'll share the basic constructs I liked in Elixir language on the first day of learning.

References: 

• Elixir: Getting Started Guide
• Elixir Primer and Resources
• Elixir (Programming Language)

Wait for the Right Moment

Imagine the picture of a lion hiding behind bushes to catch his prey. He is waiting.. waiting for the right moment to jump on to the prey for a perfect catch.

Well, that was a horrible example! It's just to convey this idea: There is always a right time to begin. Learn when to start and when to wait.

We are developers. Listen to the inner voice that whispers to us ‘wait’, before jumping on to the keyboard and start writing the code.You might have a nagging doubt in our mind. Listen to it before proceeding. This may not always be a big deal. Sometimes, it could be just a disturbing thought. Give it some time. It will crystallize into something more solid. Or it will solve the great mystery that prevents you from continuing.

There is always a starting trouble while beginning a new project. But, how do we know if reluctance to start is a good decision, or we are simply procrastinating?

There’s a technique answer the question and solve the problem. If you are not sure when to start or how to start, simply take a part of the problem which you think is difficult to solve. Start developing a prototype for it, which doubles as a proof of concept. This will result in either of the following:

• You’ll feel that you are just wasting our time. No need to think twice. Throw away the prototype and start working on the actual code.
• You’ll find that there’s something wrong with the basic premise. You'll also get to know the right way to start with. Your instincts were right. Now you can launch the project in the right direction.

Prototype always helps. This is much more acceptable than simply announcing, ‘I don’t feel like starting’.


- summary of Not Until You’re Ready, from The Pragmatic Programmer: from Journeyman to Master

Solving Impossible Puzzles

Many times, we have stumbled upon a problem which seems impossible to solve. They look like a hard nut to crack, but are they really? Sometimes, the solution to the problem lies elsewhere, in a different path. We make ourselves limited by imaginary constraints. Separate them out. Hand pick the ‘real’ ones from the pool of all limitations and constraints.

We use the term ‘think outside the box’ to recognize and eliminate constraints which are not applicable in our problem. But, if the box is the boundary of our constraints and conditions, the real challenge is to find the box! The box could be larger than what we think it is.

Don’t think outside the Box - Find the Box

Next time, while encountering a difficult problem, iterate through all possible solutions. Some solutions may seem stupid. Think twice before ruling them out. Remember the Trojan Horse story? How do you get the troops into a walled city without being discovered? It’s sure that they discarded the ‘through front door’ option as it was a crazy idea to die fast.

So, every time you get a solution, think once again. There must be an easier way.


- summary of Solving Impossible Puzzles, from The Pragmatic Programmer: from Journeyman to Master

Requirements (Part 2)

Documenting Requirements

Now, we have idea about the requirements of our user. As professionals, we would like to write them down so that we don’t miss anything from our valuable user. We have to publish a document that can be used as a basis for discussions between developers, the end users and the project sponsors. There are different ways for this.

Swedish computer scientist, Ivar Jacobson proposed the concept of use cases to capture the requirements. Use cases help us to describe a particular use of the system in an abstract fashion. But his book was a little vague, which created different opinions among different people.

One way of looking at use cases is to emphasize on their goal driven nature. Templates can be used as a starting place. Use of a formal template ensures that we include all the information needed in a use case.

Over Specifying

Never be too specific, at least in the case of requirements! Good requirement documents remain abstract. They consist of simplest statements which reflect the business need clearly. But also make sure that the requirements are not vague.

Requirements are not architecture, Requirements are not design, nor are they the user interface. Requirements are need.

- summary of Digging for Requirements, from The Pragmatic Programmer: from Journeyman to Master