Myths and truths of collocation in software development

The benefit of collocation is enormous. This doesn't just dramatically reduce the communication cost but also tremendously helpful to avoid confusion and improve team bonding. This is becoming a norm in the software development work where agility is new buzzword in the town. But how this collocation is implemented that actually is more important than just cramming people into a giant blob of space which can make more harm than benefit for the software development team.

Traditionally, the large non-software development enterprises are not designed for collocating software developers in a common space. So when they are asked by their Software development   department to create space, for instance, for their Agile software development team, the people from realty department free up a big enough conference room that can "just fit" the entire team. The expectation is, now the team would start producing the fruits of collocation. There are quire a few things missing in this expectation. Let's check them one by one:

1. Those (just freed up) conference rooms are built and organized for face to face communication in a meeting setting. A meeting is expected to run in terms of minutes, not hours. So there may not be enough space to place the computer on the table, chairs are not healthy enough for day long seating, not enough lights and ventilation for 10 to 15 people, and the list continues.

2. A team comprised of people and people have certain biological and psychological needs. People needs enough leg room, yawning space, etc. without bumping to each other. Except few people, most of the people needs off-face time. Otherwise the side effects of 24/7 cable news channels pops up.

3. Creativity needs both the conscious and subconscious minds to tango. With only the conscious mind in play, only so much can be achieved but to really come up with a good idea and solutionize that idea, the subconscious mind needs to be triggered which need a relatively quiet and private space. Specially, this is absolutely true for software programmer. They need the interaction with other fellow programmers to discuss and bring new ideas and then go back to relative isolation to concretely thought through that idea to make it to work. In lack of that, you will see people are putting on their earbuds to cancel the surrounding noise where no physical isolation is available.

4. Believe it or not, people can't work, specially people who are in creative business like software development, continuously for hours after hours without breaking away from that monotone. It works in a certain situation and setting but majority time it needs social break. So, if we force people to be in the crowd all the time, we may get lines of code that just do the work but not a breakthrough solution that will down the total cost of software development and maintenance.

I am not necessarily suggesting that people needs to be mile apart from each other but at the very least we need to recognize the need of balancing the collocation with personal space to get the best out of collocation.

Triangulation in software development requirements

I can bet that almost every software development project faces with a dilemma of questioning some of the business requirements that actually come from the very business that you are trying to systematize. This is arguably the most dangerous zone for any software development manager to get into as your suggestion would inevitably be considered as if you are questioning the business people's core skills. But you can't just accept it by its face value without critically analyzing the requirements. After all, in the current business environment, neither the business nor the technology independently drive the company's future. Now the sixty four thousand dollar question is, how would you do it in a methodical way? The answer lies in the word "Triangulation". This exists in other disciplines for long time and can be used in software development management.

Let's see a usage of Triangulation in other discipline. Triangulation is used by your Global Positioning System (GPS) device. There are at least three GPS Satellite that sends the positions of your device that help the GPS device to accurately pin point your location within a few feet range. 

Diagram 1: Framework for Triangulation of business requirement justification

You can use the same methodology to check if a business requirement is justified to be taken into the project. Take a typical business software project. Use the following triads to justify the business requirements: (1) Project Business Objective, (2) Minimum Viable Product (MVP) and (3) Total Cost of Ownership with perpetual cost of maintenance

(1) Project Business Objective is the first filter you have to let the business requirement go through. If the requirement doesn't align or contradict with the project's objective, the chance is very high that there are other motivation hidden behind the requirement that needs to be unearthed

(2) Minimum Viable Product is the Lean concept where it encourages the validated learning through early delivery of the viable software with a minimum set of features. Put that requirement to the test of MVP and see if it passes or fails

(3) Finally, there's no cost free feature in a software. Even for a feature that's working without adding a single line of code can have high maintenance cost. There are several types of cost, such as: now you have to make sure that in every upgrade you do or every new feature you add, the no-code-feature has to be tested to make sure that it functions as is; if you find a better technology/algorithm to implement in that software which would stop that no-code-feature to function, then you either have to stick to the inferior technology/algorithm or now add code to make that no-code-feature to work

Finally, never forget to identify the true need of the business (sometime referred as "problem statement") behind that requirement. You would often find that the true need may be actually vastly different than the very requirement is trying to solve and the requirement may be a solution that the business person came up with to solve that true need. Knowing the true business need would allow you to come up with a true solution of that problem which may be much simpler to implement. Undoubtedly, the simplest solution is the best solution and you would need a complex thought process to come up with that simplest solution.

A super brief note on Blockchain and Silver Bullet

The Blockchain is everywhere and the promise of the technology has inspired a big number of technologist and business executives to jump on this bandwagon. Yes, the promise of the the Blockchain technology is huge but it may be oversold specially around it's non-repudiation and decentralize nature of security. The defense is made that it's impossible, or not practically feasible, to break the chain due to its nature of linking the prior blocks using the hash and so forth. Similarly about the non-repudiation characteristic of the technology.  

I no way claim that I know how to break this or it can be broken today but I know for sure that there's nothing called absolute secured technology in the computing field. It's all about relative security where we call a system secured if its security can't be broken in relatively short period of time that makes it vulnerable. We should definitely invest big in block chain, not because of considering it as the panacea but because it is superior in the area of non-repudiation and digital identity, etc. features while keeping in mind that some time, some smart programmer will break this using a superior algorithm or a unexpected breakthrough in computing power.

How to reap the best out of Agile

As we have debunked the myth of the Agile methodology, let's look into how we can get the best out of an Agile team. There are volumes of books, training and certifications that are available on building and running Agile teams but here is a shortlist of  best practices (or you may call them as principles, if you will) to build a successful Agile team that produces the most value to its cost.

[A quick disclaimer: for simplicity, this article is focused primarily on SCRUM Agile development methodology as the term "Agile" is more familiar than "SCRUM". So, one can use the word "SCRUM" and "Agile" interchangeably while reading this article]

1. Hire Team members who believe in Agile

Eric Schmidt, the former CEO of Google, has said in his book, "How Google Works", that Google doesn't transform a person into problem-solving ninja but the company attracts those ninjas and they build the great products at Google. So, if you want to get your Agile  team to excel, hire the "smart creative" people who believe in Agile. Agile is a cultural mind set, not just a mere methodology that can be memorized. You can't turn or train people to become Agile unless they have that "Agile" gene deep into them. But you can tune them in Agile, if that is what they believe in. 

Now, let's talk in a practical term. Neither all companies are Google nor you would have freedom to pick your team at all the time. So, at the bare minimum, when you don't have that liberty to build from the scratch, before building a team for Agile, train them in Agile. Really and seriously! Train them in Agile!

2. Get business buy-in...this is the master key of Agile success

In my "Myth and Truth about Agile..." post, I have explained the importance of having the business people fully engaged in the Agile process. Without the full participation of business, there's almost no chance of running the show in Agile. There are occasions where the Software Development team claims that they are following Agile without the business participating in it which is kind of oxymoron and hard to believe.

So if you can not secure the buy-in of business into your Agile process, it's better to use other non-Agile methodologies, such as Rational Unified Process (RUP) or any other iterative method, except Waterfall. Believe it or not, those non-Agile methods work just fine when practiced in a consistent manner.

3. Avoid the quick sand of Architectural and Design soundness

Let's not use the Agile as an excuse for a poor Architecture or a closed design. Yes, the Agile doesn't let the team to spend a lot of time to focus on a robust architecture and comprehensive software design rather emphasize on working software. It asks for minimal upfront design and to continue to design over the period of product development life cycle. The crux of the game is "refactoring". The refactoring has to be done both in code and design level. This means continuous rework and occasional throw away code. If you are not ready to accept that notion of refactoring, may be Agile isn't fit for your project. 

There's a catch in that minimal upfront design through, which we forget often. The notion of “minimal upfront design” in Agile requires that the developers are skilled to design, and architect and the Designer or Architect codes. Your organization needs to be ready to invest on people who poses that level of skill set. 

4. Still you do Release Planning

Even though Project Management principles are in direct conflict with Agile principles but there's fair amount of planning involved in Agile projects. What you have in traditional project management as Work Breakdown Structure, in Agile you would have Product Backlog with User Stories. Through Affinity Sizing the product backlog would give a sense of project duration which would be broken down in to Releases based on business priority, value and dependencies to get Agile Project Scheduling.

Some Agile practitioners feel very uncomfortable with this planning aspect of Agile that's based on affinity sizing. Affinity sizing is done at high level of understanding with minimal detail. This lack of comfort is not the fault of those people but the fault lies in for not utilizing the affinity sizing with right intention. The affinity sizing can no way be used as the basis for commitment but the user stories have to go through multiple passes of refinement and re-sizing over the period of time to come to the level of confidence to make project commitments. The Cone of uncertainty in software estimation has to be kept in consideration when making commitments because "you can't beat the Cone of uncertainty but you just can be more lucky"

5. Do the right Sprint planning

Task, Task and Task....this is crucial that team members create the tasks for each user stories!

One of the objectives of the Sprint planning is to create tasks for the user stories. The purpose of keeping the Sprint shorter is to ensure that the developers can plan in detail and the manifestation of a successful planning is "lot of tasks". Human brains are not good at keeping every detail for next few months and that's why the Sprints are recommended to keep between two to four weeks. This is a very crucial tool for a successful Sprint.

Yes, the first objective is to create the Sprint Backlog (scope) for the upcoming iteration but the more important objective is to go into the next level to each of the stories and break them into tasks (tasks should be no more than that a day, and smaller tasks than a day is preferable). Effectively 80% to 90% tasks should be created in this planning meeting and put on the story board for anyone to pick up. There will be some unidentified tasks for sure, but that volume shouldn’t be more than 10% to 20%, and those would be identified during the sprint period.

At the end of the Sprint Planning, the story board should be filled with newly created tasks with the status of “Not Started”. Otherwise, it’s an indication that the stories are not groomed enough or the team doesn’t know what to do with that story. Either of them are not a good sign for a productive Agile team.

6. Delivery early and quickly

Agile is all about working software. So it is pretty obvious to have continuous delivery of working software. I would like to emphasize on it in light of Architectural components that usually takes more time to build at the beginning of a project. But it's better to keep that working software concepts even for the architectural components. 

For example, if there's framework to be built for workflow, messaging, enterprise integration, etc., it's better to plan to deliver the smallest piece of that working architectural components with minimal functional user stories (much like a Prototype but a working piece of software that would be the basis of further development). This is also aligned to the Lean concept of "fail fast and early" so that the recovery cost is low and manageable. Apart from early proof of concept, this task orientation also helps to rally the team around the delivery and gets the team in performing stage quickly. 

7. Track and remove impediments effectively

The Sprints are short. So it's the responsibility of the organization to put a framework to track impediments and remove them as quickly as possible. The agility can't be achieved if the team is playing rat and mouse game with the organizational process instead of developing working software. It sounds easy and every organization will pledge to this principle immediately as soon as you ask for it but it's actually much bigger than just giving a pledge. The entire organization needs to be oriented to Agile culture to server an Agile team effectively.

8. Use visuals everywhere and anywhere

Human brains are not naturally built for reading lots of texts and interpreting the progress status of the team from text is not intuitive. Also, as Agile is like a fast moving car running with pedal to the metal, the status of the team and product should be readable with a glimpse of an eye. So, keeping the status of Release and Sprint, in the form of burn down graph, in-front of eyes (or accessible with one click) is key. Every member of the team should be on the same page on where the team stands right at this moment. 

Kanban Dashboard is another nice tool to visually track the progress of an Agile team. Just by a glimpse at the dashboard, one can paint the right picture of the team's current state.

9. Realize the self organized team

It's easier said than done. Some time we don't even understand the breadth of the self organization aspect of team management. Managing a self organized team needs a whole new set of skill set and attitude that aren't easy for a traditionally trained manager. It would take another article to explain the managing of self organized team but in a quick nutshell: it's like you tell the team to get things done by not just doing that the "telling" thing.

Moreover, it's not that having self organized team is good to have but it's actually absolutely necessary to have the team self-organized due to the fact of business involvement and transparency in Agile. It's like you break the wall of the kitchen in your restaurant and allow the diners to come and talk to the chef and ordering their food while standing next to the cook in the kitchen. There's no place of hide and seek in an Agile team.

Myth and Truth about Agile software development methodology

The term "Agile" has now become close to a fad than true appreciation. You will see people brag about how "agile" they are in their software development organization. Not only that, I have also seen people use the term "Agile" to activity such as, project planning where no software development is involved at all. But who would counter them? You would be lectured by seasoned consultants by deconstructing word by word and how their method of creating a plan is a true agile process. On the other hand, people who claims to love Agile and live on Agile, will tell you how Agile development is much more superior than "waterfall method". By the way, who will tell them that between the stone age and industrial age, there was iron age and don't compare the industrial age with stone age by skipping the iron age civilization.

With all those, I felt obligated to explain the "Truth" of Agile software development methodology. Let's begin with a simple fact

Agile is for Software Development

Believe it or not, Agile is, and came for the software development. People may tell you otherwise by showing the linguistic deconstruction of noun vs. adjective and prove you that Agile can be used for your apple picking job as well. For the record: I had once done a "Agile snow cleaning" in a winter - just to have fun as I was tired of snow cleaning due to repeated blizzard that threw couple of feet of snow. It was just for a pure fun, no academic purpose was involved in that. But if anyone claims that Agile can be implemented anywhere, then I would argue that you can develop software using "Taekwondo" principles as well. Spinning can be endless.

Agile is not the successor of Waterfall methodology

As I had touched upon at the beginning, prior to Agile, Iterative methods were very successful and effective. Actually, if you ask me what method I would prefer: Agile Scrum or Iterative Rational Unified Process? I would tell you, pick any but be consistent

Agile is not a silver bullet

Business sometimes think that Agile is a silver bullet and solve every problem the business is facing today. Agile is rather a software development methodology that comes into existence to reduce the risk of software development and maximize the business value of software but it is not a panacea. Agile is not free of cost though. The cost includes: risk of loosing big picture of the solution, quicksand for architectural soundness, overemphasizing on working software over long term stability, fading documentation, etc.. Also, Agile demands certain creed without that it is hard, if not impossible, to be successful. Though not all the risks are due to the Agile practice but sometime due to the fact that people use Agile as escape goat. 

Agile needs business people to be involved...fully involved

When business wants the IT to be Agile, they often don't realize that the Agile is not methodology that IT will practice on their own and business will continue their existing operational model. Agile needs business to change or adapt to the Agile development model. Business flocks into Agile to get the fruit of transparency on how it's being cooked in the kitchen but to reap that fruit of Agile, business needs to play their part. It takes two to tango!

SCRUM is not synonymous to Agile

SCRUM is widely popular in the industry over other Agile implementation such as Test Driven Development (TDD), Xtreme Programming (XP), etc. So people who jumped late on the Agile bandwagon sometime confused SCRUM as the Agile process. SCRUM is just one implementation of Agile that has given a framework to manage Agile software development project very effectively. As SCRUM has taken Agile close to the business people (and also other Agile implementation are too technical for non-technical business people to grasp), they have taken the familiarization with Agile through that framework. 

Agile is a philosophy and demands cultural change

The Agile Manifesto talks about the philosophy behind the Agile Software Development methodology. As this doesn't dictates one on how to achieve those principles but this can be with a level of certainty that without changing the culture in the software development, it is impossible to achieve the principles. I have seen instances where business sprinkles the principles of Agile software development on engineering team without emphasizing the underlying cultural change and blames Agile for their failure.