Why Agile Doesn’t Work And 6 Ways You Can Fix This

Why Agile doesn't work...

According to the developers, this set of guiding principles should lead to highly-skilled software development and boost the motivation levels of engineers and product managers. However, after analyzing the work of engineers in more than 500 organizations – conducting surveys and experiments – we found that what’s happening in practice is wildly different.

In a normal (non-Agile) operation, processes and equipment are viewed as driving forces and a means of effective collaboration. We looked at a Fortune 500 business where product managers and engineers communicate exclusively using their tools, which they use to form working teams. What we found was an efficient and motivated workforce.

On the other hand, one large Fortune 100 company told us, "we’re not allowed to question the Agile process" – a statement which suggests a rigid, non-cooperative model, as opposed to an “agile”, flexible operation.

Oddly enough, in most cases, written documentation outperforms software. For example, in one large tech firm, a product development team decided to take time to write down small requirements – “user stories” – ahead of project kick-off. These were placed in the application queue as an example task for the next available engineer to start working on.

The method proved so successful that it’s now become one of many small “waterfalls”, where work is transferred from the product department to designers and engineers. This kind of process is exactly what Agile tried to eliminate. The company’s technical director said, “my engineers feel like professional chefs preparing orders in a restaurant”.

The last Agile principle “responding to changes as planned,” has been misinterpreted (by Agile teams) as not needing to have a plan in the first place. In one fast-growing tech company, the Agile team didn’t attempt to understand the organization’s broader strategy. As a result, they focused on low-value or strategically unimportant functions.

In fact, many engineers operating under Agile now consider it inappropriate to have a time frame or common milestones. Without a well-defined plan, there’s no way of knowing how to prioritize and invest in actions.

It would be great if these principles really did improve the technical motivation and productivity of a workforce but, in reality, the opposite is true. When engineers work on Agile, their motivation levels nosedive because they lose their sense of inclusion and excitement. Teams aren’t allowed to experiment, manage their work or communicate with their customers. Instead, they feel only emotional and economic pressure from their management teams. This stifles their ability to adapt, learn and drive their own work forward.

It doesn’t matter if you work as an engineer or a manager, you can make changes in order to strike a balance and increase the motivation and productivity of your team.

6 essential principles for maintaining a motivated and productive team

1. Software development is a collaborative process

By dividing processes between individual team members – without any management participation – you can never create a cohesive team. To achieve a truly productive team, instead of segmenting individuals into their separate processes, you should aim to create the feeling that all employees are partners in the work. And this feeling needs to carry through the entire development lifecycle.

From the outset, the team and its leaders should define a set of strategic goals. And if questions arise during the work, think of them as an important team assignment that provokes thought. All project tasks should be developed and executed by the entire team, including its executive sponsors (and customers).

At the core of any collaboration is the notion that each and every person involved can offer their ideas and suggestions, whenever they want. When you try to solve problems as a whole team, you’ll have a far better chance than companies using Agile-type schemes.

2. The team’s delivery department should conduct minimally viable experiments

Many teams feel they waste a lot of time on adaptation. To combat this, it’s essential to form ideas/hypotheses for strategic tasks, then try to solve them using mini experiments to see what works for clients. Experiments should be performed quickly – within a week, ideally – to promote swifter decision-making and reduce “dead time”.

3. Focus on the client

When you’re creating software (both for internal and commercial use), you should be fully focused on your customers.

The basic principles to consider are:

• “Problems” are always related to customer experience.
• Problem-solving meetings always start with client analysis and representatives of the entire team should participate in these discussions.
• Each experiment should be based on a customer-oriented hypothesis and the team should take responsibility for the result.

Using a customer-centric approach gives engineers a view of the product through the eyes of the customer. Product development workshops are a great way to get everyone on the same page during the early stages of product development, as is PoC development – which allows you to test client-oriented hypotheses and focus your development efforts.

Ultimately, to get the clearest picture of how a product affects a client, the whole team needs to work together – “with”, not just “for”, the client.

4. Use time frames to focus experimentation and reduce loss

Interestingly, adaptive software development encourages time scheduling as a way to provide experimentally-sound investments, signalling an acceptable level of quality for a given function. Hazardous conditions – i.e. projects that lack clear time frames – create emotional pressure for developers. This kills motivation and undermines confidence.

So it’s important to learn how to make mistakes and, more so, how to embrace them as part of a learning curve. The greater the uncertainty in the team and the higher the risk and, thus, the shorter the runway needs to be. That’s why breaking projects down into defined time frames tends to increase overall motivation and productivity.

5. Equip your team to focus on collaboration

To ensure a cohesive, collaborative and well-structured team, all the different stakeholders need to function as a whole – a method known as ‘pod’. Each team should contain a full set of experts. These might include executives, front engineers, interior designers, designers etc.

Many organizations set up "fake groups" – teams that appear to be working together but, in reality, are comprised of individuals working to their own goals and agendas.

Signs of fake structures include:

• Employees and managers are in a separate “agreed upon” teams. For example, the development team has special engineering “sprint teams”. This is not a group, but, as mentioned above, a team working as Agile.
• The team uses tools that hinder real collaboration. For instance, when one team of engineers was asked why they chose to use Agile software tools, they answered: "because the tools don’t allow managers to participate in our work." Obviously, this perpetuates a cycle of mistrust.
• The engineering and functions of the product have different goals. The managers of both groups use their influence to force employees to prioritize the goals of their specific function over all else. This ultimately leads to conflicts within organizations and blocks real teamwork.
• Rigid hierarchy within teams. Performance ratings, titles, promoting aspirational career growth, promotions and demotions etc. – destroys the teamwork necessary for the modules to work properly. This, in turn, leads to lower demand for the products.

In other words, the more an organization tries to fit its employees into rigid boxes, titles and roles, the more self-focused its employees will become. This makes it tough to build consensus.

6. Constantly question process

Conway’s Law is a well-known principle of engineering design. It argues: any organization that develops a system will create a project whose structure is a copy of the communication structure of the organization (i.e. the process). In other words, if you’re a close-knit team, the projects you produce will be “one”, or holistic. If you focus on user segments, your product will be optimized for this structure. That is, everything is interconnected.

To prove Conway's law wrong, you should continually adapt your structure and processes to deal with the problem in front of you. To do that, you need groups with simple and easy processes and structures that can be rapidly installed and configured to face each new challenge.

In conclusion, instead of relying on Agile frameworks as the be-all and end-all, engineering teams should continually diagnose and solve problems within their processes. In some of the most successful cases, teams diagnose their operating model on a monthly basis, then decide whether making changes to it could, ultimately, produce a better product.