Quite a bit of our time at the Confederation is spent defending NHS organisations and management from various types of attack. Much of this is ill informed and in the worst cases based on little more than the opinions of columnists' dinner party guests.

There is a good deal of evidence to suggest that NHS management at its best is as good as the best in other sectors. We also have the testimony of people coming into senior management roles from the private sector that the complexity of health care and the challenge of running an organisation populated with so many professional staff is, if not unique, particularly challenging. However, one of the effects of being continuously attacked is that a siege mentality can develop. The key to development and improvement is the chance to have time for honest reflection and learning to identify weaknesses and areas to work on. The culture of the NHS in the last few years has not been conducive to self-examination.

As we move to a more devolved system it is going to be even more important to take a critical look at NHS leadership and management. We asked a range of leaders to do this and they identified a number of areas where NHS leaders could do better. Some of these are about the technical skills required to do the job, in particular, the use of numerical data, strategic planning, working with local authorities and, of course, commissioning. Mike Farrar, chief executive of NHS North West, agreed with the need to improve technical skills but was very clear that the new world also demands a whole range of sophisticated ways of dealing with the emotional side of organisational life that are probably even more important. He says many managers need to pay much more attention to developing and using some of these so-called softer skills. Far from being soft, these are actually some of the toughest areas of management. 'These skills are essential if we want to create sustainable leaders ... it's what differentiates the great from the good because it requires people having honest conversations about really difficult issues.' He believes that these skills will be vital for the complex challenges now facing NHS management in particular, the need to change behaviour amongst clinicians, the users of services and even the public.

A second area where there is scope to improve is in the use of evidence to inform decision making. This includes more careful and rigorous analysis of arguments for changes and the use of evaluation to learn from previous decisions. This was highlighted as a potentially important element of improving the legitimacy of management with clinicians who are often suspicious of the lack of rigour in NHS decision making.

The people we spoke to thought that there was often too little leadership in evidence and they were also concerned that there was a significant weakness in middle management in many organisations.

There are a number of factors that get in the way of the development and exercise of successful leadership. Most people we spoke to recognised the need for leaders to think in terms of systems rather than just the interests of their organisation and to focus more on the medium and longer term. It was generally felt that the design of many aspects of current accountability arrangements and of system reform, particularly PBR, tended to create insularity and short termism. The very significant workload and a general feeling of overload that many leaders experience may also have contributed to these problems.

Whilst some of these issues relate to the leaders and managers themselves, and the solutions are in their own hands; quite a lot of them are a product of the system in which leaders operate. The most notable source of problems seems to be related to the way the hierarchy operates. Whilst hierarchy does have some significant advantages it also comes with major problems. It hinders and distorts communication in both directions and introduces times lags into decision making. There is an unavoidable tendency for people at the top of hierarchies to assume that they have a clearer view about what needs to be done and to underestimate the difficulties of implementation. Whilst it is quite likely that they have a different view, it is not necessarily better.

Even where hierarchy is effective there is a question about the extent to which it is scalable and whether complexity and co-ordination costs out weigh any benefits gained from increasing scale. For example, at what point does the system get so large that values become meaningless slogans? Large scale means that small errors multiply into major problems. It may also mean that policy makers tend to concentrate on making policy that at best is aimed at the average PCT or Trust, but often appears to be targeted at the worst performers.

Perhaps the most pernicious effect of hierarchy is the creation of a dependency culture. This is particularly true in a risk averse setting such as the NHS. If the culture works by top down direction and boards and local leaders are expected to look upwards for instruction and performance management then it is likely that this will create disempowerment and even a failure to pay attention to some key areas - because they are being dealt with elsewhere in the hierarchy and often because the hierarchy is directing their attention elsewhere. The more the dependency culture develops the more the upper levels of the hierarchy need to intervene.

Devolution to the front line and increased autonomy for trusts are clearly parts of government policy. The anxiety is that hierarchy tends to be very resilient and to reassert itself at the first sign that things are not going to plan.

The arguments for increasing autonomy are partly the reverse for those against hierarchy but there are some additional aspects worth rehearsing. Greater autonomy tends to be associated with improved problem solving, innovation, increased responsiveness, reduced stress and higher levels of discretionary effort. But, the potential for too much autonomy to create chaos is considerable. The crude view of markets being based on adversarial relationships that seems to underpin some views of the new system is a caricature of real markets in complex and dynamic environments in which collaboration and competition co-exist. Autonomy therefore has to be accompanied by a clear set of outcomes and objectives, responsibility to the wider system, appropriate behaviours and levels of effort contributed by each organisation and leadership.

There is much to be proud of in NHS management but it is important that we are not complacent or afraid to point out where it is failing to deliver or not living up to the values that should underpin it. There has not always been enough willingness to do this. Leadership development is important but not enough; the environment in which leaders operate also needs to change and it is this rather than debates about independence that is likely to really make a difference. The Foundation Trust movement is a major step in this direction. We need to think about what the equivalent would be for PCTs.

The challenges of leadership in the NHS will be published at the NHS Confederation annual conference, 20-22 June

Nigel Edwards
Director of Policy
NHS Confederation

Published in HSJ on 21 June 2007