Professional anxiety emerged as the main barrier to progression in recent research commissioned by Badenoch and Clark, the leading professional services recruiter. The research covered over 1,000 professionals in middle management and above, 38% of whom highlighted anxiety as holding back their own progress.

Whilst the reported levels of anxiety among men and women were much the same, the variation between small, medium and large organisations is striking.

In small companies, as many as 35% of employees reported that no support structures were in place to support staff in reaching senior levels. Similarly, 43% said that anxiety was a career obstacle. 16% of medium-sized companies had no support structures with 33% of staff feeling anxious. In large companies, just 13% reported that no support structures existed with 36% reporting anxiety.

The figures reported for medium and large organisations raise the question whether, as companies grow in size and HR sophistication, the returns on investment in support structures diminishes.

First let’s step back to get a perspective from 20 years ago. Tom Peters had written Liberation Management: Necessary Disorganization for the Nanosecond Nineties.

He argued that stable bureaucratic organisations with hierarchies and reliable career ladders were headed the way of the dinosaurs, whilst talent would thrive in a state of permanent flux. Important work would get done in small, autonomous, project-oriented teams, in the way world-class teams of directors, actors and technicians form and disperse to make movies.

Peters thought that project work and the professional services firm, where there is little career structure (beyond whether you make it to partner) was the future of work, with individuals self-managing their own personal brands.

Organisations and bureaucracy have proved a tad more durable than Peters suggested, but he had a point.

Yet in holding up the movie production team or the professional services firm as a model for the future, recurrent anxiety is built in. These are workplaces where you’re only as good as your last job (or your last quarter’s billings). Many potential movie stars will spend a lot of time serving in restaurants.

The latest research from my Cass colleague professor Laura Empson underlines that professional service firms are hives of insecurity where even the firm’s leaders wrestle all the time with the ‘fragile’ and ‘contingent’ nature of their authority.

Anxiety is not unique to professional service firms – some psychologists even study all organisations as defences against anxiety – but they do give it particular expression.

If Peters is right, this is where the future is headed anyway. Executives should grin confidently, realise that if they play their cards right, power belongs to them more than it belongs to organisations, and leap confidently from one organisation to another building their personal brand, networks, career and pension fund as they go.

On this analysis organisations can do very little to reduce anxiety. They simply don’t control the future, their own or anyone else’s, as was once believed. Some organisations may even think that in this sort of world, it is in their interest, so far as they can, to keep their people anxious and a bit foggy about their careers.

In many cases that will be an unwise judgement. Yet even where the judgement is well-founded, organisations should not get complacent about how anxiety may be blocking their progress: witness the idea that organisations should incorporate risk appetite as a systematic dimension in talent management.

A current movie can teach us something about professional anxiety. As well as winning numerous awards, the strong commercial performance of Alfonso Cuarón’s 2013 film starring Sandra Bullock suggests it struck a chord with people’s experience. The movie could easily have been called Anxiety rather than Gravity. Let’s look at the movie as an allegory about today’s workplace.

A thoroughly modern career person ‒ a professional and a mother ‒ navigates a wide open career space, in which the possibilities are theoretically limitless, and as a result both awesome and terrifying.

From this perspective, the film starts with exactly the kind of transient, high-performing team which Tom Peters praised. With expert skills and the latest equipment at their fingertips, working in highly sophisticated yet at times casual co-ordination, the select few create value for the millions back on Earth, while enjoying vistas which the rest of us will never see.

Until, of course, it all falls apart. Sandra Bullock’s Dr Ryan Stone faces a dystopian struggle in space, where she flees desperately from one space station (or organisational survival vehicle) to another. She cannot survive without them, but their ability to protect her is limited.

The conventional HR response to such anxiety is to focus on skills. If organisations equip their executives with high quality skills, then they will be able to survive in a wide range of environments outside those organisations. This means that executives don’t need to be too anxious even if career visibility within their own organisation is limited.

But the movie is not so much about skills as stark aloneness. That strikes a chord with my own coaching work with senior executives. The higher executives climb, the narrower – sometimes to the point of vanishing – becomes their circle of career support: individuals with whom they can share confidential anxieties about their career and who understand enough about their business world to grasp what those anxieties mean.

This is true even of executives with bulging diaries and LinkedIn contacts, because many things rule people out from possible membership of that executive’s circle of career support: clients, subordinates, competitors or superiors have to be impressed; the anxiety-provoking issues are too sensitive to discuss at an industry or supplier social event; high technical complexity blocks meaningful conversation with a sports or pub friend or a family member.

Aloneness may well be a contributing factor to career anxiety. If so, organisational responses based solely on developing skills or recognising achievements may show diminishing returns.

It is clear that although anxiety is a key challenge for both organisations and their employees, there is no one correct way to come to terms with it. Neither employer nor employee should expect one organisation, or each employee on their own, to solve it all.

For employees it is wise to manage actively their circle of career support, not letting it dwindle away, and steering a middle course between the high confidence of Tom Peters that the future is fun and belongs to individuals and their personal brands, and the high anxiety of Gravity.

For employers, Badenoch and Clark’s research suggests that different sizes of organisation face different versions of the challenge. Small organisations should not neglect putting in place visible career support for their talent. Actively helping key executives find mentors in larger, non-competing organisations can help, as can selective investment in business school open programmes. The latter not only teach skills, they also help individuals benchmark themselves against a wider population, and extend their circle of career support.

Medium-sized organisations are doing better than other employer groups, perhaps by providing some structured support, with a large enough employee group for individuals to have supportive, non-conflicted relationships in the workplace. But anxiety is still a challenge. For the most senior or talented individuals, periods of coaching linked not only to particular business goals but also intended to strengthen the individual’s confidence about where their career is going should pay dividends. It is difficult for internal coaching to provide the same career reassurance as a widely experienced outsider.

Larger organisations might want to be aware of potential paradoxical effects. They probably have considerable investment in HR procedures and apparently no shortage in a large organisation of potential colleagues, peers and internal mentors for executives to draw on; and executives’ diaries may be crammed full with meetings. Yet the effect of rank and hierarchy, rivalry between business units and other factors may leave rising executives unexpectedly alone in terms of career support. Coaching can help but the field is wide open for other creative innovations.

For example, while large organisations have become allergic to funding places for their executives on open business school programmes, at more senior levels single company programmes can lack diversity and the scope for executives to gain an outside window on their skills and careers.

To overcome this challenge, two or three organisations of comparable scale and global reach, but in non-competing sectors, could combine in a leadership development consortium with a business school provider. Rather than dilute leadership values to a lowest common denominator, such a programme should send participants back to the workplaces with a sharper, less platitudinous understanding of what their company stands for.

For the employees, this could well help not just overcome anxiety but also gain a safe space to extend their circle of supportive career relationships outside the politics of their own company.

Dr Douglas Board, career coach and senior visiting fellow, Cass Business School, is the author of ‘Choosing Leaders’ (Gower 2012).