There are times in our lives when the stories through which we constitute our identity no longer make sense. Illness, bereavement, divorce, redundancy, new job, promotion, relocation, falling in love, marriage, parenthood and other major life events can cause us to question the familiar stories we tell ourselves and others about who we are.

Many of us experience something like this at some point in our lives. At such times our identities are threatened. They can become particularly malleable and open to change but also liable to be frozen defensively. And it’s not just individuals who are affected in this way; the stories that sustain groups, societies, nations, and whole civilisations also collapse.

Deep ecologist and Earth-scholar Thomas Berry once said:

It’s all a question of story. We are in trouble just now because we do not have a good story. We are in between stories. The old story, the account of how we fit into the world, is no longer effective. Yet we have not learned the new story.

Put another way, the old story of the industrial-growth society has passed its sell-by date. We know that it no longer works (indeed that it has only ever worked for a privileged minority) and many of us crave a different kind of social, business and political leadership that is more focused on helping us find new and healthier stories than on propping up the old one.

We are already – collectively and individually – searching for new stories but it seems clear that we can’t go from the old story to the new story without experiencing a dark night of the soul, a place of narrative wreckage, a shifting, confusing time of emptying that will allow something else to emerge. It’s also questionable that we should be looking for the new story at all.

We should not underestimate either the importance or the difficulty of this essential task. It is at times such as these, when old certainties are collapsing, that we are at our most vulnerable to manipulative stories that blame others and promise simplistic solutions.

As John Michael Greer points out in The Long Descent, our situation is not a problem that can be solved but a predicament to be lived through. It calls for grown-up, post-heroic leadership and responsible, adult citizenship. Only then will we end our attachment to the old story and learn how to live well in the “multi-storied” space that exists somewhere between the apocalyptic story of catastrophic collapse and the utopian story of socio-technical or back-to-the-land solutions.

In story terms, the danger is that we will grab on to yet another heroic narrative, hoping that we can put things right with a bit of magic, luck and courage. Hollywood abounds with such environmental blockbusters: The Day After Tomorrow, Avatar, The Core, 2012, to name but a few. They are entertaining adventures but have little to teach us. It’s time to slacken our grip on the Hero’s Journey (always a male archetype) and look to other types of story to guide us. I find myself asking: what stories would we tell if we considered our purpose to be neither destroying nor saving the world but learning to live more beautifully in the world that we have?

Perhaps our stories might resemble more closely something like that told in the award-winning Beasts of the Southern Wild, which shows the very plausible consequences of melting icecaps, massive storms and rising sea levels on a group of misfits, outcasts, and exiles living below the radar in ramshackle huts and boats on the wrong side of the levee in southern Louisiana in a community they call the Bathtub. The film is a blend of magical realism and gritty drama that above all champions the importance of community. People struggle to survive but never forget to celebrate the extraordinary adventure of being alive, finding beauty even in the midst of death and disaster.

It is easy to become dispirited and overwhelmed by the difficulties that surround us; it is so much easier to pretend that they do not exist or to believe that it’s not worth doing anything because nothing we can do will make any difference. Yet we need not lose heart; there is much joy to be found and much good we can do by living more simply and more harmoniously with our planet. We might not be able to change the story in one fell swoop; nevertheless the stories we tell and the stories we live do matter.

This blog first appeared in Guardian Sustainable Business Blog 30 April 2014

Photograph: c.20thC.Fox/Everett/Rex Feature