The stoutness strikes you immediately. The James Caird was built as a whaler, and whales were not hunted in anything flimsy. She is about 23ft in length. But when you know of the voyage she was asked to undergo, the Caird suddenly starts to look very, very small indeed. Picture three men lying on a bed of rocky ballast, chasing sleep in the bow. Another mans the tiller, and two more busy themselves chipping salt ice off the slithery makeshift deck, or pumping water from below. Meanwhile, the James Caird pitches and tosses through the dark canyons and beetling cliffs of the South Atlantic Ocean, the most treacherous stretch of water on the planet, through 800 miles and 17 days. Tortured by thirst, these half-dozen men are buoyed by only the faintest hope of arrival: navigation is all but impossible and, with the wind ripping into their feeble sails, they drift on knowing that if they miss the minuscule island they are aiming for they will be swept out to the open ocean, inviting certain death not only for themselves but also for the 22 men they have left behind, marooned on a remote crag in the Antarctic.

The James Caird's journey from Elephant Island to South Georgia is regarded as one of the greatest sea voyages ever made. With tight-lipped understatement, Sir Ernest Shackleton described it as 'a tale of supreme strife in heaving waters'. And here the boat stands, at the National Maritime Museum in Greenwich, where it is on loan from Dulwich College. Two years ago, a retired railwayman called Roy Vincent went to inspect the Caird at her permanent home. There was a drinks party, and few of the other guests twigged that his father, John, was one of the hardy seamen who had been on the Caird in her finest hour. 'Nobody seemed in the slightest bit interested,' he says, 'so I kept my head down. I wandered about the exhibition and just wondered how the heck the Caird made it.'

But that journey is only one part of the remarkable story of the Imperial Trans-Antarctic Expedition, which set out from England in August 1914 with the declaration of war ringing in the ears of all aboard, to make the first crossing of the Antarctic continent. It is a story of survival and comradely fortitude in the face of extreme danger and hardship.

Despite the loss of his ship Endurance in the ice, Shackleton managed to shepherd all his men back to safety through a two-year odyssey. Only now, as film makers fall over one another to bring it to the screen, is it coming to be seen for what it is: a tale of polar heroism that is much more in sympathy with our times than the romantic, vainglorious spurning of life orchestrated by Captain Robert Falcon Scott. For most of the past century, Scott was held up as the paragon of British polar exploration. Since Roland Huntford's 1979 book, Scott and Amundsen, exposed the hero as a bungling amateur who put glory above survival, a pedestal has stood empty. And yet when Huntford's biography of Shackleton was published in 1985, an American reviewer wondered who would read a life of a man nobody had heard of. That is no longer the case, and his time has come. 'The world being what it is now,' says Huntford, 'the country finally lost its taste for the elegiac, sacrificial hero. It prefers survivors. Shackleton, of course, was the great survivor. He is really a Homeric figure.'

It took American interest to recognise Shackleton's achievement for what it is, perhaps because he seems to embody practical American virtues more than Scott's doctrinaire English ones. He was Anglo-Irish, a bit of a hustler, and essentially classless. First came a beautifully illustrated book called The Endurance, by the New York writer Caroline Alexander. A touring exhibition lionised Shackleton in all corners of the United States. Two business guides emerged, offering 'leadership lessons' based on Shackleton's way of handling those under his command.

Camera crews ventured to Elephant Island and South Georgia to capture the breathtaking scenes of Shackleton's triumph for both an Imax film (released this autumn) and a two-hour documentary shown on Channel 4 last year. Meanwhile, Wolfgang Petersen, the director whose seafaring movies include Das Boot and The Perfect Storm, is working to get Endurance, his own Hollywood account of the expedition, off the ground with Russell Crowe in the lead role. 'It's not a flame that needs keeping,' says Alexandra Shackleton, who has inherited her grandfather's tenacious appearance. 'Every time I think it has peaked, some extraordinary new manifestation appears: Nasa teaching astronauts the Shackleton way; Shackleton as an icon for business executives...'

A new Channel 4 production will be screened on January 1 and 2. With a budget of £10.5m, Shackleton is the most expensive British television drama ever made. Written and directed by Charles Sturridge and produced by Selwyn Roberts, the team who made the award-winning Longitude, it has the whiff of pedigree about it, and it stars Kenneth Branagh as the expedition leader. Most impressively of all, the four-hour drama captures the terrible beauty and solitude of the frozen end of the Earth, as much of it was shot in testing conditions on pack ice off the coast of Greenland.

Ill prepared and underfunded, the expedition was, in its stated aim, an abject failure. The Endurance left South Georgia on December 5, 1914, carrying a crew of 28 men - sailors, scientists, surgeons, an artist and a photographer - and a team of 69 sledging dogs. Thanks to her reinforced hull, which was designed to withstand the abrasions of the ice, she was able to nose through the pack to within about 80 miles of land, but by January 18, 1915, the floes had closed around her and held her fast.

Shackleton had no choice but to spend the long Antarctic winter immobilised as the floe drifted back northwards in the Weddell Sea. By September, the grip of the ice was such that the Endurance could endure no more. The crush bent, lifted, tilted and eventually squeezed her, and in November she sank through a hole in the ice. By this time Shackleton had long given up hope of crossing the continent, and had hastily set up camp on the floe. They tried to haul three of their four lifeboats towards the edge of the ice, but progress was so slow that they chose instead to sit tight until the onset of summer made channels in the ice.

In April 1916, after slaughtering and eating the remaining dogs, they clambered into the lifeboats and rowed for seven days across more than 100 miles of bitterly cold sea towards Elephant Island, off the northern end of the Antarctic peninsula. No man had set foot there. When they made land for the first time in 497 days, they were overwhelmed by hunger, thirst, exhaustion, frostbite and the dawning knowledge that nobody would think to look for them in this snowy desolation of crags.

Given the demoralised condition of his men, after a few days' rest Shackleton took the only option available, foolhardy though it was: to modify and reinforce the biggest of the three boats, take five men and make a dash to the whaling stations of South Georgia. Frank Worsley, the captain of the Endurance, was able to take only four readings with his sextant in mountainous seas, and even when they did sight land they were overwhelmed by a storm that all but drove the Caird against the cliffs. Land they did, but on the wrong side of the island. Two men were close to death, and the whaler was beyond use. Shackleton again took the only option: to lead his two most trusted colleagues over the unmapped inland range of South Georgia, riven by crevasses and peaks. The 22-mile crossing took 36 hours. Three times they clambered up towards gaps between a wall of high peaks, only to turn back when confronted by sheer drops on the other side. Eventually they had to slide down a steep slope of 1,500ft to reduce their altitude before nightfall.

During one rest, Shackleton's companions fell asleep. He woke them after five minutes, telling them they had slept for 30. When they staggered into the whaling station Stromness, they were unrecognisable, ghosts of the men who had left South Georgia 17 months before. A blizzard, which they would not have survived, blew that night.

It took Shackleton four attempts, on four borrowed boats, to reach Elephant Island; three times, the ice held him back. When he got there on August 30, 1916, 128 days after the Caird set sail, it was to find every one of his men still alive.

'In memories we were rich,' he later wrote in his memoir, South. 'We had pierced the veneer of outside things. 'We had suffered, starved and triumphed, grovelled down yet grasped at glory, grown bigger in the bigness of the whole.' We had seen God in all his splendours, heard the text that nature renders. We had reached the naked soul of man.'

The key to an understanding of Shackleton is that he was no member of the establishment. The son of a doctor, he was an erratic businessman, an indifferent father to his three children, and for several years had an American mistress. 'Of course he wasn't without his flaws,' says Alexandra Shackleton. 'But the flaws do get ridiculously exaggerated. He only had one girlfriend, and now he's a womaniser. It's not actually the same thing.'

His great strength as a polar explorer was that in the far south, these worldly cares dropped away. It greatly assisted him that his career in the Merchant Navy instilled a negotiable attitude to hierarchy. He was a member of Scott's 1901-04 expedition. In her book Terra Incognita, Sara Wheeler describes how Scott's quarters segregated him from the rest of the men. When Shackleton led his own Nimrod expedition of 1907-09, which got nearer to the pole than anyone previously, he also set up a hut, but there was no segregation. You can still gauge the social disparity in his group in the voices of the descendants. It's hard to imagine any common ground between, say, Roy Vincent, a retired railwayman, and Peter Wordie, a retired ship-owner. Of their fathers, John Vincent was a seasoned trawlerman and the ship's boatswain, whom Shackleton demoted for bullying; James Wordie was the expedition glaciologist, who later became president of the Royal Geographical Society and master of St John's College, Cambridge. Other men loathed each other, none more than Thomas Orde-Lees, a pernickety officer who avoided manual labour, and Henry 'Chippy' McNish, the ship's carpenter. Only death could unite them: they are buried in the same tiny cemetery in New Zealand.

And yet Shackleton's instinctive democracy allowed him to foster a spirit of community. Although racked by sciatica, he worked harder than anyone, while his gift for pastoral care and psychological insight allowed him to treat each man according to his needs. Where appropriate he would, in Huntford's phrase, play mother or martinet. He faced open mutiny only once: when, hauling the lifeboats across the ice, McNish declared that after the sinking of the Endurance he was no longer under Shackleton's command. It was McNish who made the Caird seaworthy for the great crossing, and he was one of the six men who sailed her. But on their return home, Shackleton refused to recommend either him or Vincent for a Polar Medal.

It is a narrative that nobody would dream of making up. But Shackleton's story was very quickly forgotten. He died in 1922 in South Georgia, on an ill-planned fourth trip to old haunts. It seems telling that Anne Fright, niece of his second-in-command, Frank Wild, and the only living relative of a survivor to have met the great man, has little memory of him. 'I met him at a garden party in north London. I was about four. Shackleton won a coconut, which he gave to me. I remember more the coconut.' The story was even lost to the children whose fathers came off the ice with Shackleton. Like many other offspring, Peter Wordie heard very little about the ordeal. 'I think it's very simple. If you go off to Antarctica and you come back alive while a lot of your friends have died in the great war, you feel guilty. Your expedition was a total failure. You achieve none of the things you set out to do.'

Other mementoes of the expedition survive unfreighted by negative associations. The South exhibition at Greenwich includes not just the James Caird, but artefacts from the Endurance, including the sextant, compass and chronometer used by the skipper Worsley to plot the crossing to South Georgia. Among the Shackletoniana in a polar auction at Christie's in September was the diary of the surgeon Alexander Macklin, which fetched £98,000. A tin containing remains of an Endurance expedition biscuit was bought by a museum in Co Kildare for £6,500.

Then there is the photographic record - movie and still - of the young Australian Frank Hurley. When the Endurance was already half submerged, he hacked into the ship's darkroom to retrieve about 500 photographic plates from 4ft of freezing water. With Shackleton he selected over 100 and brought them home, along with three rolls of film that he used on a pocket camera on Elephant Island.

This archive has helped to underpin Shackleton's memory while the reputations of other polar heroes have ebbed away. It has also helped to enthuse film directors about the visual possibilities of polar regions. But that isn't the only reason why maritime storytellers such as Sturridge and Petersen are magnetised to Shackleton. In his quest to get the expedition off the ground, they see their own job description writ large. 'The thing that drew me to him,' says Sturridge, 'was not the expedition but the setting up of the expedition. The parallel was so extraordinary: having an idea that no one else really believes in and having no backing, and then ending up with these pottier and pottier sources of finance. And then casting it and persuading those people that it's going to be like this and setting off and doing it. The thing about the disaster is one assumes that's where the story starts. What you realise is that he was completely exhausted and absolutely out of his head before the adventure started. And that's when I began to get interested in him, because it became much more real.'

The notion of filming Shackleton's story was put to Sturridge when he was making Longitude. The immediate question, once he'd agreed to do it and identified Branagh as the only actor he could think of to play him, involved authenticity: how to film cost-effectively on the ice in a way that looked real but didn't put cast and crew at risk. He and the producer, Selwyn Roberts, considered various options: Norway (too expensive and not flat enough), Baffin Bay in Canada (too inaccessible), Franz Josef Land (distances too great). Eventually they plumped for the west coast of Greenland, which promised ice floes large enough to film on, and a mountain range that could stand in for the peaks of South Georgia. A Norwegian icebreaker was hired to house the 100 or so cast and crew who spent a month there in June of this year. At the same time the Kaskelot, a ship dressed up with sails and steam to play the Endurance, was sailed up from Milford Haven to Reykjavik, where some filming was done, and then on to Greenland.

As soon as the production got to Greenland, they suddenly knew how Shackleton and his men felt. The ice never behaves predictably. 'It was like working on the moon or Mars, because you were in this hostile, fantastically beautiful, transient-by-the-hour environment,' says Sturridge. 'You can't construct or plan. The environment you worked in was different on Monday and different on Tuesday. When we got up in the morning, we never knew what we were going to do.'

Within hours of arriving in the ice, the icebreaker got stuck. Sturridge proceeded to break two safety guidelines: don't work on small floes at the edge of the ice, and don't do the small boat sequences with actors. 'I said, 'I thought this was an icebreaker.' The captain said, 'This is much too tough, this ice, we can't go anywhere.' I said, 'We'll get off the ship and shoot.' 'You can't shoot. The ice floes are too small.' I said, 'Look, mate, we're a factory, we have to shoot every day.' So we're off the ship, on a small floe, partly just for me to prove to everyone that we can carry on working. The next day we got out of the ice, but there was very little we could do, so we had 42 people on the boats. It was calm, but we were on the bit everyone said we shouldn't be on. We had 34 days to shoot, and every day we didn't shoot we wouldn't get another day.' They were due to film one of the expedition's encampments on a huge floe, which promptly split at precisely the point where they were moored. Ocean Camp was pitched instead in a studio at Shepperton.

While thrown together in the relative comfort of the icebreaker, cast and crew were granted a microcosmic view of the way Shackleton's disparate party of seamen and scientists was forced to cohere in cramped quarters for months on end. According to Branagh, it helped them edge just that little bit closer to an understanding of what the experience might have been like: 'Shackleton's fascination was with group dynamics in unusual situations. You put people in an odd environment and have them living at very, very close quarters 24 hours a day, in this case surround them by a force that's somehow saying, 'Think about yourself, think about what's back home, think about why you were more depressed than you've ever been this morning and you laughed the laugh of such happiness this afternoon.'

You read about the expedition, and that intensity of experience was all par for the course. I remember all the boys being on open boats in the middle of an icy sea. Other support boats had gone miles away because it was a helicopter shot. The helicopter was stuck somewhere, so we were on our own rowing on the Arctic Ocean and talking to each other across the boats. There was a tiny bit of nervousness in the air because the radio had fallen in the water. An extraordinary experience - I felt completely and utterly alive and joined to these guys at the hip. Everyone came back and had a big drink back on the boat.' Even food gave them a key into the psychology of survival. 'We understood after a while why there were riots about the size of your seal steak. At four in the afternoon, a few chocolate bars came out on that ice, and if they were not distributed in an egalitarian fashion it was a very, very ugly atmosphere for a while, from sane people going back to a warm boat quite shortly.'

At times in the shoot there was genuine danger. Most of the actors had gone home, but three stayed behind to shoot the inland crossing of South Georgia: Branagh, Kevin McNally, who plays Worsley, and Mark McGann, who plays Shackleton's rock-like Irish henchman, Tom Crean. Time was running short, as it did for the three men in their dash across the island. In case they were snowed in, the crew had gone up with tents and 48 hours' worth of supplies. The daunting moment came when the three actors stood at the top of the sort of slope that their characters had once, in dire emergency, been forced to slide down. 'There just wasn't time or money to have three stand-ins do it, and we all had signed up to it,' says Branagh. 'We had been brought to the top and it had to be virgin snow. Suddenly there was complete silence behind me. And then I suddenly thought, 'F me, that is steep.' You were reminded of what Shackleton and the boys considered: do you think there will be a big rock just below the surface of the ice that may kill us? Or, this may be masking some sort of crevice. We did it, and the sense of exhilaration - we were thinking, 'You just don't get the chance to do this.'' To this day, even with infinitely better equipment and without the gruelling preamble of a 17-day sea journey, nobody has beaten the record of a 36-hour crossing of South Georgia.