Metallica - They're Back from never never land

Metallica are about to release their best album in years. Ben Ratliff joins the band on tour

JAMES Hetfield, Metallica's mordant singer and guitarist, is loudly practising vocal exercises. Lars Ulrich, the group's affable drummer, follows him. Then comes the band's long- haired surfers: the lead guitarist Kirk Hammett; and the bassist Robert Trujillo, the newest member, heavy-featured and mild-mannered.

The band are in a guarded outbuilding of Cotroceni Stadium in Bucharest, Romania, several hundred feet beyond the stands. A low-ceilinged room serves as a pre-concert practice space, with a green drum-kit, racks of guitars and basses, and equipment for digital recording. The band need their 20-minute warm-ups for physical preparation – they are all in their mid-forties now.

A photographer asks them to stand together. "Again?" Hetfield mumbles. "We did that in '84." Office humour; nobody laughs. The guitarists start playing entwined riffs and after ten minutes they move into Creeping Death, a song from 1984, tonight's opener. It is gothic early Metallica, written from the perspective of the tenth plague visited on Egypt.

"No new songs tonight," Ulrich says apologetically, as an assistant wraps gripping tape on his fingers. "I'm kind of new-songed out, to be honest."

The concert is what most fans probably wanted anyway: music recorded between 1983 (Kill 'Em All, the first Metallica album) and 1991 (Metallica, aka "the Black Album"), but nothing from the often-reviled second half of the band's career. There are flames and fireworks; the crowd chant and headbang all the way through a rainstorm. "You're going to sing as loud as you can?" Hetfield bellows before Seek and Destroy, the final encore. "You're going to make Metallica proud of Bucharest?"

Metallica will face the present soon enough, when they release their new album Death Magnetic next week. Produced by Rick Rubin, it is far better than anything they have recorded in the past 12 years; it sounds as if they have woken out of a daze. But it may also be seen as a regression, evoking the band's sound from the mid-1980s.

Metallica's music was athletic then, crazy with grim, loud ornament: Hetfield's death-fantasy lyrics, songs within songs, strafing and high-pitched guitar solos. But almost from the start, "progress equals integrity" was an article of faith. Each of their evolutions seemed to challenge hardcore metal's values of speed and power and emotional guardedness.

There was one apostasy after another: ballads, acoustic-guitar sections, the banning of guitar solos, the cutting of hair. Finally the group hired a performance coach – a therapist, more or less – who played a major role in Metallica: Some Kind of Monster, the 2004 documentary about their near-breakup.

Rubin found the film "brutal". It was the group at its worst, he says, "artistically and personally". He wants the band members "to try to erase many years of thinking about either needing to change their sound, or evolve. If your marching orders have been 'change, change, change', then letting go of those preconceived ideas is in its own way a new idea."

As Ulrich explains it: "Rick put this mantra over our heads, which was, 'Don't be afraid of your past. You don't have to copy it, but it's OK to be inspired by it.'"

So what's this new record about? Innovation fatigue? Nostalgia? Or could it embody a quality that is not usually associated with metal, but probably should be: refinement?

Almost every summer Metallica mop up Europe. In July the group played stadium shows in cities including St Petersburg, Riga and Sofia, drawing 19,000 to 50,000 people per concert. The Bucharest show draws a sellout crowd of 23,000. Eastern Europeans love their metal, and Romanians seem particularly well-suited to it. The city is architecturally ghoulish: an elegant 19th-century European capital whose communist government left dull and crumbling boxes everywhere. Transylvania lies just a few hours to the northwest. During the show Hammett plays a guitar bearing an image of Bela Lugosi as Count Dracula.

Before Metallica, which pushed the band toward Establishment acceptance, the band were both a midsize worldwide success and a subculture. Before the Bucharest show I ask a man in his late thirties how difficult it had been to find Metallica records in the 1980s, during the dictatorship of Nicolae Ceausescu. "Impossible," he says. "Somebody would smuggle one in, but you didn't know that person. You asked around, and somebody would give you a cassette: a copy of a copy of a copy."

But hatred is a sign of life in metal, and Metallica are also hated. Each year about a million people go to Metallica concerts. And yet a good portion of fan talk about the band's career forms a hard braid of insults. Some believe Metallica have not been any good since, well, since before Ceausescu fell. This logic cuts off Metallica's best years after Master of Puppets, from 1986. That was the last album with the band's first bass player, Cliff Burton, who died in a bus accident when Metallica were touring Sweden that year.

Some say they haven't been any good since the Black Album, which inflated Metallica's music into a plush sort of darkness, with shorter songs and bigger melodies. Yet another faction says they haven't been any good since Load (1996) and Reload (1997), which moved still further from Metallica's baseline metal identity, throwing aside Hammett's peacockish, modal guitar solos for slow-and-easy riffs.

Then there was St Anger, in 2003, which seemed to confuse almost everyone. Metallica's second bassist, Jason Newsted, had quit, and the album radiated anxiety. The songs were loud and solo-free, with a trashcan drum sound and confessional lyrics by all the band members, rather than by Hetfield alone. Still it sold nearly two million copies.

Finally, in 2004, came the film that detailed the band's woes: Newsted's leaving; Hetfield's ten-month departure for rehab, not only to treat his alcoholism but also his rage, which had led to blackouts; the hiring of the coach-therapist; and the making of St Anger. It shows the band members acting like lords and like children. Hetfield and Ulrich are almost personality opposites: one tight-lipped and traditional, the son of a truck-driver (Hetfield); one manic and progressive, the son of a Danish tennis pro and jazz critic (Ulrich). Throughout the film Hammett listens to their arguments with intense discomfort, pleading for reason.

There has been a happy ending. The band cooled out, rebonded and went back on the road with new bass player Trujillo, a sweet-tempered man and a powerful musician. But the film left you wondering whether Metallica had become irredeemably decadent, and whether a kind of steady, low-grade irritability had been their motor. Interestingly, the band members have not distanced themselves from the film. Answering questions about their history, they refer to it almost as if it were an album.

Members of the group have known Rubin since the mid-1980s, when he produced Reign in Blood, the third album by Slayer, one of Metallica's early thrash-metal competitors. He has since produced landmark records in metal, hip-hop, country and pop, by artists including Public Enemy, Johnny Cash and Red Hot Chili Peppers. In his first meeting with Metallica two years ago Rubin gave the band a writing assignment. "I asked them to imagine themselves not as Metallica," he says. "I said to them, let's say there was a battle of the bands coming up and nobody knew who they were, and they can't rely on any of their hits to get them over. What would that sound like?"

He told them that Master of Puppets was the band's best album. He asked them to imagine that it represented only half the material they owed the record company in 1986. What would the other half sound like? "It can't sound like those songs, because you already have those songs," he explained. "The exercise wasn't to rewrite songs like that, but to write songs in that spirit." Hetfield says trying to go back in time was "a nice idea, and pretty near impossible. We know too much. You can't make yourself a virgin again. But I got what Rubin meant."

In any case the group was already preparing to perform Master of Puppets from start to finish for the album's 20th anniversary. Metallica were ready to revisit their 1986 head space.

The new songs are nasty and complex, with double and half-time rhythm switches, twinned and harmonised guitar solos and a few sweeping melodies recalling the Black Album. In The Day That Never Comes, All Nightmare Long and others you think a song has reached its apex or endpoint and then a new door opens, a new tower starts to rise.

In Some Kind of Monster Hetfield, just returned from rehab, worries aloud that he will only write recovery songs. He has written some here: Broken, Beat & Scarred, for instance, with its call to "show your scars". But for much of the rest of the album Hetfield's death-drive is back, unsullied by positive thinking.

Before the show in Bucharest, I ask Hetfield if it was difficult to climb back into the person he used to be.

"Yeah," he says. "I would say that was a different person. I know more now. But on this record I needed to take the reins again, and get heavy and scary with myself again. I don't have to be afraid of the anger. I know how far I want to go with it, and I've gone far and still been OK."

Fans want Metallica to be a bit naïve: lumpen bootstrappers, freaked and fascinated by violent fantasy, straining against their limitations. Death Magnetic, on the contrary, is knowing. But it isn't smug. The album bets on the fact that these musicians have matured, and can prove it through music that's more complicated than what they've become used to, but is still theirs. In that galloping, baroque old style, they sound as if they're pushing, but not straining. "There's a little more calmness around our playing," says Hetfield. "We're not so focused on whether or not we can play it. We're better."

• Death Magnetic is released on Warner Bros on 15 September.