Robin Williams: 'I was shameful, did stuff that caused disgust – that's hard to recover from'

In the normal order of things, an interview with a Hollywood actor observes the form of a transaction. The actor wants to promote their film, and ideally talk about little else – least of all anything of a personal nature. The newspaper is mildly interested in the new film, but hopes they can be tempted to talk about other matters – best of all their private life. Sometimes the agreement is explicit, but most of the time it is mutually understood, and so the interview tends to proceed rather like a polite dance, with each party manoeuvring in its own interests. On this occasion, however, the convention appears to have been turned on its head.

Robin Williams's new film, World's Greatest Dad, is brilliant. Having starred in a lot of unspeakably sentimental dross in recent years, here he is at last in something clever and thoughtful; a dark, slightly weird comedy that touches on all sorts of interesting themes that I'm hoping he'll talk about. Williams, however, has other plans. It is almost impossible to get anything coherent out of him about the film, or any of the issues it raises. He is vague, tangential and at times more or less incomprehensible – until the conversation turns to more personal matters, at which point he becomes lucid and forthcoming. What Williams really wants to talk about, it turns out, is his relapse into alcoholism, his rehab and his open-heart surgery.

Unfortunately, it takes me some time to cotton on to this, so I keep asking questions about World's Greatest Dad. Williams plays Lance, a failed writer, failed teacher and single father of perhaps the most irredeemably dislikable teenager ever to appear on screen. His son Kyle is addicted to hardcore internet pornography and is almost universally loathed – until he accidentally dies. His father fakes a suicide note, and when it is leaked, the school magazine reprints the letter, its poignancy prompting a posthumous revision of everyone's former low opinion of the boy. Soon a juggernaut of confected grief is roaring out of control.

Unable to resist the allure of his new popularity, Lance proceeds to fake a whole journal, passing it off as his son's and fuelling the insatiable hunger for loss. A bidding war breaks out between publishing houses, the journal becomes a bestselling book, and Lance winds up on a daytime TV show, like a pseudo celebrity, peddling his mythical son's tragedy to the nation.

The film is a devastatingly funny indictment of the modern grief industry, but when I ask Williams if he thinks it's getting worse, he says mildly, "Well, I think people want it. In a weird way, it's trying to keep hope alive." So does he not share the film's judgment on mawkish sentimentality? "Well, you just try and keep it in perspective; you have to remember the best and the worst." It seems as if he's about to engage with the question – "In America they really do mythologise people when they die," he agrees – but then he veers off at a tangent, putting on Ronald Reagan's voice but talking about the ex-president in the third person: "Maybe he was kind of lovable, but you realised half way through his administration he really didn't know where he was."

I wonder if Williams had experienced a little bit of the film's theme himself, when his great friend Christopher Reeve died. Was it hard, I ask, to see fans mourning Superman, when to Williams he was a real person, a real friend?

"He was a friend," Williams says solemnly. "And also knowing him, especially after the accident and everything he went through – it was a weird thing." What was it like, I try again, to grieve privately for a public figure? "Well, it's a whole different game," he says, but then starts talking about the death of Reeve's wife a year later. "It happens all the time, I know, but I know their kids, they're amazing, and to see them go through so much loss in one year – that's tough."

I ask about the media's role in the manufacturing of grief, but instead he recalls a talkshow he saw where a man confessed to adultery before a female studio audience. "Idiot. Why don't you just go bobbing for piranha? These women are screaming 'You bastard!', but the idea of being on TV overrode everything." He adopts a southern redneck accent: "'Ah'm on TV, y'all.' You're a schmuck, why would you do that?" Then the accent again: "Ah'm on tee-vee, ah'm gonna be fay-mous.' Yeah, for all of five minutes, big time."

We're not making much headway on the grief industry, so I try internet porn. Williams's three children have grown up through the internet age, so I'm curious about his views on its impact on adolescents. "It's just like – there's everything you could ever think about online." But what does Williams actually think about it; is it liberating and a good thing, or corrupting and a bad thing? "It's an old thing," he shrugs. "Look at the walls of Pompeii. That's what got the internet started." Then he starts talking rather boringly about iPhones, and how it's now possible to do video-conference calls on a mobile.

My worry beforehand had been that Williams would be too wildly manic to make much sense. When he appeared on the Jonathan Ross show earlier this summer, he'd been vintage Williams – hyperactive to the point of deranged, ricocheting between voices, riffing off his internal dialogues. Off-camera, however, he is a different kettle of fish. His bearing is intensely Zen and almost mournful, and when he's not putting on voices he speaks in a low, tremulous baritone – as if on the verge of tears – that would work very well if he were delivering a funeral eulogy. He seems gentle and kind – even tender – but the overwhelming impression is one of sadness.

Even the detours into dialogue feel more like a reflex than irrepressible comic passion, and the freakish articulacy showcased in Good Morning Vietnam has gone. Quite often when he opens his mouth a slur of unrelated words come out, like a dozen different false starts tangled together, from which an actual sentence eventually finds its way out. For example, "So/Now/And then/Well/It/I – Sometimes I used to work just to work." It's like trying to tune into a long-wave radio station.

I find myself wondering if alcohol abuse might have something to do with it. Williams used to be a big-drinking cocaine addict, but quit both before the birth of his eldest son in 1983, and stayed sober for 20 years. On location in Alaska in 2003, however, he started drinking again. He brings this up himself, and the minute he does he becomes more engaged.

"I was in a small town where it's not the edge of the world, but you can see it from there, and then I thought: drinking. I just thought, hey, maybe drinking will help. Because I felt alone and afraid. It was that thing of working so much, and going fuck, maybe that will help. And it was the worst thing in the world." What did he feel like when he had his first drink? "You feel warm and kind of wonderful. And then the next thing you know, it's a problem, and you're isolated."

Some have suggested it was Reeve's death that turned him back to drink. "No," he says quietly, "it's more selfish than that. It's just literally being afraid. And you think, oh, this will ease the fear. And it doesn't." What was he afraid of? "Everything. It's just a general all-round arggghhh. It's fearfulness and anxiety."

He didn't take up cocaine again, because "I knew that would kill me". I'd have thought it would be a case of in for a penny – "In for a gram?" he smiles. "No. Cocaine – paranoid and impotent, what fun. There was no bit of me thinking, ooh, let's go back to that. Useless conversations until midnight, waking up at dawn feeling like a vampire on a day pass. No."

It only took a week of drinking before he knew he was in trouble, though. "For that first week you lie to yourself, and tell yourself you can stop, and then your body kicks back and says, no, stop later. And then it took about three years, and finally you do stop."

It wasn't, he says, fun while it lasted, but three years sounds like a long time not to be having fun. "That's right. Most of the time you just realise you've started to do embarrassing things." He recalls drinking at a charity auction hosted by Sharon Stone at Cannes: "And I realised I was pretty baked, and I look out and I see all of a sudden a wall of paparazzi. And I go, 'Oh well, I guess it's out now'."

In the end it was a family intervention that put him into residential rehab. I wonder if he was "Robin Williams" in rehab, and he agrees. "Yeah, you start off initially riffing, and kind of being real funny. But the weird thing is, how can you do a comic turn without betraying the precepts of group therapy? Eventually you shed it."

Williams still attends AA meetings at least once a week – "Have to. It's good to go" – and I suspect this accounts for a fair bit of his Zen solemnity. At times it verges on sentimental: he asks if I have children, and when I tell him I have a baby son he nods gravely, as if I've just shared. "Congrats. Good luck. It's a pretty wonderful thing." But it may well be down to the open-heart surgery he underwent early last year, when surgeons replaced his aortic valve with one from a pig.

"Oh, God, you find yourself getting emotional. It breaks through your barrier, you've literally cracked the armour. And you've got no choice, it literally breaks you open. And you feel really mortal." Does the intimation of mortality live with him still? "Totally." Is it a blessing? "Totally."

He takes everything, he says, more slowly now. His second marriage, to a film producer, ended in 2008 – largely because of his drinking, even though by then he was sober. "You know, I was shameful, and you do stuff that causes disgust, and that's hard to recover from. You can say, 'I forgive you' and all that stuff, but it's not the same as recovering from it. It's not coming back."

The couple had been together for 19 years, and have a son and a daughter, both now grown up; he has another son from his first marriage to an actress in the late 70s. Williams is now with a graphic designer, whom he met shortly before his heart surgery, and they live together in San Francisco. "But we're taking it slow. I don't know, maybe some day we'll marry, but there's no rush. I just want to take it easy now. This is good news. It's the whole thing of taking it slow. And it's so much better."

Williams thinks he used to be a fairly classic workaholic, but at 59 is now taking it slow professionally too. "In one two-year period I made eight movies. At one point the joke was that there's a movie out without you in it. You have this idea that you'd better keep working otherwise people will forget. And that was dangerous. And then you realise, no, actually if you take a break people might be more interested in you. Now, after the heart surgery, I'll take it slow."

Williams has been nothing if not prolific. After first finding fame in the late 70s as a kooky space alien in the sitcom Mork and Mindy, he became better known as a standup comedian, but his astonishing performance in Good Morning Vietnam earned him an Oscar nomination in 1988, with two more in the following five years, for Dead Poets' Society and The Fisher King. Mrs Doubtfire, in which he dragged up to play a nanny, brought wider mainstream success, and in 1998 Good Will Hunting finally won him an Oscar. In recent years, however, he has made an awful lot of what would politely be described as less critically acclaimed films.

Some of them have been downright awful; schmaltzy family comedies drenched in maudlin sentiment, such as the unwatchably saccharine Patch Adams or, even worse, Old Dogs. When I ask why he made them, he says: "Well, I've had a lot of people tell me they watched Old Dogs with their kids and had a good time." It didn't offend his sense of integrity? "No, it paid the bills. Sometimes you have to make a movie to make money." He didn't mistake them, he adds, for intelligent scripts: "You know what you're getting into, totally. You know they're going to make it goofy. And that's OK."

Like many people, I had always been confused by Williams's film choices. The sharpness of his early standup just seemed so incompatible with the sentimentality of his worst movies, and if, as Williams claims, Old Dogs simply paid the bills, he must have one very high-maintenance lifestyle. When I watched World's Greatest Dad I just assumed it echoed his own sensibility more accurately than all the other rubbish he has made. But actually, having met him, I'm not sure it does. I don't know whether it was rehab or heart surgery, but he seems to have arrived at a place where sentimentality can sit quite easily.

I ask if he feels happier now, and he says softly, "I think so. And not afraid to be unhappy. That's OK too. And then you can be like, all is good. And that is the thing, that is the gift."

World's Greatest Dad is released on 24 September

ncG1vNJzZmivp6x7tbTEoKyaqpSerq96wqikaJ6Zobpwfo9qZ2irlaV8c3yOq6aboZ5ixKq4y6KYpqtdrLyzuMOsZKCqlZbBpr%2FTZpuanF2WuaS7x6ijZpyiqrS0