Iain Duncan Smith’s second epiphany: from compassion to brutality | Polly Toynbee

I've seen his benefit sanctions inflict misery on places like Easterhouse, where poverty made him weep a decade ago

Iain Duncan Smith visiting Easterhouse in 2002: 'In Easterhouse, as everywhere, the bedroom tax causes mayhem. There are no one-bedroom homes, so £14 or £25 cuts to benefit are widespread among many with spare rooms.' Photograph: Murdo Macleod

In Easterhouse this walk has become a Via Dolorosa, a path trod regularly by visiting journalists and TV crews seeking the sacred spot where Iain Duncan Smith (nearly) wept in epiphany a decade ago. It sickens the locals, groaning when yet another visitor asks the way. The picturesque boarded-up Glasgow tenement, where Duncan Smith was photographed looking stricken, was already condemned, and long ago replaced under Labour's Decent Homes programme. But life behind front doors took a desperate turn for the worse, largely due to the man who underwent a Damascene conversion here – only to undergo a reverse conversion later.

Bob Holman, the 76-year-old lifelong community organiser who founded Fare – a celebrated community centre staffed mainly by unemployed volunteers – was touched by Duncan Smith's conversion: "I thought him a decent man" – perhaps misled into trusting a fellow Christian.

Tim Montgomerie, founder of ConservativeHome and now the Times comment editor, arranged that visit as Duncan Smith's adviser. When Holman asked why, he was told: "We're interested in compassionate Conservatism." Later Holman was asked to speak on poverty at the Conservative party conference. In return Holman invited IDS to speak at Labour's 2005 conference, where the ex-Tory leader called Labour's poverty measure inadequate, saying: "Everyone should have enough money to live properly in their community."

What happened? "He wept at the plight of the poor, yet now hands out punishments that must bring tears to their eyes," Holman says. Duncan Smith once called Holman "a living saint", and indeed he has an air of gentle naivety, not bitter but disappointed – convinced that he was not duped, but that IDS was seduced by power.

Now Holman is outraged at the suffering he sees as deliberately inflicted by his erstwhile friend. This week emerged the long-delayed figures for people "sanctioned", their benefits withdrawn under a tightened screw. I have visited food banks, Citizens Advice bureaux and social fund offices from Stoke to Leeds: everyone says the same – of the people reduced to penury and starvation, most are those whose benefit is delayed or withdrawn. The 580,000 sanctions between October and June, hailed by the government as ending a "something for nothing culture", removed anyone deemed not trying hard enough to find work.

The numbers vanishing from the unemployment register hugely improves the apparent tally of those finding work. In the last three months the government boasts of 120,000 fewer unemployed claimants, in contrast with the much larger numbers shown in the internationally recognised labour force survey of numbers wanting work. Those in work have undoubtedly risen, but knocking people off the claimant count is a win-win for Duncan Smith, who can pretend that all who have left the register are working, although many are now queueing at food banks.

Holman, Fare and the local Citizens Advice know who is knocked off benefits and why. There is the man with acute psychosis from a background of extreme abuse (for which his stepfather is in prison) who was declared fit for work, found a job he couldn't cope with and was cut off benefit for six months when he dropped out. A 50-year-old was struck off when told to apply for 10 jobs a week online but, barely literate, he couldn't use a computer.

Behind the scenes, Duncan Smith's sanctions cause a special kind of hell for jobcentre staff, mostly decent people. A regular "deep throat" correspondent describes the work: "You park your conscience at the door," he tells me. "Sanctions are applied for anything at all to hit the targets."

Many claimants don't know what's happened until their benefit suddenly stops. Many are semi-literate or have bad English: "It's very easy to hand someone two sheets of A4 and get them to 'agree' to 50 'steps' towards work and then sanction them when they don't even know what a 'step' is. The claim is shut down for two weeks and sanctioned for two weeks, so the person disappears from the figures."

People are often sanctioned for a no-show at appointments they never knew about. If they call to rearrange an appointment, "we don't answer the phones, so that's a bit tricky". A flowchart on the wall shows how to raise a successful sanction.

If advisers often find "good cause" not to sanction the vulnerable, at monthly performance reviews their managers "examine our sanction-raising figures" against an "expectation" of how many should be cut off. A "PIP" (personal improvement plan) for each member of staff detects low sanctioners and "manages them out" of their jobs.

Someone with a disability who is knocked off employment support allowance can reclaim while awaiting an appeal. "But we are explicitly forbidden from telling them that – in black and white in the briefing pack – so these often very ill, quite confused and low-capability people are easy meat."

Here's another perversity in this brutal system: advisers are so busy knocking off easy, vulnerable cases that they have no time to chase up devious criminal gangs and fraudsters who take longer to catch. Meanwhile, out of fear or confusion, and faced with a 55-page online application form, many stop trying to claim, only to have Duncan Smith pretend they must be cheats who have been chased away.

In Easterhouse, as everywhere, the bedroom tax causes mayhem. There are no one-bedroom homes, so £14 or £25 cuts to benefit are widespread among many with spare rooms. Labour has called a full-day debate in the Commons next Tuesday: every Labour MP should give testimony from their constituency, as should all Lib Dem and Tory MPs who see the evidence in their own surgeries. Outside, there will be a lobby of parliament. Rachel Reeves, Labour's shadow work and pensions secretary, is calling on people to join in, details here from campaigner Sue Marsh's blog.

Duncan Smith invents figures, defying even the UK Statistics Authority's rebukes. His Centre for Social Justice churns out reports that blame poverty on the poor and their failure to marry, while all his own policies are in crisis. This week universal credit was blasted by the Commons public accounts committee, while Deloitte is bailing out of his Work Programme. Easterhouse is left asking: what second epiphany led him to cut £23bn from children, the sick and the unemployed, shrinking their incomes by a quarter?