The education secretary, Michael Gove, removed "children, schools and families" from his department's title within two days of taking office and replaced them with "education". The change was in homage to his belief that knowledge is separate from our families. His speeches often say as much – proclaiming that we can overcome our background if we just work hard, be nice, learn the classics. It's a noble dream. But we're kidding ourselves if we think that families don't affect education, and we're especially kidding ourselves if we don't see how policies beyond those in the education department are currently tearing some children's lives apart.

Since 2010, more than 300,000 public-sector workers have been made redundant and thousands of jobs have been lost as high street chains shut up shop. New private-sector jobs have provided some relief, but these positions are often low-paid, temporary, part-time, and the insecurities once solely associated with unemployment have become indefinite. Children with disabled parents are seeing their families deal with punishing changes to disability living allowance as well as the punitive introduction of the bedroom tax. And as the housing benefit cap takes hold and families are forced to move "somewhere cheaper", many children will be wrenched from their schools.

Dealing with such flux is tough on adults, but it's also confusing for children. It affects small things ("Sorry love, no money for Brownies this week"), it requires facades ("I can't come to your birthday party, we're away that day"), and there's the trauma of watching your parents hollow out with worry as safety nets are pulled from underneath them.

As educators, this creeping instability matters. It matters because of the statistics: children in low-income homes achieve an average 1.7 GCSE grades lower than wealthier peers. But it also matters because hungry children are distracted children. Because moving schools means losing friends and studying a disjointed curriculum. Because tight budgets make parents less able to pay for educational "extras" – revision guides, internet access – while at the same time, the libraries and youth services normally filling this gap are also being axed. And when your family is on the breadline it becomes harder to justify the cost of higher education. As an ex-student once put it: "My dad can't afford a headstone for my mother's grave. Do you honestly expect me to get into £40,000 of debt for a degree when I could get a job and help him out?"

Cynics argue that small cuts do not make a real difference to families. But those on the frontline know that's not true. Once I taught a GCSE student whose attendance suddenly became erratic. On investigation, we discovered that he was sharing his school trousers with his brother. Having badly ripped his own, his mother couldn't immediately afford new ones, so she was only sending one child a day while she scraped together the cash for new ones. It sounds so archaic and ridiculous that a hard-working child's future was at risk because of some trousers. But these are the decisions you make when you have only a few pounds to spare each week.

Changing the name of the education department distances Gove from the welfare reforms, but it doesn't distance our classrooms. The economy affects children. Callous welfare reform affects children. And while teachers make their classrooms a safe haven, while we rightly tell our students that trying hard and learning means we have the greatest shot at our dreams, it is also important that we do not allow the government to believe it is off the hook. Make no mistake: these changes will harm the education of many. And taking the words "children and families" off the door does not mean you get to wish that reality away.

Laura McInerney was a teacher for six years and is now a Fulbright scholar

Guardian