Conservative councillors also criticised the council’s general investment strategy.
The council currently has about £1.8bn of debt as it continues to fund a range of investments.
They include sites in the town such as Birchwood Park, Time Square and Golden Square Shopping Centre, but it has also invested in supermarkets in Greater Manchester, solar farms in York, Hull and Cirencester, a BT development being built in Salford and a third share of Redwood Bank.
An energy firm part-owned by the council collapsed in 2022.
Mr Matthews said £23m of profit from investments go to support services for residents.
Mr Mundry said he was “disappointed” by the Conservative group’s amendments.
He said he was hoping for a “credible” alternative budget from the opposition instead of “negativity”.
But Mr Critchley said the group had been “denied the required information which would have allowed an alternative budget to be presented” and if they had presented an alternative budget, it would have included a review of investments.
Liberal Democrat councillor Ian Marks said with increasing financial pressures he agreed with the administration that “creative” ways needed to be found
He said the group supported “many but not all of the investments made by Labour”.
“We accepted that setting budgets is extremely difficult these days and we’ll support the 4.98% rise,” he said.
Liberal Democrat Bob Barr added: “This is our responsibility, the responsibility of all councillors regardless of party to pass a balanced budget.”