Jonathan Woodgate said the challenge that led to Marcus Browne's red card was “horrific” and cost his team the game.
He also confirmed that Middlesbrough will appeal Paddy McNair's sending off and claims Swansea shouldn't have been awarded a crucial first half penalty.
Boro were reduced to nine men at the Liberty Stadium and the head coach was less than impressed with the officiating.
Steve Cooper's City took the lead when Andre Ayew converted a controversial 22nd minute penalty but the visitors drew level just before the hour mark through Marcus Tavernier.
Boro looked well placed to hang on for a draw before Browne, who had already been booked for simulation just minutes after coming off the bench, was sent off for a reckless challenge on Mike van der Hoorn.
Ayew made it 2-1 and Sam Surridge added a third before McNair was sent off for lashing out at Matt Grimes.
After match, Woodgate said: “When we get to 1-1 there's only one team that's going to win the game in my view.
“I think the referee gave a penalty in the first half which I didn't think was a penalty, Dael Fry gets the ball through the legs of Ayew, you can see it clearly. He makes a bad decision.
Most Read
“I said at half-time if we get the next goal there's only one team that's going to win the game. We get that situation, we're looking good, the place is quiet, we're thinking, 'We've got this'.
“Then the decision from Marcus Browne to make an horrific, an horrific tackle on the Swansea player, I can't condone that, it's a terrible tackle that deserves a straight red.
“I don't condone those types of tackles and he's let his team down. I've watched it back it's a straight red all day long.
“But the ill-discipline from Marcus Browne really cost us that game. He'll definitely get fined.
“It has to change - how the players are going on,” added Woodgate. “It's rash decisions, he's just had a yellow card for simulation, surely you've got to think to yourself, 'Don't go in for this'. And he goes in for a two-footed tackle, not one, two! It's very frustrating but I can't go onto the pitch telling them what to do, how to make tackles.
“Paddy's is not a red in my view, but no complaints over Marcus Browne's. We'll definitely be appealing Paddy's, but not Marcus Browne's, not a chance. Paddy McNair's is a yellow card in my view.”
Woodgate said Boro should have been awarded a free-kick before McNair committed his offence.
Don’t miss
“The game should have been stopped in the first place because we were breaking and one of their players brings down Tavernier,” he explained. “That should have been a yellow in itself and then we get a red for Paddy.
“Today some of the decisions were awful, really, really bad. I don't want to get into trouble about it but, yes, (officiating) has cost us a few times and Steve (Cooper) will say the same.
“If you look at the game in general we were fine, we coped really well. We had a lot of possession of the ball, I'd say we dominated the ball in the first half.
“They got that goal, the penalty, then we came out in the second half and we were playing really well and we get the goal and there's only one team going to win the game in my opinion.
“It was a good performance barring the tackle. When we've got 11 men on the pitch it's a good performance but ill-discipline costs you and we look to the next game now.”