Fernandes looked uninspiring as the Red Devils were beaten 3-0 by Manchester City at Old Trafford in the Premier League on Sunday.
Ten Hag handed Fernandes the captaincy over the summer after taking it off Harry Maguire.
Keane was left in an umbrageous position with Fernandes' performance on the day and wants Ten Hag to select a new leader.
"The first thing I would do, I have to tell you having watched him again, I would definitely take the captaincy off him, 100 per cent," he said on Sky Sports.
"I know it's a big decision and obviously they changed the captain with Maguire. Fernandes is not captain material. I think he’s a talented player, no doubt about it. But what I saw with him today, I know we’ve discussed his before. His whinging, his moaning, his throwing his arms up in the air constantly. It really isn’t acceptable.
"I would take it off him. Where do you make changes? I would start with that because the manager can do that. He’s the opposite of what I’d want in a captain."
