The Brazilian shockingly joined the Gunners this summer in a move worth £8m but Arsenal’s defensive woes continue.
The North Londoners have conceded 15 goals in 11 Premier League matches, keeping only two clean sheets in the process.
Arsenal are six-points away from the top four and are yet to win a league match since their 1-0 win against Bournemouth in early October.
Sherwood questioned why Arsenal failed to snap up Cahill who was available for free, but instead opted for the former Paris Saint Germain and Benfica defender.
The 50-year-old believes Cahill is a better defender compared to Luiz and he would have been a better fit for Unai Emery’s team.
"Their problem was obvious to everybody. Not only the people who go to the Emirates, the people who watch Premier League football know that Arsenal cannot defend," Sherwood told Premier League Productions.
"It wasn’t a hard job to go and find better defenders than what they already had. An easy job, just go out and do it. Try and patch it up, find something.
"Gary Cahill, he was an obvious choice. We’re talking about someone who’s a Premier League winner, a leader-type character, and they end up going and getting David Luiz.”
"Now, I agree, good value [for] David Luiz, but go and find someone better.”
"They needed to do that, they haven’t done it, the recruitment’s terrible, and at the moment they’re getting results that they deserve."