Carlo Ancelotti's side lost 3-2 at the Vallecas as they failed to reclaim top spot from Barcelona, who hold a two-point lead.
Real Madrid have now conceded 13 goals in as many La Liga matches this season, keeping just two clean sheets along the way.
Rayo Vallecano's winner came via a re-taken VAR-awarded penalty, with Luka Modric and Eder Militao getting on the scoresheet for Los Blancos.
"Rayo are having a great season, we knew it was going to be difficult, and we weren't up to the task in the first half," the Belgium international Courtois said.
"We conceded too many silly goals, due to mistakes or not being 100 per cent in the match.
"Then the penalty, bad luck. The referee told me that because of Carvajal, that's what they told him in the VAR, that he gets into the area earlier.
"Then I wanted to stay in the centre, but I threw myself and I had a better corner than the one I stopped. It's a shame, I saved the penalty and in the end everything changes. I hope it's our only defeat in LaLiga.
"With so many games, it may be normal that we are tired. The draw against Girona was more bad luck. Today was not our best day and Rayo played well."
Real Madrid return to action against Cadiz on Thursday before the league takes a break for the World Cup.
