This is the second straight edition of FIFA's flagship tournament the 2014 winners have failed to make it out of their group.
As defending champions in Russia four years ago, Germany under Joachim Low, finished bottom of their group, which comprised Sweden, South Korea and Mexico.
This time in Qatar, led by Hansi Flick, the Die Mannschaft could only secure third spot behind Japan and Spain.
Germany finished level on points with Spain but had an inferior goal difference compared to their fellow Europeans.
Serge Gnabry gave Germany a 10th minute lead at the Al Bayt Stadium but was cancelled out by Yeltsin Tejeda.
A Manuel Neuer own goal briefly had Costa Rica dreaming of a place in the knockout rounds but Kai Havertz struck twice to turn the game on its head before Niclas Fullkrug sealed the win with his second goal of the tournament.
In the other Group E game, Japan came from behind to beat Spain 2-1 to secure top spot.
Spain will face Morocco in the round of 16 while Japan take on Croatia.
