Klinsmann was appointed as manager on a deal until the end of the season last November, replacing Ante Covic in the role with the club in the relegation zone.
The former Bayern Munich and Germany national team manager lasted just ten games in charge of the capital club, winning three, drawing three and losing four.
Hertha Berlin are 14th on the Bundesliga table, six points above the relegation zone.
They have won just once in their last five league games.
Writing on his Facebook page on Tuesday, the former Tottenham Hotspur striker elaborated on his decision to step down as manager.
"At the end of November we met the wishes of the club management with a highly competent team and helped them in a difficult time," the 1990 FIFA World Cup winner wrote.
"We were on a very good path in the relatively short time, and thanks to the support of many people, we are now six points away from the relegation places despite mostly difficult games.
"I am firmly convinced that Hertha will achieve the goal – remaining in the division.
"As head coach, however, I also need the trust of the people involved for this task, which has not yet been completed.
"Unity, cohesion and focus on the essential are the most important elements, especially in the relegation battle. If they are not guaranteed, I cannot exploit my potential as a coach and therefore cannot live up to my responsibility.
"Therefore, after much deliberation, I came to the conclusion that I would give up my position as head coach at Hertha and retire to my original long-term role as a member of the supervisory board."
