Rummenigge assumed the role in 2002 and has been instrumental in the success story of the Bavarians both domestically and in Europe.
The 65-year-old was expected to remain in the role until the end of the year but has elected to step down early.
Rummenigge told the club's official website: "It is the strategically most sensible and logical point in time. It's the end of the financial year, and a new section begins with a new team of coaches.
"Oliver Kahn as the new CEO should be responsible for the new season right from the start – also in the interests of the people involved and thus in the interests of the future of FC Bayern.
"I said two years ago that we were planning a prudent transition, and that is now finally being made. That's how it should be."
Bayern president Herbert Hainer added: "Oliver Kahn will take over responsibility at the head of FC Bayern München AG as early as July 1st.
"After his extremely successful professional career and his time as an entrepreneur, he has now been intensively involved in the board of directors of FC Bayern München AG for a year and a half.
"He was already involved in the decisive decisions for the future of FC Bayern and set the course for the next few years with the FC Bayern AHEAD strategy project. We are all convinced that Oliver Kahn is the right man to successfully shape the future of FC Bayern Munchen AG."
