The Nerazzurri were stunned in the opening three minutes when Christian Kouame gave the visitors the lead but Lautaro Martinez got the equalizer for Conte’s men on the stroke of halftime.
Inter went ahead 2-1 seven minutes after the break after Federico Ceccherini scored an own goal.
The Milanese club threw away their lead five minutes later when Franck Ribery teed up Gaetano Castrovilli with a sublime pass after an impressive run down the left for the Italian to score Fiorentina’s second of the night.
Federico Chiesa gave La Viola another lead midway through the game but Romelu Lukaku bundled in an 87th-minute leveller before Danilo D'Ambrosio thundered in a back-post header to take all three points.
Speaking to the media after game, Conte was impressed with how his team attacked Fiorentina but insists Inter must show similar desire in defending.
“The three points are certainly important, but it’s only fair to evaluate the entire performance too,” the former Juventus coach told DAZN.
“The positive is that we scored four goals and created many chances to get more. I think their goalkeeper (Bartlomiej) Dragowski did really well.”
“We attacked in numbers and created problems for Fiorentina, but at the same time we weren’t focused enough to deal with their counter-attacks.
“The lads know we need to improve on this, as we must attack and defend in numbers. Balance is the key to everything, we were unbalanced at times this evening and paid for those mistakes.
“We must try to defend by going forward, not sitting back on our goalkeeper’s lap. We want to press the opposition. It was (Aleksandar) Kolarov’s first game, (Danilo) D’Ambrosio was playing as a centre-back, but the whole team has to work the right way.
“I was impressed by the way we attacked today, even if I think we had a few too many players going forward and that left some gaps for Fiorentina to counter into. We need to pay closer attention."