Nuno Esperito Santo’s men held their own for a vast majority of the game but were undone in the third minute of second half stoppage time when Rashford’s effort deflected off Romain Saiss into the visitors’ net.
The win moves Manchester United second on the Premier League table on 30 points, just two behind champions Liverpool, who travel to Newcastle United later on Wednesday.
"I think for us we cannot look too far ahead," Rashford told Amazon Prime after the game.
"We're a team that's still doing a lot of work on ourselves and to be looking at the table so early in the season is a bit stupid of us.
"We have to take it one game at a time and if we can keep finding ways to win like this we'll see where we are at the end of the season."
United’s current league position is the closest they have been to the top of the table at this stage of a season since the legendary Sir Alex Ferguson retired in the summer of 2013.
Ole Gunnar Solskjaer’s men will host Aston Villa in their next league game on Friday.
