The 75 passengers on board had to evacuate the plane by emergency slides while amateur footage shot from the ground showed black smoke spewing from the right engine shortly after takeoff.
Heathrow was temporarily forced to close both its runways while emergency crews put out a fire on the Oslo-bound Airbus A319, and British Airways cancelled all of its short-haul flights until 1500 GMT.
The airline said a "technical fault" was responsible but declined to comment on speculation that the plane's engine had failed.
"We have begun a full investigation into the incident," a British Airways spokesman said.
The London Ambulance said it had treated three people for minor injuries.
David Gallagher, a passenger on the flight, said "big flames" had been visible from within the cabin.
"About eight or nine minutes into the flight there was a loud popping sound, not an explosion but definitely not usual sounds," he told BBC television.
"There was some concern from passengers -- people gasping and louder exclamations.
"The captain came on very calmly, said he was aware of the situation and that everything was running normally and he was going to run some tests to see what the right course would be.
"Then another five minutes after that there was a loud sound, and this time the right engine was clearly on fire."
He praised the cabin crew as "outstanding and completely calm, reassuring and professional during the whole incident".
A British Airways spokesman stressed that there was "absolutely no indication" that the incident was terror-related.
"This is a purely technical issue," he said.
The airline had been expecting its busiest day of Britain's bank holiday weekend on Friday, with 128,000 passengers due to travel.
Passengers at Heathrow -- one of the busiest airports in the world -- were experiencing long delays.
A British Airways statement said: "The closure of one runway meant that we needed to cancel all our short-haul flights in to and out of Heathrow up until 16:00 (BST) today.
"These cancellations will help us to stabilise our schedule, allowing us to get as many customers away as possible in these difficult circumstances."