A spokesman for the Afghan defence ministry said that there were civilian casualties but he did not give any further details.
"A suicide attacker has detonated himself within 30 metres (100 feet) of the defence ministry gate, there have been casualties among civilians, but defence ministry personnel have not suffered any casualties," Zahir Azimi told AFP.
One soldier covered in blood at the scene said he had helped carry five people from the attack site.
Gunfire erupted after the loud explosion was heard across the city, and reporters travelling with Hagel were informed that he was not under any threat as streets around the blast were closed off by security forces.
"The secretary was in a briefing when the incident occurred. The briefing continued as planned without interruption," a spokesman for the Pentagon chief said.
An official at the location told reporters that the bomber had arrived on a bicycle.
"We can confirm a suicide improvised explosive device attack, followed by small arms fire, near the Afghan ministry of defence compound in central Kabul this morning," a statement sent from the NATO-led International Security Assistance Force coalition said.
"No ISAF troops were directly involved."
The Taliban immediately claimed responsibility for the attack.
"This was not a direct attack to target him but we want to send a message that we are always capable of hitting Kabul even when the top US defence official is there," Zabiullah Mujahid told AFP by telephone.
Hagel arrived in Kabul as the ISAF military coalition prepares to pull out by the end of next year and leave Afghan security forces to battle the Taliban insurgency that has raged across the south and east of the country.
"We have a lot of big issues and challenges ahead as we prepare for a responsible transition," he told reporters on his plane before he arrived late on Friday.
"That transition has to be done right, it has to be done in partnership with the Afghans (and) with our allies."
Hagel was sworn in 10 days ago as heavy cuts loom for the US military, but he said Americans realised that Afghanistan remained a major conflict zone with US troops fighting against Islamist militants since the 9/11 attacks.
"We have 66,000 troops still at war in a combat zone, that reality is there," he said. "I don't minimise or marginalise anything just because we may be transitioning to a new phase, we're still at war in Afghanistan."
A total of 100,000 international troops are currently deployed in Afghanistan, with all combat forces due to exit by the end of 2014.
President Barack Obama last month announced that 34,000 US troops would withdraw in the next year, halving the size of the current 66,000-strong US force.
Afghan forces are assuming control of security across the country, but there are widespread fears that they will be unable to face down the Taliban and that the country could tip into greater instability.