"As far as we are concerned, we don't have any problem or objection to hand over the case to the CBI. If the state government comes out with a reference, we will hand over the case to the agency today itself," Mr Singh told reporters on the sidelines of a function in Jammu.
He added that the court was another option through which the case could go to the CBI.
As per my knowledge, a petition (for a CBI probe) is pending there, he said.
The minister of state in the Prime Minister's Office, however, refused to answer any further questions and left.
An eight-year-old girl was kidnapped on January 10 and her body was found on January 17 in the Kathua district.
During investigation, the state police's crime branch alleged she was drugged and repeatedly raped before being killed.
Eight persons, including a juvenile, were arrested in connection with the crime and their trial began yesterday in a court, where they pleaded not guilty and asked for a lie detector test.
Many people in Kathua, including the families of the accused, have been demanding a CBI probe into the incident, which was, however, rejected by Chief Minister Mehbooba Mufti last month after two BJP ministers, Lal Singh and Chander Prakash Ganga, met her and raised the demand.
Both the ministers later resigned over their participation in a rally in support of people arrested in connection with rape and murder case. Yesterday, Jammu and Kashmir Governor NN Vohra accepted their resignations.