Both the victim Shivaji Shinde (45), a head constable at the local crime branch, and accused Pulja Singh Thakur lived in Datta Nagar area under the Shivaji Nagar police station limits, a senior police official said.
Earlier, four cases were registered against Thakur at various police stations.
According to the officer, Thakur, who was out on bail, was suspicious that Mr Shinde was responsible for his arrests in the past.
Shivaji Shinde was killed on the spot. A murder case was registered and efforts are on to catch the accused, the officer added.
Get Breaking news, live coverage, and Latest News from India and around the world on NDTV.com. Catch all the Live TV action on NDTV 24x7 and NDTV India. Like us on Facebook or follow us on Twitter and Instagram for latest news and live news updates.