He was 86 and is survived by his wife and two children.
His son, Ashok Singh, vice president of the Madhya Pradesh Congress Committee, said that the veteran Congressman suffered a heart attack late Saturday night and died before he could be rushed to hospital.
His last rites would be performed today in Gwalior, he informed.
Rajendra Singh served as a minister in the Madhya Pradesh government from 1975-1977 and was first elected as MLA in 1972, his son said.
Condolences poured in with Congress general secretary and former Madhya Pradesh chief minister, Digvijay Singh, describing him as a "close family friend".
The Leader of Opposition in the Madhya Pradesh Assembly, Ajay Singh, called Rajendra Singh a dedicated soldier of the Congress and said his death was an irreparable loss to the party in the state.