"Pakistan is regularly resorting to ceasefire violations and because of this, I feel Pakistan is not showing interest in improving relations with India," Mr Singh said during a press conference in Jammu.
The minister is on a four-day visit to Jammu and Kashmir since September 9.
"Our Army and BSF troops are giving a befitting reply. We will create such conditions that they (Pakistan) will be forced to stop ceasefire violation today or tomorrow," the minister said.
Apart from holding security review meetings, Mr Singh met delegations from various sections in the Srinagar summer capital before beginning his visit to Jammu region, where he continued with his interactions.
Mr Singh was also briefed about the latest equipment being inducted by the BSF.
"Since 2014, Pakistan has resorted to over 400 ceasefire violations every year. This has to be stopped by Pakistan," the minister said.
He was accompanied by Union Minister Jitendra Singh and Deputy Chief Minister Nirmal Singh.
The Union home minister reached out to the the people who were living along the Line of Control but were forced to shun their homes and hearths and take shelter in camps in Noushera sector of Rajouri district for over four months.
The Home Minister said that the country is proud of the border dwellers and that they are a "strategic asset" for India.
It is the biggest strategic asset of India. If we get strategic successes, it is because of your contribution," he said.
The Union minister said the Centre has decided to set up an expert group to study the problems and challenges of the people living along the border.
"This expert group or study group will give its opinion and we will act on that," he said.
The minister said 60 bunkers have been built and a decision has been taken by the government to construct more bunkers.
The minister also hailed the troops of the Army and the BSF, and said the country is proud of them.
He said the border residents are undeterred by the ceasefire violations by Pakistan.
"They have also contributed to defending the borders. This contribution cannot be forgotten," he said.