The magnitude 6.3 quake left some homes with rooftop tiles broken and cracked walls, while goods fell off store shelves, according to the Meteorological Agency and Japanese TV news footage.
The agency had earlier said the magnitude was 6, but later upgraded it.
The quake was centered on Awaji Island, just south of Kobe, at a depth of 15 kilometers (9 miles).
The quake was in the area where a magnitude 7.2 quake killed more than 6,400 people in 1995.
TV news footage showed that some areas on the island had liquefied, a common effect of strong earthquakes. Japan is among the most quake-prone nations in the world.
In March 2011, northeastern Japan was struck with a giant quake and tsunami, killing nearly 19,000 people and setting off a nuclear disaster.
The agency warned there may be aftershocks for about a week.