Prime Minister Yoshihide Suga has called an emergency task force to respond to the disaster, NHK added, as it broadcast Twitter footage of floods that destroyed homes swept away debris.
Atami is a quaint seaside resort area in Shizuoka prefecture, about 100 kilometres south-west of Tokyo. The area that was hit by the mudslide, Izusan, includes hot springs, residential areas, shopping streets and a famous shrine.
Heavy rains have lashed the Tokyo metropolitan area and the Tokai region in central Japan, while residents have been orders to evacuate in Hiratsuka, Yokohama, Chiba and Shizuoka.
Atami received 313 millimetres of rain in 48 hours, according to NHK, above the July average of 242.5 millimetres.