The official, who works at the waterworks bureau in the western city of Kobe, began his designated lunch break early 26 times over the space of seven months, according to a city spokesman.
"The lunch break is from noon to 1 pm. He left his desk before the break," the spokesman told AFP on Thursday.
The official, 64, had half a day's pay docked as punishment and the bosses called a news conference to apologise.
"It's deeply regrettable that this misconduct took place. We're sorry," a bureau official told reporters, bowing deeply.
The worker was in violation of a public service law stating that officials have to concentrate on their jobs, according to the bureau.
The news sparked a heated debate on Japanese social media, with many defending the official.
"It's sheer madness. It's crazy. What about leaving your desk to smoke?" said one Twitter user.
"Is this a bad joke? Does this mean we cannot even go to the bathroom?" said another.
The city had previously suspended another official in February for a month after he had left his office numerous times to buy a ready-made lunch box during work hours.
The official was absent a total of 55 hours over six months, according to the city.