Japanese tourist Tosho Sai, who was staying in a nearby building, said a security guard told everyone on his floor to leave after a window in the unit next door was hit by embers.
A French business traveller passing by the site said he saw "lots of debris falling" from the tower.
"It's truly an apocalyptic scene... Really very, very scary for everyone", he told AFP
Large pieces of debris -- apparently from the site -- were visible on the ground, and an acrid smell permeated the air, an AFP reporter on the scene said.
Flames were seen on the roof of an office tower across the street at around 3:30 am, prompting fears of a wider inferno in the densely built area.
Police said five buildings in the surrounding area had reported fires, though some had been quickly extinguished.
The building was billed as a 42-storey "harbourside icon" in the making, intended to house the historic Mariners' Club and a new hotel, according to the website of its developer, the Empire Group.
The HK$6 billion ($764 million) redevelopment project was greenlit in 2019 and was originally expected to be completed in the first half of 2023, according to local media.
Empire Group did not immediately respond to AFP's request for comment.
