The 60-year-old was detained at a private hospital in the Sicilian capital Palermo.
He had been having appointments at the hospital for some time, Italy’s Ansa news agency reported, adding police had secretly moved officers into the building overnight to guard the safety of other patients.
Italian news agencies said he is believed to be suffering from cancer.
Messina Denaro was taken to a secret location immediately after the arrest, Italian state TV reported.
Italy’s prime minister Giorgia Meloni thanked authorities for the arrest and deemed the operation a “great victory for the state that shows it never gives up in the face of the mafia”.
Messina Denaro was sentenced in absentia to a life term for his role in the 1992 murders of anti-mafia prosecutors Giovanni Falcone and Paolo Borsellino.
He also faces a life sentence for his role in bomb attacks in Florence, Rome and Milan which killed 10 people the following year.
Last year police said he was still able to issue commands concerning the way the mafia was run in the area around the western Sicilian city of Trapani, his regional stronghold, despite being on the run.
Messina Denaro is also accused of being solely or jointly responsible for numerous other murders in the 1990s.
In 1993, he helped organise the kidnapping of a 12-year-old boy, Giuseppe Di Matteo, in an attempt to dissuade his father from giving evidence against the mafia, prosecutors say.
The boy was held in captivity for two years before he was strangled and his body dissolved in acid.
Last of three top-level mafia bosses to be captured
Messina Denaro was the last of three fugitive top-level mafia bosses who had eluded capture for decades.
His arrest comes 30 years and a day after the capture of convicted “boss of bosses” Salvatore “Toto” Riina, who spent 23 years on the run.
“It is an extraordinary event, of historic significance,” said Gian Carlo Caselli, who was prosecutor in Palermo at the time of Riina’s arrest. He said the fact the arrest coincided with the anniversary of Riina’s detention made it even more significant.
However, he warned Cosa Nostra had shown its ability to survive the arrest of senior figures and regroup in the past.
The record for the longest time on the run is held by Bernardo Provenzano, who was captured in a farmhouse near Corleone, Sicily, in 2006, after 38 years as a fugitive.
Once Provenzano was arrested, the hunt focused on Messina Denaro, but despite numerous reported sightings over the years he managed to elude arrest until today.
All three top bosses were ultimately arrested in the heart of Sicily, with the authorities saying they rely on the contacts and confidentiality of fellow mobsters and complicit family members to move from hideout to hideout while being supplied with basic needs such as food and clean clothing, and evading capture under a code of silence known as “omerta”.