One protester was killed and several more were hurt when security forces opened fire.
The crowd had been addressed by Mir Hossein Mousavi, who believes the vote was fixed in favor of President Mahmoud Ahmadinejad.
Mr Ahmadinejad has dismissed the claims and says the vote was fair.
The BBC's Jon Leyne, in Tehran, says Monday's rally was the biggest demonstration in the Islamic republic's 30-year history and described it as a "political earthquake".
The government had outlawed any protest following two days of unrest, with the interior ministry warning that "any disrupter of public security would be dealt with according to the law".
Despite this, correspondents said riot police had been watching the rally during the afternoon and had seemed to be taking no action.
But reports at 2045 local time (1615 GMT) said shots were being fired.
"There has been sporadic shooting out there... I can see people running here," Reuters quoted a reporter of Iran's Press TV as saying from Tehran's Azadi Square.
"A number of people who are armed, I don't know exactly who they are, but they have started to fire on people causing havoc in Azadi Square."
A photographer at the scene told news agencies that security forces had killed one protester and seriously wounded several others. A man is said to have been arrested over the shooting.
He said the shooting began when the crowd attacked a compound used by a religious militia linked to the country's powerful Revolutionary Guard.