Berlin's openly gay mayor, Klaus Wowereit, said the monument was a reminder of the ongoing struggles that still confront gays.
"This memorial is important from two points of view - to commemorate the victims, but also to make clear that even today, after we have achieved so much in terms of equal treatment, discrimination still exists daily," Wowereit said as he inaugurated the memorial alongside Culture Minister Bernd Neumann.
Nazi Germany declared homosexuality a threat to the German race and convicted some 50,000 homosexuals as criminals. An estimated 10,000 to 15,000 gay men were deported to concentration camps, where few survived.
"This is a story that many people don't know about, and I think it's fantastic . . . that the German state finally decided to make a memorial to honor these victims as well," said Ingar Dragset, a Berlin-based Norwegian who designed the memorial along with Danish-born Michael Elmgreen.
The commemoration "unfortunately comes too late for those who were persecuted and survived in 1945," said Guenter Dworek, of Germany's Lesbian and Gay Association. "That is very bitter."
He said the last former prisoner that his group knows of died in 2005. Wowereit echoed his regret over the time it took to honor the Nazis' gay victims.
Few gays convicted by the Nazis came forward after World War II because of the stigma attached to homosexuality.