With dust and clay blowing off the court, Federer was ousted from his first clay-court tournament of the season.
"I don't think I played terrible. I thought it was OK," he said. "It was still the first week of clay, so I don't expect myself to play my very best. If I would, that would be a pleasant surprise."
Federer, the winner of 16 major titles, lost to Melzer for the first time in four matches. It was also the first time this year Federer was beaten by someone other than Rafael Nadal or Novak Djokovic.
"I think I should have definitely gotten one of the sets," said Federer, who wasted seven break points against the seventh-seeded Austrian. "Every time I had this slight opening, things didn't go my way."
Nadal, the six-time defending champion, fared far better. The top-ranked Spaniard beat Ivan Ljubicic 6-1, 6-3 for his 35th straight win at the tournament. Nadal will next face third-seeded Andy Murray, who downed Portuguese qualifier Frederico Gil 6-2, 6-1.
Melzer sealed the victory when the second-seeded Federer chopped a weak forehand into the net. Melzer next plays fourth-seeded David Ferrer of Spain, who defeated 11th-seeded Viktor Troicki of Serbia 6-3, 6-3.
"I have beaten Rafa last year. I have beaten Nole [Djokovic]. So this was the one missing," Melzer said. "I had a few shots on the line, a few bad bounces that were going my way. You need that to beat Roger."
Melzer received treatment on his back after the first game but had no trouble after that. Federer credited Melzer's attacking play but said the left-handed Austrian also enjoyed some luck.
"Either he shanks them and they stay in play, he hits the line," Federer said. "All those things accumulate to something quite frustrating. That's what made it hard."
Melzer said Federer caught the breaks in the fourth round of the U.S. Open last year.
"When we played at the U.S. Open, there were a lot of key points where I felt I was unlucky," Melzer said. "I thought, luck is going to turn. It did today."
Federer said he couldn't remember losing to a left-hander other than Nadal in seven or eight years. The Swiss star said the conditions were tough on both players.
"I think probably being down in the score didn't help, that the wind started to come maybe after the first set when I was down," he said. "Even in the wind, I had all my chances to come back into the match.
"He did well. I think he played aggressive, was able to mix it up. Obviously, I wasted way too many break-point chances today, which was unfortunate," he said.
Two years ago at Monte Carlo, Federer lost in straight sets in the third round to countryman Stanislas Wawrinka -- ranked 16th at the time, seven places lower than Melzer. Federer went on to win the 2009 French Open and complete his career Grand Slam.
The last time Federer lost before the semifinals was at Wimbledon last July when he was beaten by Tomas Berdych in the quarters. Outside the Grand Slams, Federer hadn't lost this early since his second-round exit to Ernests Gulbis at the Italian Open nearly a year ago.
This latest defeat raises more questions about whether Federer is slipping further behind Nadal and Djokovic.
"Well, always a surprise when Roger lose, because he's unbelievable what he did in this sport," Nadal said. "But, you know, the day was difficult, I said before. That probably didn't help him a lot."
Nadal lost his serve once when the players swapped breaks at the start of the match. Although he saved two break points serving for the match at 15-40, he was troubled more by the weather than by Ljubicic.
"It was a very difficult day to play tennis. A lot of wind, changing directions the wind all the time," Nadal said. "He didn't play well. That's for sure. Ivan didn't have his best day."
Murray said he liked the way he was hitting against Gil, but Nadal is an altogether different opponent.
"I'll have to play one of the best matches of my life," Murray said. "But that's what I've got to be ready for, prepare myself mentally to do. If I do that, I'll definitely make it very difficult for him."