In five seasons with the Cavs, Brown made the playoffs every year, took the team to the Finals, was named the NBA's coach of the year in 2008-09 after leading Cleveland to a league-best 66 wins, and then followed that up with a 61-win campaign.
All it got him was a pink slip at the end of the 2009-10 season, as Cavaliers owner Dan Gilbert let Brown go and hired current coach Byron Scott in an attempt to appease free agent LeBron James who had yet to make "The Decision" and sign with Miami.
Instead of being spiteful about the way it ended, Brown insists he is simply grateful for his time with the franchise.
"I understand this is a business," Brown said after his team's shootaround Friday. "I appreciate the opportunity that Dan Gilbert gave me, but it's his money, it's his team and he decided to go in a different direction. I just appreciate the five years that he gave me."
The Lakers hosted the Cavaliers on Friday night and will not have a road game in Cleveland this season because of the lockout-shortened 66-game schedule.
Before Friday's game, Brown was asked if he was "surprised" that he was fired.
"Was I surprised? No," Brown said. "It was tough. There was a lot of uneasiness and unsuredness, I guess, with the LeBron situation. You know, it's a championship team when you have a guy like LeBron. ... Everybody expects you to win a championship and only one team can do it. It takes a lot of luck, it takes a lot of skill and being in the right place at the right time and all that, and it didn't happen. They say Dan Gilbert is entitled to do what he wants because it's his team and he pays the bills. So, it is what it is, or it is what it was. I'm excited to be here in L.A. and I'm excited to be coaching this Lakers team."
Earlier, Brown looked at the situation with some perspective.
"I understand that life is bigger than what we do," Brown said. "It's not the end of the world if I get fired. I make great money. I enjoy what I do. There are people out there doing real jobs where if they get fired, you understand that the impact is so much greater. I would feel like a spoiled child if I were to rant and rave about getting fired after I made millions of dollars from a man and I had an opportunity that not many people probably would have given me with my age and experience and so on and so forth.
"So, I respect him for the opportunity, I respect him for the time I had there and I feel like I do a pretty good job of keeping things in perspective."
Brown became the second-youngest coach in the league when the Cavaliers gave him his first head coaching gig of his career back in 2005 at the age of 35. Prior to becoming the head man in Cleveland, he had spent 13 years working his way up the ranks as a video coordinator and scout with Denver, followed by assistant coaching jobs with Washington, San Antonio and Indiana.
Brown was 272-138 (.663) in his five seasons with the Cavs, but said he knew his success did not guarantee permanent job security.
"You always hope [you will stay coaching in one place], especially when your family enjoys an area and you've had success and you like everybody in the organization and you truly believe if you catch a few breaks you might be able to win it all," Brown said. "But, at the end of the day, you also have to keep it in perspective and not think too far ahead because in this business, turnover and change and problems can, not just creep up on you, it can all of the sudden happen and so you just have to mentally be prepared for it."
"I have fond memories of the city and the people there first, and then second, the organization and the players," Brown said. "The players are great guys. So, to see them, it will be great to see them and at the end of the day, I hope we beat them."
Dave McMenamin covers the Lakers for ESPNLosAngeles.com.