His fears were warranted.
Tony Allen scored a season-high 26 points and the Grizzlies, playing with just one of their starters, stunned the Cavaliers 106-103 on Monday night.
"We're all in this league for a reason," James said. "We didn't respect them tonight and they beat us."
Vince Carter made four free throws in the final 13.4 seconds to steal the upset by a Memphis team missing Mike Conley, Zach Randolph, Chris Andersen, Matt Barnes and had more coaches than players sitting on the bench.
"The guys played their hearts out," Memphis coach David Joerger said. "We had guys cramping, but they left it all out there. It's a happy locker room. The phones are blowing up from everybody in Memphis. All of our guys that (are) back home like Marc (Gasol) and Birdman (Andersen) are jumping and down in their living room right now."
Lance Stephenson added 17 points and JaMychal Green 16 points and 10 rebounds for the Grizzlies, who forced the Cavs into a season-high 25 turnovers and never backed down against the Eastern Conference's top team.
Allen returned after missing eight games with a knee injury for Memphis, which only dressed eight players.
"I don't care who you are playing, if it's my son's little league team you're going to lose when you give up that many points off turnovers," James said. "We were very careless with our mindset -- and with the ball."
Perhaps overconfident following weekend wins over Washington and Boston, and against a wounded Memphis team without, the Cavs were outplayed at both ends. Cleveland trailed the Grizzlies the entire game before taking its first lead on Irving's steal and layup with 2:42 remaining.
Memphis figured to buckle, but the Grizzlies wouldn't give in and took a 101-100 lead on Mario Chalmers' layup with 25 seconds left.
Allen then tied up James on a drive, forcing and winning a jump ball with 17 seconds to go. Chalmers made a free throw and Carter made two free throws before James hit a 3-pointer to bring the Cavs within 104-103.
Carter, who finished with 15 points, made two more from the line and the Grizzlies escaped with an unexpected win when Irving's 3-pointer at the horn hit the back of the rim.
"I just thought they were tougher than us tonight," Cavs coach Tyronn Lue said. "Thought they played hard and out-scrapped us. You got to respect everybody in this league. Everyone gets a paycheck."
Only an hour before tip-off, Joerger still wasn't sure who would start.
The Grizzlies initially said only Conley (foot) and Andersen (shoulder) would be out, but then Joerger broke the news that Randolph (knee) and Barnes (hamstring) were also sidelined, making things even tougher on a Memphis team already without Gasol, done for the season with a broken right foot.
The short-handed Grizzlies faced a daunting task, but they weren't intimidated and nearly led from start to finish.
"This is big time for us," Carter said. "We had a lot of guys out. When you see shots go down you gain confidence. We did a great job of executing and not getting rattled."
James chalked up the unexpected loss to a team still figuring things out.
"We're a team that's still learning, man," he said. "We got some young guys and a group that's been together for a little over a year. It's another learning experience for us."
Grizzlies: Conley underwent an MRI and the team is hoping to have a better sense of how long he will be sidelined later this week. ... Memphis has lost more playing time due to injury or illness this season than any other current playoff team. ... The four missing starters average a combined 42.9 points.
Cavaliers: James said the chance to play one final game against Olympic teammate Kobe Bryant would not take precedence over any decision for him to sit out and rest. The Cavs visit the Lakers at Staples Center on Thursday night. Bryant is retiring at the conclusion of his 20th season in the NBA. James said he and Lue have discussed possible games he'll sit but "it's still day-to-day."
Grizzlies: At Boston on Wednesday.
Cavaliers: At Sacramento on Wednesday.
David Fizdale, an assistant with the Heat, has been offered the head-coaching job with the Grizzlies and the sides are working on a contract, according to ESPN and a Vertical report.
Check out top NBA playoff moments from the past week, including the Toronto Raptors' victory over the Cleveland Cavaliers to tie their series 2-2 and the Oklahoma City Thunder's victory to claim a 3-1 lead in their series with the Golden State Warriors.
The Jump crew examines Steph Curry's game and wonder if the MVP is hiding something or if his game is hurt.