Last week, linebacker Ray Lewis said he won't concern himself with rankings until the season is over.
But weeks continue to pile up and Baltimore's defense isn't budging from near the bottom.
After surrendering 396 total yards to the Patriots Sunday night, the Ravens are still 27th in the league in average yards allowed per game.
So what's got to be fixed?
Several Ravens coaches and defenders all had different reasons for the struggles so far, which likely means the unit will have to shore up more than one area:
Strong Opposing Offenses
The Ravens have taken on the No. 5 (Philadelphia), No. 8 (Cincinnati) and No. 9 (New England) offenses in the league in terms of yards per game. While strong performances against Baltimore are one-third of the reason they are ranked so high, the Ravens aren't alone in seeing what those offenses can do.
"This offense is really, really good. That quarterback [Tom Brady] is really, really good," Head Coach John Harbaugh said.
The Ravens will get more of [add] a reprieve this Thursday as the Cleveland Browns are ranked 26th in the league offensively. After that it's the top-ranked Kansas City Chiefs.
Issues In The Secondary
Baltimore has been particularly susceptible in the secondary, an area of strength when looking at the roster heading into the year with Ed Reed and Bernard Pollard at safety and young cornerback talent in Lardarius Webb, Jimmy Smith and Cary Williams.
Yet the Ravens are ranked 28th in the league in average passing yards allowed per game (289.7).
"We've got to be sounder in the pass game," safety Bernard Pollard said. "We've got to be focused; we've got to be on the attack at all times. We can't let off."
Smith and Webb said the Ravens defensive backs are still a confident group.
"I guarantee you we'll be correcting some things this week and moving on," Webb added. "I think we still have a great defense. I think we still play like the Ravens, we just have to correct some things."
Getting Caught In Hurry-Up
A few Ravens were bothered by the fact they let the Patriots sneak up on them with some quick snaps and a hurry-up attack. That's what led to a wide-open, 59-yard reception by Wes Welker on a wheel route.
Pollard said the Ravens "opened up the gate" on a lot of the Patriots' big plays. Nose tackle Terrence Cody said some Ravens were caught without the play call and not lined up a few times and that "it's not what they did; it's what we did."
"There's always going to be stuff every game, but this was one where I felt like we got out of our rhythm a little bit," outside linebacker Paul Kruger said. "They were running the hurry-up offense on us and that's always tough. I think we can do a better job of that."
Not Enough Pressure
Without leading sack artist Terrell Suggs, the Ravens' pass rush was a major concern heading into the season among pundits.
Still, Baltimore has eight sacks, ranking them at a respectable eighth in the league. Baltimore got to Brady twice. Defensive tackle Haloti Ngata has upped his pass rush and already has 2.5 sacks this season.
Yet Ngata feels Baltimore needs to do a better job with its basic pass rush. On some plays, Brady plenty of time to make a play, and the Ravens didn't get much pressure outside of blitz packages.
"Definitely up front, we need to put more pressure on our four-man rushes," Ngata said.
Hurting Themselves With Penalties
The Ravens had two personal penalties and one unnecessary roughness foul against New England. Each extended eventual scoring drives.
Ngata had a personal foul when he got into a post-play scuffle. Pollard had one for the same reason. Reed was flagged for a supposed helmet-to-helmet hit, although replays showed he hit with his forearm.
The Ravens have been part of a number of post-play scrums in their first three games. It seems replacement referees are starting to crack down on such fights more.
"We have to stop hurting ourselves," inside linebacker Dannell Ellerbe said. "There are more scuffles now than I ever remember, but you have to be the bigger man."
Plenty Of Good To Build On
Not everything is bad for the Ravens defense.
While it didn't get any turnovers against New England, it has still created six turnovers (three fumble recoveries and three interceptions) in three games and scored one touchdown (Reed).
Baltimore is playing good red-zone defense, which is why it is 27th in yards allowed but 17th in average points allowed per game (22.3).
The Ravens have allowed opponents to get inside their 20-yard line 13 times (tied for the fourth-most in the league). But they've surrendered touchdowns on six of 13 opportunities (46.2 percent), tied for 10th in the NFL. Baltimore has held the opposing team to no points twice.
Ngata felt the Ravens were improving on run defense. In Week 1, they gave up 91 yards to Cincinnati's BenJarvus Green-Ellis and 129 total rushing yards. In Week 2, they held Philadelphia Pro Bowler LeSean McCoy to 81 yards (3.2 average) and again 129 total rushing yards. In Week 3, Baltimore held New England's Stevan Ridley to just 37 yards (2.8 average) and the Patriots to just 77 total rushing yards.
Overall, the Ravens aren't too concerned as they are 2-1.
"We want to be a great defense, we want to limit the yards and all that," Cody said. "But Dean Pees told us himself that he's not worried about the yards. He's worried about the win. Not all games are going to be pretty."