There were signs that Jerry Richardson, owner of the Carolina Panthers, had a coarse view of people of color. Often, they were in plain sight. 

Rewind back to 2011, when the NFL was embroiled in a labor dispute. The owners had locked the players out, and the two sides were furiously negotiating a new collective bargaining agreement. One of the men representing the owners was Richardson.

During the various meetings, Richardson spoke of the players in such derogatory terms that it shocked some in the room, according to one person who was there. This person also felt Richardson was speaking so negatively because the player base is mostly African-American.

Richardson, this person remembers, spoke about the players as if they were servants to the owners.

Some in the room came to give Richardson a nickname: Foghorn Leghorn (the blustery Looney Tunes cartoon character with a Southern accent).

Fast-forward to now and the devastating report from Sports Illustrated that stated Richardson, on at least four occasions, reached significant monetary settlements for inappropriate workplace comments to female employees, and in at least one instance directed a racial slur to a black scout.

The NFL is investigating, and the Panthers have announced that Richardson will sell the team.

Considering how quickly the NFL and the team appeared to move, numerous league officials believe the Richardson story could end up being one of the biggest black eyes for the league since the Ray Rice fiasco.

"This is potentially the ugliest story our league has faced in a long time," said one league official.

It's believed the NFL's investigation will produce more harassment and racial allegations against Richardson. And it could uncover other workplace violations and racial issues among other franchises.

One team executive told B/R the Richardson case could open football's Pandora's box. Indeed, there is some genuine panic across the NFL over workplace harassment.

While the league has guidelines under its personal conduct policy to deal with workplace harassment and discrimination for its employees, this Panthers case doesn't involve some mid-level front office executive. This is an owner. And it's unclear whether the NFL is equipped to deal with allegations against one of its primary members.

The league will maintain that NFL Commissioner Roger Goodell has the authority to discipline an owner. But history has shown the commissioner and the league office haven't been exactly adept at the whole discipline thing. Goodell has botched almost every case he has handled, including Rice, Deflategate, Josh Brown and others.

It's also possible other owners could mount some type of response, or the league could hire an outside agency. Or if Richardson were part of a larger ownership group, the other partners could discipline him. But all of those solutions are problematic.

The NFL said it has hired an outside firm to investigate Richardson. But the league used one with Brady and came up with "more probable than not."

Owners in the NFL have long been positioned as practically all-powerful; they are one in an exclusive club of 32 entities. To see true justice done, there may be a need to create an entirely new set of rules for these incredible times.

All of this reminds me of a story former Raiders CEO Amy Trask—full disclosure: a friend—tells in her book (which I edited), You Negotiate Like a Girl, Reflections on a Career in the National Football League. At one of her first league meetings, an owner, not realizing that Trask was there on behalf of the Raiders, asked Trask to get him coffee. No woman who wasn't an owner had ever attended one of the meetings before.

Trask believed it was an honest mistake, but what happened at another owners meetings not long after that one wasn't.

"There was a fairly heated discussion going on on the floor of the meeting," Trask remembered in an interview for the radio program Only a Game, "and I stood up to make a point, and when I was done, the owner of another team—and I have never shared which owner, nor will I ever—he stood up and said, 'Listen, Girlie.'

"I just laughed. I laughed aloud...at the silliness that a man in the then-20th century just called an executive from another club 'Girlie.' Many women and men I respect have said to me that I should have chastised him for that. Well, you know what? I know that I would rather be yelled at, scolded, hollered at than I would be ignored or dismissed with laughter. And frankly, I think laughing at him was far more effective than any speech I could have made."

It worked. The owner sat down, embarrassed.

Of course, not every woman in the NFL is a high-ranking front office executive, as Trask was. Nor will every owner stand down.

And while it may not be too hard to laugh off a belittling comment from a rival, when the person asking you for foot massages or brushing against you (as Richardson is alleged to have done) is the one signing your paycheck, making an effective response can become complicated. If the Richardson stories prove true, NFL workplaces may be far worse than we ever knew.

Now the question is: What can the NFL do if the league's most powerful people commit these offenses?

Mike Freeman covers the NFL for Bleacher Report. Follow him on Twitter: @mikefreemanNFL.