It’s not uncommon today to see the Confederate flag at the center of controversy, particularly when it comes to violence and the way we treat race. But it’s worth noting that the situation—the history of the flag—isn’t as cut and dry as some would have you believe.

There were actually three different official Confederate flags during the Civil War. What we think of today as the Confederate flag has very little to do with the war or the Southerners who fought in it.

The original flag, called the “Stars and Bars,” was approved in 1861. It featured a dark blue field in the upper left corner and three stripes, two red and one white. Its seven stars represented the seven states wanting to secede from the union: South Carolina, Mississippi, Florida, Alabama, Georgia, Louisiana, and Texas. It looked so much like the Union flag, however, that soldiers couldn’t tell the difference in the heat of battle.

That led to the second Confederate flag, proposed by General P.G.T. Beauregard. The “Stainless Banner” was a blue St. Andrew’s Cross with white stars on a red field. In battle, the design was added to a white flag. This led to problem number two: When the wind wasn’t blowing, the flag looked like a white flag of surrender.

In came flag number three, the “Blood-Stained Banner.” It wasn’t the official flag of the Confederacy, but it was flown briefly in battle, including by General Robert E. Lee in northern Virginia. This is the flag we think of today as the Confederate flag.

After the Civil War was over, the battle flag wasn’t really around much, though it occasionally showed up at events commemorating fallen soldiers. Most notably, the flag was raised over the State House in South Carolina in 1961 to commemorate the 100th anniversary of the beginning of the Civil War. It was later moved to a soldiers’ monument, where it’s currently protected by the 2000 Heritage Act.

So why the history lesson? Because it’s important to note that there’s much more to the Confederate flag and the culture it represents than just its recent status as a symbol of racism and oppression. In fact, the majority of Americans surveyed in a 2011 Pew Research poll (58%) had a completely neutral reaction to seeing the flag on display.

It’s easy to dismiss the flag as a unilaterally negative thing to be censored and removed from our sight. And there’s no denying that it definitely has some negative connotations—if we’re talking about its most recent version.

But we can’t just dismiss the fact that the flag also represents the (admittedly problematic) history of an entire segment of the U.S. population, making it perfectly understandable that there’s still some emotional attachment to it, particularly for those knowledgeable about its history and meaning.