No offence...

Blue has answered this very, very well. I was going to say things along similar lines, but I'll cut it down to this:

You're right, the Union was racist, the CSA was racist. But we shouldn't judge historical characters by our standards, but their contemporary standards. What would now be considered homophobic would be considered perfectly normal. Would we say the Founding Fathers were homophobes? No.

But, in the context of that period, there can still be such a thing as a racist state. If a state if FORMED on the principle that one race can own another, when slavery has been illegal in Great Britain and the Northern United States for decades, and they're clinging onto their 'negroes,' then they are racist by contemporary standards.

I simply don't think the statement, "The CSA wasn't (mainly) racist," is a sensible one. Going back so far, when racism wasn't neccecarily viewed as a 'bad thing,' it still means something, and the CSA was the embodiment of that.