A house divided

In 1790, when the Census was carried out, the Union contained 3.9 million inhabitants. Virginia being the most populous state, with19 percent of the total population of  the Union: 747,610. The problem was that 293,000 of them were slaves. Many other slave states were in a similar position, with nearly half of their populations being slaves.
During the years of the Articles of Confederation, the south argued that slaves should not count as part of the population, since the could not vote, and the north argued that they should. This was not done out of kindness or cruelty on either side. The reason for this was that the taxes paid by each state under the articles was apportioned by population. The more people, the more taxes.
During the ratification and writing of the Constitution, the states changed positions. The new problem was that population equaled votes in the electoral college and representatives in the legislature. The northern states did not want to see the southern states dominate the political landscape, and so the three fifths compromise was born. The states agreed that the slaves, who could not vote, would count as three-fifths of a person for the purposes of calculating representation in the House of Representatives, and for the purposes of calculating electoral votes.
Even so, these 'extra' votes that were allowed by counting slaves allowed the south to dominate the political scene for the first 80 years of the union's existence. It was only through the passage of the 13th and 14th amendments that mooted the three fifths compromise and would have shifted political power to the southern states.
The Fifteenth Amendment mandated that the right to vote could not be denied on the basis of race. The north could not have that, so they proposed literacy exams, poll taxes, and other means of disenfranchising the black vote. This would have the effect of disproportionately disenfranchising blacks.
This was about politics and about power, just as the issues are today. You can trace the political questions about illegal immigration and other issues back to politics. Politics has never been about right and wrong, only about power and the desire for more of it.