Goldenberg explains how Cain became associated with black skin.
David Goldenberg, The Curse of Ham: Race and Slavery in Early Judaism, Christianity, and Islam (Princeton: Princeton University Press, 2003), 181
If this is so, we ought first to look at the Peshiṭta, the Syriac translation of the Bible, and when we do we find the answer to our problem. The Peshiṭta renders the text in Gen 4:5 wʾtbʾš lqʾyn ṭb wʾtkmrʾpwhy "and Cain was very displeased and his face ʾtkmr.” The Syriac word ʾtkmr is a form of the root kmr, which commonly means “to be black” and then, in a transferred sense, “to be sad.” These two meanings of kmr are behind Cain’s change of color in the Adam-book, which would have understood tkmr to mean “became black” rather than “became sad.”