I bet the slaves and servants queried slavery, but the people in power weren't listening to them. Also, when kidnapped, hostages sometimes develop Stockholm Syndrome (which is affection for their kidnappers) as an psychological survival mechanism. I suspect any "affection" she felt for her slave owner was born of that desire to protect herself and her children from harm.