Why voters don't join my party

I had this problem in many of my games. Once I reach nearly 100% voting intention and similarly high approval the people with the highest approval won’t join my party, but people with 80% approval or something like that will. I would excuse this if it was a temporary problem, as voters adjust their loyalties slowly, but it happens almost always and doesn’t go away no matter how long I play, making my party membership small compared to my approval ratings.