Bismillah wassalaamu alaykum wa Rahmatullahi wa Barakatuh.
It appears to me that Muslim men, perhaps more so those who reverted and had relationships before Islam, have a conflict. They want the piety and beauty of an Islamic wife and family, but the base desires of sexual freedom and a woman like those on the street still exist. After all, the biggest jihad is against our nafs.
They want sexual excitement and spontaneity which dies a slow death with having a family. Pregnancy,babies, children, etc. make acting on impulse and dressing to tempt and invite virtually impossible. Men have to repress their desires, do without. I'm not talking about actual intercourse, as a wife can be willing every day maashaa Allah, but the whole atmosphere and freedom of interaction is what they don't have. They may wish for the days before children came along, or yearn for the day when the children are all grown, to regain that free and impetuous sexuality.
A busy wife is a blessing, and a turn off. She doesn't have time for doting on her husband; other people and things take priority. So, while a man may understand and appreciate her ability to run the house, raise the children, and do all the extra she does, he is not getting what he wants/needs to feel satisfied. These are the biggest factors I see and have understood from my husband - polygany analyst and survivor - over the years, maashaa Allah.
Men have different tolerance levels and will handle the situation in a different way. Most think they are too much man for one woman and seek other outlets. Problem is, for those who don't actually try polygany, they don't see the reality of it. Unless they take a childless woman who won't have any or a woman whose children are all grown, they will have the same exact situation with number two or three or four, as with number one as soon as they fall pregnant. They also don't have the chance to compare the women to see and appreciate those things in number one that subsequent wives lack.
Depending on how the man deals with his repressed feelings, he could be irritable, distant, withdrawn, or immerse himself in other activities - halaal or haraam. He will struggle with himself about his repressed feelings and will not want to admit that changes in his behavior are due to them...or justify his behavior due to them.
My husband, upon reading this, agreed that I have just about summed it up alhamdulillah!