First off, Sita would have had sex with a demonic king, which would have been the major issue. Secondly, it was Sita herself who decided to go off to the jungle to preserve her husband's family name. Rama knew that Sita had remained faithful, and had she had sex with the demonic king, he wouldn't have cared because he knew she would only waver if she had been raped. Laxmana, Rama's brother even threatened anyone who spread rumors soiling Sita's pure name. The issue however was that the relationship of Rama's kingdom was extremely close between the king and the people, meaning they had absolute trust in the king, which was the reason it was the best kingdom in the land. So, to preserve this dynamic, Sita forced Rama to send her off.
As for the question, of holding to chastity, the issue in much of western culture is an inability to refrain from the temptation of sex, which has projected itself onto an idea of chastity. Sex is so embraced by the media and pushed so much onto society, that it surrounds us, which is probably why chastity seems like such a foreign and strange idea to most people. The truth is, this just serves to mask the unwillingness of certain people to restrain their own desires, something that was upheld in earlier generations. Since around 1970, the media and the prevalent culture has been embracing the idea of doing what makes you happy. Make your own choices, be yourself, do what you want, and so people do what they want, namely sex, ignoring much of the consequences on other peoples lives, as well as often times their own. In earlier generations, the idea was on supporting society and helping others and collaborative work. One was still an individual, but an individual serving the higher purpose of supporting society. The difference was that people suppressed their desires to make themselves stronger and to avoid harming society and themselves, worrying heavily about the consequences. The result was an avoidance of an addiction to sex, one of the most powerful desires, as well as something with potentially large consequences.