The rights groups say that, along with other people who do not conform to the gender binary, trans people can face intrusive challenges when using male or female toilets.
The government hopes to introduce the legislation, which will need to be approved by Parliament, later this year.
The change to building regulations will also allow contained, universal toilets in addition to single-sex toilets where space allows, or instead of single-sex toilets where there is not enough space.
A universal toilet is defined by the government as a self-contained room with a toilet and sink for individual use.
The government said gender neutral toilets were leading to increasing waiting in shared queues and less privacy and dignity.
“These regulations will guide organisations to design unisex and single-sex toilets, ending the rise of so-called gender-neutral mixed sex toilet spaces, which deny privacy and dignity to both men and women, ” Equalities Minister Kemi Badenoch said.
“Today’s announcement will also create better provision for women so that our particular biological, health and sanitary needs are met.”
When asked on ITV’s Good Morning Britain if gender-neutral toilets would persuade people to vote Conservative following poor local election results, Women’s Health Minister Maria Caulfield insisted: “This is an important issue for women.”
The new legislation will also apply to buildings going through major refurbishment. Exemptions include care homes, cells in custody suites and schools.
News has asked the Department for Levelling Up, Housing and Communities when it intends bring forward the long-talked about legislation before Parliament.
A consultation, which received 17,000 responses, found 81% agreed with the intention for separate single-sex toilet facilities and 82% agreed with the intention to provide universal toilets where space allows.
Mermaids, a transgender youth support charity, said in response to the government proposals: “In order to ensure everyone is served fairly and that everyone can feel comfortable using public toilet facilities, not only are gender-specific facilities in which trans people can feel safe in using vital, but gender-neutral facilities are also greatly necessary to ensure non-binary people’s experiences with toilet facilities are one of comfort.”
