Many buildings have both male and female qualities. Perhaps the texture is male, and the shape female. Or, the color is female, but the proportions... Well, you get the point. A building that has both male and female characteristics might be called androgynous.