Yes.
Under Section 20 of the OHS Act 2000, if the workplace designates that particular equipment is to be used then you must co-operate with the safety requirements of your employer. Both your employer and yourself may be fined if you are not wearing appropriate PPE.
However, the OHS Regulation 2001 provides that PPE must be appropriate to the individual and controls the risk for that individual. This means that the employer may need to consider alternatives if the PPE provided to you does not adequately protect your health and safety. (The level of comfort and fit can be subjective and should be resolved in consultation.)
For example if you have a medical condition and provide a medical certificate indicating that certain PPE worsens the medical condition, or that the use of certain PPE is inappropriate because of the condition, then your employer may have to consider providing specially designed PPE (if possible) or placing you in an area where the PPE in question is not required.
The employer must also ensure the person using PPE is provided with instruction and training, which may affect the level of comfort of the equipment. The employer must also ensure the equipment is properly maintained, repaired and provided in a clean and hygienic condition.