I know your mother's responses are confusing and invalidating, I'm so sorry for the suffering you're experiencing because of it.
Your mother's responses are actually very common in this kind of situation. I have read about such responses in at least one resource for recovery from childhood sexual abuse. The person who receives the disclosure expresses belief and offers support, and then takes it back. Saying she believes you believe it seems that she validates you, yet denies the event occurred, so it's like gaslighting, a total redefinition of your experienced reality to make herself feel better, to both accept and reject you.
You asked why she doesn't believe you. I offer here one possible perspective.
Believing you creates cognitive dissonance.
The act of molestation is a monster. But it gets conflated with those who perpetrate, are harmed by, or fail to prevent or protect from it.
Your brother perpetrated a monstrous act. Your mother loves him. Because of conflation, to accept he did so makes someone she loves a monster. This creates cognitive dissonance and she has to find a means to resolve for herself the conflict between two opposites in one person: a son worthy of her love, and a monster.
She failed to prevent or protect you from the monstrous act. It is hyperbole to think only a monster would do so, but hyperbole means her identity may also be conflated with the act. She cannot be a good person/mother to both of you and be a monster, so perhaps she resolves the inner conflict by validating both you and your brother. The monster therefore cannot exist, but both of her children still can.
You, however, cannot deny the monster exists, and you cannot deny that you exist. Your mother's reframing may create for you cognitive dissonance that suggests you exist, but your perceptions are disabled, as if you were visually impaired, such as blindness or depth perception issues.
Often in family abuse, not just sexual, the victim becomes, to those who hear their honest identification of abuse, conflated with the monster for identifying it out loud. They are accused of trying to destroy the whole family unit, because denial is part of the glue that holds the family unit together; otherwise, the family becomes monstrous if the act is acknowledged. No one would want to embrace a monster, and so as long as everyone is conflated with the act, in order to relieve the cognitive dissonance, at least one member must be identified as the monster and rejected: the perpetrator, the victim, the bystander, or the family unit.
Generally, the family unit comes first, and the victim comes last, because the victim is perceived as the primary threat to all the rest. The actual monster, the act, remains unaddressed. The perpetrator is not held accountable. The bystander is not held accountable. The family is not held accountable. The victim is the only one who is not accountable, yet is made accountable for everyone's suffering that the monster and the perpetrator caused. The victim is disbelieved and demanded to stop pointing out the monster, or be punished and/or banned for causing the suffering the monster instigated.
The family unit is responsible for protection. Once aware of the monster, it is responsible to your brother to point out his active engagement with it; to point out the monster to the bystanders in order to increase their awareness for the safety of the family unit and its members that monsters threaten; and to the victim to atone for not providing protection with awareness, acknowledgement, acceptance, inclusion, and support.
You have fulfilled your responsibility to protect your body and sense of self by acknowledging the existence of the monster and its perpetrator, and that they caused you harm. You are holding them accountable and not yourself, who is blameless.
You have fulfilled your responsibility to the family unit by pointing out there is a monster and a perpetrator.
I know from experience it is difficult to not have the acknowledgement and support of the family unit. I hope there is at least some comfort in knowing how it functions, that you are not to blame, and that you are in fact aware, brave, and very powerful.
From one childhood family abuse victim (not sexual) to another, I compassionately empathize with and support you. I hope you find empowerment. I wish for your well-being, happiness, and safety, and your freedom from all monsters and the past harm caused by them.
