Forgiveness is a major component in relationship and restoration. It can be difficult to try to apply it within yourself. Perhaps a better view might be to use the shame and guilt for having done something wrong to learn, motivate to do better, and recognize the need to make restitution, apology, or express contrition.
However, once everything useful has been extracted, shame and guilt can be left behind. Holding on to it may indicate a problem with a sort of self-harm, need to feel control, or even reduce anxiety. These dynamics should be investigated to see if what is being retained is done so in an unhealthy way.
If we take more of a learning and improving model of life, we may have an easier time letting go of past mistakes. If we use a failing or error model of life, we can more easily see ourselves as defective.