I think "selfish" has unfairly negative connotations.
Yes, it's selfish to want to keep somebody who doesn't want to stay, but that's because love is selfish. Love is also beautiful and, I think, as vital as food or water, the desire to consume which is also entirely selfish, but who would tell you it's wrong to want to eat and drink? Nobody worth listening to.
My point is this: there is absolutely nothing wrong with considering the effect his death would have on you. In fact, I think I think it would be an insult to not feel pain in response to it, even though, considering you're here, you understand that it's what he wants. You love him, so of course it would be painful for you. That's nothing to feel ashamed of. I guarantee he's considered the impact it would have on you and I guarantee that the thought of you in pain is painful for him in turn- because he loves you too, and so your pain hurts him as well.
That's how important selfishness can be. It motivates us to do some bad things, sure, but it's also what motivates us to protect and comfort the ones we love: so we don't have to experience the pain of them experiencing pain. That's a side effect of love. And that's amazing.
Your "selfishness" is just evidence of how much he means to you. You have every right to feel this way. It is selfish. And that is okay.