Why should I have to spend time with my Dad's 23 year old girlfriend?

You don't have to like or be around your dad's jumpoffs if you don't care to. But, you should respect her. Don't be crass or rude. The person you care about in this situation is your dad. Therefore, that is the only person you need to be concerned about, in my opinion. If something you are doing is hurting your dad, then maybe you should rethink your actions. If your dad is fine with you being distant from his GF, then I don't see a problem with it. Now, if your dad marries this woman, you will need to develop a cordial relationship with her out of respect for your dad.