I wouldn't spend too much time trying to understand "why" your father couldn't be the parent you wanted or needed. I have been down a similar path and have found no definitive answer.
My eighth date with my wife was our wedding. At eighteen she was desperate to get away from an over-protective mother. At twenty, I too was desperate ... I wanted to prove that I was straight. I had to be straight because in the early 1970s, the alternative was too loathsome to contemplate.
We got to know each other as a young couple living in the northeastern Thai town of Nakhon Phanom, where I was stationed with the U.S. Air Force. We got along rather well, but the relationship was devoid of passion. I was coming to terms with the fact that despite all of my persistent denial, there wasn't a heterosexual cell in my body.
I was also realizing that my wife simply didn't feel the way that other people did. Her emotions were perfunctory ... she was highly sexual, but we had no romance. Rather, we established a casual friendship. When we divorced after seven years of marriage, I became the primary custodian of our son. He never felt loved by his mother and when he reached the age of fifteen he stopped his weekend visits to her house.
I will always remember a phone call that I once received from her (we were then living about thirty miles apart). Could she park her vehicle in my carport for a few days, she asked, and could I drive her to the airport? I answered in the affirmative, but naturally wondered what necessitated all of this. "Oh, it''s Dad," she replied with her usual flat affect. "Dad kicked."
My ex-wife is in no way a bad person. To the contrary, she was always responsible and conscientious. She was simply not equipped with depth of emotion that the majority of people appear to have. That isn't a fault--even though it prevented her from meeting my son's yearning for maternal affection. It simply indicates the range in which she normally moves on the social emotional continuum.
Despite my attempts to explain this to my son--he is now in his late forties--he has virtually no relationship with his mother. It may well be that his mother and your father couldn't activate their parental instincts when you and he needed them to do so for the simple reason that they were never gifted with those instincts. I know that doesn't make the wounds any less difficult to heal. But if nothing else, please know that you are far from being alone.