The "99 Problems" hitmaker was brought up single-handedly by his mother, Gloria Carter, from the age of 11 after his father, Adnes Reeves, walked out on him and his three siblings when he was a child.
The star later learned his dad left after his brother's tragic murder in the streets of Brooklyn.
Jay-Z was reunited with Reeves after the release of "The Black Album" in 2003, and he admits that patching up their differences was vital in order for him to become a good husband to Beyonce, whom he wed in 2008.
Speaking on Howard Stern's radio show on Monday, Jay said, "I'd dropped like, seven albums. It wasn't like he saw me make a little bit of money and ran to the money, it wasn't that. He was still who he was.
RELATED: Jay-Z says Beyonce is not pregnant
"My mom actually made that [reunion] happen. She's super strong ... I'd pretty much built up a wall to that emotion, which wasn't healthy. I went in [to the reunion meeting] with a lot of anger, thinking I was gonna tell him off, and it didn't happen like that ..."
Reeves was battling a long-term illness when he met up with Jay-Z for the first time in over two decades, and the rapper was unaware he would lose his dad within weeks.
Jay-Z, 40, recalls, "He was actually a very tall, strong man, I had no idea that he would die the next month. He looked healthy and strong. He had a bad liver from drinking and doing all the other things he'd been doing his entire life. We didn't discuss [his illness], we had so much to discuss that day."
RELATED: See more pics of Jay-Z
Ultimately, the superstar is thankful he found it in his heart to fully forgive Reeves.
He adds, "You don't even get to the point because you don't allow yourself. Your father, growing up, is like Superman to you, he's everything, so once you feel that sort of hurt, you never allow yourself to get close to that feeling. You don't even wanna feel that good because you know the flip-side of that. You don't let people get that close so they won't hurt you."