Joseph became a great athlete. He decided to pursue his athletic career in baseball. He received a baseball scholarship in 1921 thru 1925 to Georgia Tech. While playing starting catcher for Tech, he studied Civil Engineering. Georgia Tech won the Southern Conference. Papa Joe batted .355 at 76 at bats. After his college days, Joseph was drafted and signed a professional contract with the Cleveland Indians. He had a long career, expanding over 20 years as a player and a manager. He traveled all over the United States, played for numerous teams and won many championships. He was a member of the Philadelphia Athletics 1931 World Series Team. He played for Portland of the Pacific Coast League, which was the Pacific Coast Champions in 1932. He was considered the best receiver in the Coast League. His baseball career also took him to the Southern League, where he played for Atlanta and Birmingham teams. As a manager in Kanapolis, his team had a winning streak that won 57 out of 70 games. An 81% winning record. Kanapolis won two conference championships under Palmisano's management.