Phelan was born in Aberdeen, Mississippi to James Phelan Sr., a member of the Confederate Congress, and Eliza J Phelan. He was of Irish descent on his father's side, and Scottish descent on his mother' side. He attended various schools. In 1874 he went abroad, and in 1878 received his Ph.D. from the University of Leipzig, having written his dissertation on the life and works of Philip Massinger. Returning to the United States, he married Mary Early of Virginia, with whom he had three children.