He and his business partner, Henry W. Sage, together made a great deal of money selling lumber from forests in New York, Wisconsin and Michigan, and operating a large lumber mill in Wenona, Michigan, now part of Bay City, Michigan. They also co-founded the town of Wenona.
He married Rhoda Charlotte Southworth in Dryden. She was born September 19, 1819, also in Dryden, the daughter of John Southworth of Salisbury, New York and Nancy Ellis Southworth of Dryden. She gave birth to his only daughter, Jennie McGraw. After Rhoda's death in 1847, he married her sister, Nancy Amelia Southworth in 1849.
John died May 4, 1877 in Ithaca, New York. His daughter, Jennie inherited his large fortune.
Upon Jennie's death, some of this fortune was bequeathed to Cornell. A dispute over this gift led to the Great Will Case, ultimately decided by the United States Supreme Court against Cornell in Cornell Univ. v. Fiske et al. (1890). His former business partner, Henry Sage, made a large donation to Cornell in the name of Jennie to replace the lost funds.
The McGraw name graces the principal clock tower of Cornell. Additionally, McGraw Hall is one of the buildings on the main arts quad of Cornell University. Among other uses, it was the first home of Cornell's business school, now known as the S.C. Johnson Graduate School of Management and located in Sage Hall.