Goh was a son of a rich Malacca family. Although his father, Goh Leng Inn, was a manager of a rubber plantation, his mother was from the family that produced famous Malaysian politicians Tun Tan Cheng Lock and his son, Tun Tan Siew Sin, who would someday be Goh's lifelong political opponent.
After studying at Anglo-Chinese School (ACS) in Singapore, Goh went on to graduate from Raffles College (today the National University of Singapore) with a Diploma in Arts, and entered the Civil Service to focus on social welfare issues. Shortly after the start of the Second World War, Goh joined the Singapore Volunteer Corps, a local militia, but after the fall of Singapore he returned to his previous work. In 1946, he joined the Department of Social Welfare, and was active in post-war administration. During the Japanese occupation in 1942, Goh married Alice Woon and they had a son.
Following the Second World War, Goh won a scholarship, which enabled him to pursue a degree at the London School of Economics (LSE). Goh graduated with first class honours in economics and with the help of a University of London scholarship, awarded in 1951, was able to continue with doctoral studies. Goh completed his Ph.D. in Economics in 1954, and returned to the Department of Social Welfare, where he served as the Director of the Social and Economic Research Division.
During his time in London, Goh met fellow students that were seeking independence for Malaya, including Lee Kuan Yew and Toh Chin Chye. A student discussion group, the Malayan Forum was organized with Goh as the founding chairman. Following his return to Singapore, Goh detached himself from active politics, but retained ties to the People's Action Party (PAP), of which he was a member. In August, 1958, Goh resigned from the Civil Service to work full-time for the (PAP), and was a key member of the Central Executive Committee. He was later to become its vice-chairman. On a PAP ticket, Goh successfully contested the Kreta Ayer seat in the 1959 Legislative Assembly elections, and joined the first government of Prime Minister Lee Kuan Yew as Minister of Finance. In this role, he was to assume stewardship of Singapore's economy, and in the following decade, Goh was to serve as the chief architect of the country's economic development. He came to believe that capitalist principles, properly nurtured, was efficacious to the generation of wealth. Hon Sui Sen succeeded him as the Finance Minister in 1970.
In the 1960s, there were great pressures from communist agitators working through the Chinese-medium schools and trade unions. Divisions existed within the PAP as well, with the pro-communist faction working to wrest control of the party from the moderate wing, of which Goh and Lee Kuan Yew were key members. A key source of division was the issue of merger with Malaya, to form a new state of Malaysia. Goh and the moderates believed that this was a necessary condition of Singapore's economic development because Malaya was a key economic hinterland; merger would also provide an alternate vision against communism for Singapore's Chinese majority. In July 1961, sixteen members of the pro-communist faction broke away from the PAP to form the Barisan Sosialis, and captured control of the main trade unions.
The Singaporean government won approval from Tunku Abdul Rahman for a merger in 1961, with the Tungku being motivated by a desire to stabilise the security situation in Singapore, and notably to neutralize the perceived communist threat. Singapore merged with Malaya and the British Borneo states in 1963 to form the Federation of Malaysia. Merger however proved to be problematic for the Singaporean leaders. There was a clash of fundamental principles, both political and economic; notably on the issue of Malay dominance. Communitarian violence in 1964 was inflamed in Singapore by Malay and Chinese activists. Goh played a crucial role in orchestrating the subsequent secession of Singapore from the federation in August 1965. He would later claim this was the "best thing that ever happened to Singapore".
Upon independence in 1965, Goh also became Minister for the Interior and Defence until 1967, assuming responsibilities for strengthening Singapore's military and domestic security capabilities. A key policy was the creation of National Service, a mandatory conscription system for able-bodied young males. In 1979, Goh moved on to the Education Ministry, where his Goh Report greatly influenced the development of Singapore's education system, introducing key policies such as streaming and religious education.
Goh retired as Deputy Prime Minister in December 1984, due to personal reasons.