Zimmer began his career studying Sanskrit and linguistics at the University of Berlin where he graduated in 1913. Between 1920-24 he lectured at the University of Greifswald, moving to Heidelberg to fill the Chair of Indian Philology.
In 1938 he was dismissed by the Nazis, and he emigrated to England where between 1939-40 he taught at Balliol College, Oxford. In 1940 he moved to New York where he eventually accepted a Visiting Lecturer position in Philosophy at Columbia University. One of his students during this time period was Joseph Campbell. He died there, of pneumonia, the following year (1943).
Zimmer's method was to examine religious images using their sacred significance as a key to their psychic transformation. His use of (Indian) philosophy and religious history to interpret art was at odds with traditional scholarship. His vast knowledge of Hindu mythology and philosophy (particularly Puranic and Tantric works) gave him insights into the art, insights that were appreciated by Joseph Campbell among others. Campbell edited many of Zimmer's writings after his death. The psychiatrist Carl Jung also developed a long-standing relationship with Zimmer, and incidentally edited a volume of Zimmer's entitled Der Weg zum Selbst (The Way to the Self)(the two men first met in 1932, after which Zimmer, along with Richard Wilhelm, became one of the few male friends of Jung). Zimmer is credited by many for the popularizing of South Asian art in the West.
He married in 1929 Christiane, daughter of Hugo von Hofmannsthal.