Born in [[Bordeaux], he was the son and grandson of musicians of Italian descent. From the age of 8 he played flageolet and accordeon, and then began violin studies with Baudoin. From 1855 Colonne studied at the Conservatoire in Paris, where he won first prizes in both harmony and violin. For almost a decade (1858-67) he was first violinist at the Opéra in Paris, as well as playing second violin in the Quatuor Lamoureux. In 1871 he directed concerts at the Grand-Hotel and Massenet's music for the staging of Les Érinnyes in 1873.
In 1873, Colonne, along with the music publisher Georges Hartmann, founded the "Concert National" at the Odéon Théatre. Two years later, the venue changed to the Théatre du Châtelet and the name of the enterprise was changed to 'The Association Artistique du Châtelet'. The Association's performances eventually became known as Concerts Colonne; and this name continued to be used until the 1960s.
In 1878 he had met Tchaikovsky during the Russian composer's visit to Paris, and as well as giving the premiere of his 4th Symphony remained in contact, which led to 'exchange' concert trips for Colonne in Russia.
Noted for his interest in Berlioz (who was then more highly regarded in the English- and German-speaking countries than in France), Colonne also stood out for his support of the music of Wagner, Mahler and Saint-Saëns. He introduced the descriptive note into programme bookets. He was also the first conductor of eminence to make commercial gramophone (phonograph) records, for the Pathé company in 1906. The works ranged from Beethoven to Widor, and announcements by Colonne were included.