The origins of the term "cracker" are unknown, with multiple theories to explain its history. 
One theory holds that the term comes from the common diet of poor whites. According to the 1911 edition of the Encyclopedia Britannica, it is a term of contempt for the "poor" or "mean whites," particularly of Georgia and Florida. Britannica notes that the term dates back to the American Revolution, and is derived from the cracked corn which formed their staple food. (In British English "mean" is also a term for tightfistedness, with no malice implied.)
Another theory is that slaver foremen in the antebellum South used bullwhips to discipline African slaves, with such use of the whip being described as 'cracking the whip'. In this folk etymology the white foremen who cracked these whips were thus known as 'crackers'.   
An alternate whip-related theory is that the term is linked to early Florida cattle herders (Florida crackers) that traditionally used whips to herd wild Spanish cattle. These cowboys were distinct from the Spanish vaqueros of Florida. The crack of the herders' whips could be heard for great distances when they were used to round cattle in pens and to keep the cows on a given track. Also, "cracker" has historically been used to refer to those engaged in the low paying job of cracking pecans and other nuts in Georgia and throughout the southeast U.S.
Yet another theory is that the term derives from an Elizabethan word used to describe braggarts. The original root of this is the Middle English word crack meaning "entertaining conversation" (one may be said to "crack" a joke); this term and the alternate spelling craic are still in use in Ireland, Scotland and Northern England. It is documented in Shakespeare's King John (1595): "What cracker is this same that deafs our ears with this abundance of superfluous breath?"
Historically the word suggested poor, white rural Americans with little formal education. Historians point out the term originally referred to the strong Scots-Irish of the back country (as opposed to the English of the seacoast). Thus a sociologist reported in 1913: "As the plantations expanded these freed men (formerly bond servants) were pushed further and further back upon the more and more sterile soil. They became 'pinelanders', 'corn-crackers', or 'crackers'." 
I should explain to your Lordship what is meant by Crackers; a name they have got from being great boasters; they are a lawless set of rascalls on the frontiers of Virginia, Maryland, the Carolinas, and Georgia, who often change their places of abode.
Frederick Law Olmsted, a prominent landscape architect from Connecticut, visited the South as a journalist in the 1850s and wrote that "some crackers owned a good many Negroes, and were by no means so poor as their appearance indicated."
In 1947, the student body of Florida State University voted for the name of their current athletic symbol of "Seminoles," out of more than 100 choices. The other finalists, in order of finish, included Crackers, Statesmen, Tarpons and Fighting Warriors.
Crackin' Good Snacks (a division of Winn-Dixie, a Southern grocery chain) has sold crackers similar to Ritz crackers under the name "Georgia Crackers". They sometimes were packaged in a red tin with a picture of The Crescent, an antebellum plantation house in Valdosta, Georgia.
"Cracker" has also been used as a proud or jocular self-description. With the huge influx of new residents from the North, "cracker" is used informally by some white residents of Florida and Georgia ("Florida cracker" or "Georgia cracker") to indicate that their family has lived there for many generations. However, the term "white cracker" is not always used self-referentially and remains a racist term to many in the region.
Before the Milwaukee Braves baseball team moved to Atlanta, Georgia, the Atlanta minor league baseball team was known as the "Atlanta Crackers". The team existed under this name from 1901 until 1965. They were members of the Southern Association from their inception until 1961, and members of the International League from 1961 until they were moved to Richmond, Virginia in 1965. However, it is suggested the name was derived from players "cracking" the baseball bat and this origin makes sense when considering the Atlanta Negro League Baseball team was known as the "Atlanta Black Crackers".
In 2008, former President Bill Clinton used the term "cracker" on Larry King Live to describe white voters he was attempting to win over for Barack Obama: "You know, they think that because of who I am and where my politic[al] base has traditionally been, they may want me to go sort of hustle up what Lawton Chiles used to call the 'cracker vote' there."
Singer-songwriter Randy Newman, on his socio-politically themed album Good Old Boys (1974) uses the term "cracker" on the song "Kingfish" ("I'm a cracker, You one too, Gonna take good care of you"). The song's subject is Huey Long, populist Governor and then Senator for Louisiana (1928-35).
Major, Clarence (1994). Juba to Jive: A Dictionary of African-American Slang. Puffin Books.