The story (mostly false) goes that Marie Antoinette, shortly before becoming a foot shorter, was talking with an advisor. The advisor told her that the peasants had no bread, and she responded, "Let them eat cake!" Ignoring the historical accuracy of the quote or the players, the language and its relation to food is what I'm interested in. (History? Pah!)
A more accurate quote is closer to, "The peasants don't have baguettes*," and, "So let them eat brioche!" The thing is, back when the phrase was popularly introduced in the English Language, there wasn't really an appreciation of the many kinds of french breads that exist as there are today. And, truth be told, I suspect a great many people still don't have a full enough appreciation of the different bread types, so it's not like a more accurate translation is going to work its way into the mainstream.
Still, a guy can dream.
*- Okay, okay. Baguette is a shape, and the actual type of bread is the lean bread known as "le pain." However, "pain" being a very distinct word in English that nobody uses for bread, it would completely confuse people. Frankly, I'm very close to banning this phrase in English or maybe altogether. There are just too many problems with it.