Some people think
From about the 7th century C.E., the process of burial was firmly in the hands of the Church (the Christian organization), and burying the dead was only allowed on the lands near a church (now referring to the building), the so-called churchyard. The part of the churchyard used for burial was called
As the population of Europe started to grow, the capacity of graveyards was no longer sufficient (the population of modern Europe is almost 40 times higher than it was in the 7th century). By the end of the 18th century, the unsustainability of church burials became apparent, and completely new places for burying people, independent of graveyards, appeared—and these were called
The etymology of the two words is also quite intriguing. The origin of “graveyard” is rather obvious; it is a yard filled with graves. However, you might be surprised to hear that “grave” comes from Proto-Germanic *graban, meaning “to dig”, and it is related to “groove” but not to “gravel”.
Of course, the word “cemetery” did not appear out of the blue when graveyards started to burst at the seams. It comes from Old French cimetiere, which meant, well, graveyard. Nevertheless, the French word originally comes from Greek koimeterion, meaning “a sleeping place”. Isn't that poetic?