Since the early times caves are used as a place of worship. The oldest examples are known from the Stone Age, were caves were used for paintings, sketches and sculpures, many of them with a cultic background. Unfortunately we know very little about this, as we only have the artworks as remains of those people. All attempts to explain the intentions are just fantasy. And those caves are natural, so there are other categories in the archaeology section.
The cave churches we talk about on this page, are subterranea, which means artificial cavities, made by man. Often, when man had no natural caves for worship, he tried to build artificial caves replacing them. Sometimes it was an overhanging rock or a small cave which was widened until it fit the needs, sometimes it was a soft layer of rock which was used to dug out the cave for the church.
The name cave church is not restricted to Christian churches, actually we include all underground spaces created for any kind of worship. There are caves of the Mithras cult, hellfire clubs of the 19th century and catacombs used by the Christians when they were persecuted in second and third century Rome. A better term is actually cave temple.