In the beginning of the novel Cold Mountain, a character recollect his childhood at a school house (maybe in the 1840s or 1850s in rural North Carolina) and describes flinging his hat out onto the playground from inside the building. Is this anachronistic? If not, what did 19th century playgrounds look like? When did they start to resemble the fort like structures with swings and such we know today?
Tyler Durden's comment does a great job with the first two parts of your question. This answer addresses when playgrounds began to look like the things we have today.
Short Answer: The modern American playground was championed by progressives in the 1880s-1890s; the most common playground equipment was all invented by the 1920s; and New Deal money made playgrounds ubiquitous in the 1930s.
Long Answer: The modern American playground was a pet project of progressives such as John Dewey and Teddy Roosevelt. The first public playground was opened in San Francisco in 1887. It included swings, slides, cart-rides pulled by goats, and a Roman Temple carousel. (source)
New York City owes its playgrounds to progressives' ability to organize and lobby. In 1898, the Outdoor Recreation League was founded, which went around placing slides and seesaws in parks near NYC slums. In 1902, a Reform party mayor had the Parks Department take responsibility for all of these parks; in 1903, Seward Park was opened as the first municipal park in the country to be equipped as a permanent playground. In 1906, a national playground advocacy group was founded (The Playground Association of America). (source)
In Chicago, progressive Jane Addams led the charge for playgrounds. Here's a picture of Chicago's Hull House playground in 1895. It's still mostly a wide open ground (but note play structures in the back and on the right). According to the Chicago Tribune, there were swings, rope ladders, and hammocks on the playground. (source)
Jane Addams was so dedicated to playgrounds that Hull House offered a two-year course on the topic of the playground. Carmelita Hinton, an alumna of this program, went on to invent and patent the jungle gym in 1920 together with her husband.
The New Deal provided the funds to make the playground ubiquitous. The infamous Robert Moses ("a product of the progressive playground movement") was appointed NYC's park commissioner in 1934:
In 1934, after seventy-five years, Central Park still had only a single playground, the Heckscher Playground in the southwest part of the park. In just three more years, it had twenty-two, including seventeen (twenty by 1941) "marginal playgrounds" dotted along the park's outer rim -- each equipped with slides, swings, jungle gyms, playhouses, and sandboxes and circled by benches for mothers and nurses.
By the end of his seven years, he had overseen a tripling of NYC's playgrounds, from 119 to 424. (source)
I'm sure there's something to be said for postwar suburbanization and the proliferation of "commercially available residential swing sets" (the first of which was apparently developed by a New England company in 1945). Maybe this is when we begin to see little fortresses like OP mentions. But for the most part, the modern playground had arrived by the 1930s.
PS: There are some great pictures of British kids sitting on wooden slides circa 1920 here.