All of these assumptions assume that Humans just up and died. a gradual decline or massive war, or impact event could easily change some of these results.
While the large number of extinctions have already been covered in the other answers. its worth noting that humans bones would survive in certain places but definitely not all. the majority of climates in the world will be unlikely to provide reliable sources of human remains. however some places will, areas with very low humidity is a very good start. and those remains may have some form of prosthetic or obviously refined additions. Certain types of fillings will still be evident. along with metal plates holding bones together etc.
The problem is will they really survive... for most of the world no. the reason: it is becoming quite common to cremate our dead. far more common than it has been for a 100 years or so. so there will be evidence, but alas it will be hard to find
Even "Non-Biodegradable" plastics, degrade over time, and 40,000 years is long enough for this to happen, they won't be gone completely but they would only appear as particulates in a sedimentary layer in Core samples, so anything made of plastic will most likely be gone. metal structures around the world would have collapsed and while it would be evident that some of this structures existed at some point the materials would be largely unclear at first sight (yes in depth analysis would show they were RSJs etc). but this wouldn't be as important as Steel. Steel does not occur naturally, it an Alloy, and the Modern Human World is basically made out of Steel. from buildings to car parts etc. this would be the first clue as to at least an industrial level society had lived here.
The Pyramids would still exists, yes it would continue to crumble, and most likely appear to be massive sand dunes, but once excavation happened on these big mounds they'd find the stone work to be definitively refined and cut, even to this day thousands of years after being built after the limestone has eroded from the faces and the stone beneath has begun to crumble. the stones that line the internal chambers are still in almost perfect condition.
Gold, nowhere in the world does gold occur naturally in such large and pure amounts than in National Reserve Stores. the Bank of England, US Federal Reserve etc all have large amounts of Gold stored away. if the future society found these, they'd be 100% certain that this was some form of financial store. even if everything surrounding the Gold had collapsed and turned to mud (not literally) the Gold would still be perfect evidence.
Any material in significantly higher purity than occurs naturally would point to a refining process. then each of those materials would then show what level that society was at: little Steel, little Gold, but high concentrations of Brass and Iron, pre-medieval times, Steel, Gold (not super high purity though) Brass Copper etc. points to medieval. Steel and gold, and copper etc, industrial. and then Palladium super pure Aluminium, pure gold etc. Modern age.
if we assume the future society uses anything similar to Carbon Dating, then they will quickly discover that there was a huge amount of radioactive material in the atmosphere at one point. Carbon/lead dating easily show these effects from the atomic testing in the 40s and 50s to the Chernobyl Disaster and presumably Fukushima as well. and again certain materials point toward nuclear tests. Cesium isotopes for example are evident in lake beds world wide after nuclear tests and these show up in Lead Dating techniques
While earths Satelites will have all re-entered in 40,000 years, the third stage of Apollo 12, is in incredible high orbit of earth, it didn't reach its intended trajectory and got flung out into solar orbit and then re-entered Earths Sphere of influence, at this time it is unclear if this will enter Earths Atmosphere but its possible it'll be out there for good.
The Lunar landing sites are have test equipment and parts that will degrade over time due to solar radiation, but most of it will take so long that they will survive 40,000 years
Voyager 2 is on its way out of the solar system, it was nothing really to worry about, it will probably be pretty banged up in 40,000 years, but it will still be flying out into space for someone to find.