Hartley, 28, headed home from a 91st-minute corner to send the Pilgrims to Wembley for the first time in 20 years.
"I've never experienced anything like that, it's the biggest moment of my career so far," he told BBC Sport.
"I was sat in the dressing room and the boys were celebrating and I was just thinking 'what's happened?'"
Having drawn the first leg 2-2 at Fratton Park, the Pilgrims - who finished fifth in League two despite having led the league by five points in November - had almost all the chances in the second leg at Home Park.
But the game seemed to be heading for extra time until former Stevenage and Hartlepool defender Hartley came up with the winner in front of the Argyle end.
"I remember connecting with the ball and it hit the back of the boy's head and hit my head again and went in," he said. "Then I woke up and there was some big bald guy kissing me in the face.
"We set up perfectly to nullify Portsmouth's threat and I think we look like a real good outfit.
"I don't think Accrington or AFC Wimbledon will fancy playing us at Wembley."

The woman, aged in her 20s, was attacked after leaving the event's Chinawhite enclosure early on Sunday, with men she met there.
She walked with them to the Harpsden Road and Reading Road area and was raped outside a car, which the men then drove off in.
A passerby found the victim near the Three Horseshoes pub shortly after.
Daniel Grist, secretary of the regatta, said: "We have been liaising with Thames Valley Police, regarding this terrible incident, which took place after Regatta facilities had closed and racing had finished for the day on Saturday 1 July.
"The Chinawhite area is not located on land managed by Henley Royal Regatta but we will continue to assist Thames Valley Police and are working with them to secure CCTV footage to assist in identifying the perpetrators."
Thames Valley Police is appealing for witnesses, and specially-trained officers are supporting the woman, who left the Chinawhite enclosure between 00:00 and 02:30 BST.
Det Ch Insp Lis Knight said such incidents were rare in the area.
The Â£3m rowing regatta sees more than 200 races take place on the river Thames over five days.
The annual regatta, which has been held since 1839, was expected to attract 200,000 spectators to this year's event.

The material would "probably" be shipped back to the USA, said California-based SpaceX.
The section, measuring about 32ft (10m) by 13ft (4m), was spotted between Bryher and Tresco on 26 November and brought ashore by boatmen.
Company representatives have travelled to Tresco to dispose of the wreckage.
"It's about being a responsible steward of our rockets after they are used," a spokesman said.
SpaceX previously confirmed the wreckage was from the first stage of a successful mission to the International Space Station from September last year.
"We don't need it for testing or analysis, it is a pretty straightforward piece of carbon fibre material," said the spokesman.
"There are no electronics or anything. We are just trying to clean up after ourselves."
The Receiver of Wreck Alison Kentuck said the debris officially belonged to SpaceX.
Many islanders wanted the debris added to the Valhalla collection of figureheads from Scilly shipwrecks in Tresco Abbey Garden
A spokesman for the UK's National Space Centre in Leicester said it had been "desperate" to keep the wreckage.

The 51-year-old man, who has not been named, was arrested in North Yorkshire on Wednesday following a complaint made to Durham Constabulary.
The force said the man had been bailed pending further inquiries.
The vicar has been suspended for the duration of the investigation by the Bishop of Leeds, the Diocese of Leeds said.
It added in a statement: "Appropriate pastoral and safe guarding arrangements have been put in place.
"No further statement will be made at this time."
Peter Hartley says scoring the injury-time winner for Plymouth Argyle in the League Two play-off semi-final against Portsmouth is his greatest moment.
[NEXT_CONCEPT]
Organisers of the Henley Royal Regatta are supplying hours of CCTV footage to police after a woman was raped.
[NEXT_CONCEPT]
A section of a space rocket which was found off the Isles of Scilly, has been dismantled for disposal, its owners have confirmed.
[NEXT_CONCEPT]
A vicar arrested in connection with historical sexual abuse has been suspended by the Church of England.