Rosser was commissioned a first lieutenant and became an instructor to the famed "Washington Artillery" of New Orleans. He commanded its Second Company at the First Battle of Manassas in July 1861.
He was noted for shooting down one of George B. McClellan’s observation balloons, a feat that won him promotion to captain. He commanded his battery during the Seven Days Battles of the Peninsula Campaign, and was severely wounded at Mechanicsville. Rosser was promoted to lieutenant colonel of artillery, and a few days later to colonel of the 5th Virginia Cavalry.
He commanded the advance of J.E.B. Stuart’s expedition to Catlett’s Station, and was notable in the Second Battle of Bull Run, where captured Union commander John Pope’s orderly and horses. During the fighting at Crampton’s Gap at the Battle of South Mountain, his cavalry delayed the advance of William B. Franklin’s VI Corps with help from John Pelham’s artillery. At Antietam, his men screened Robert E. Lee’s left flank. He temporarily assumed command of Fitzhugh Lee’s brigade during the subsequent fighting against Alfred Pleasonton.
He was again badly wounded at the Battle of Kelly’s Ford, where “the gallant” Pelham was killed. Rosser was disabled until the Gettysburg Campaign, where he commanded his regiment in the fighting at Hanover and the East Cavalry Field at Gettysburg. He was promoted to brigadier general of the “Laurel Brigade,” which had gained fame under Turner Ashby. During one of his October–November West Virginia raids near Chancellorsville, Virginia, in November, Rosser seized a Federal wagon train containing much of the ammunition reserve of the I Corps and V Corps of the Army of the Potomac. He was distinguished again in the 1864 Overland Campaign, driving back a large force of Union cavalry and artillery at the Battle of the Wilderness.
Rosser was yet again wounded at Trevilian Station, where his brigade captured a number of prisoners from former West Point classmate and close personal friend George Armstrong Custer. The Federal rout at Trevilian Station became known to the Confederate forces as the “Buckland Races.” His brigade later gallantly fought against Philip Sheridan in the Shenandoah Valley, and he efficiently commanded Fitzhugh Lee’s division at Cedar Creek. A rare defeat where Custer overran Rosser’s troops at the Battle of Tom’s Brook allowed Custer to repay Rosser for Trevilian Station. For no tactical reason, Custer chased Rosser’s troops for over 10 miles and the action became known as the “Woodstock Races” in Union accounts. Custer had also captured Rosser’s private wardrobe wagon at Tom’s Brook and Rosser immediately messaged him.
Douthern press as the “Saviour of the Valley,” and was promoted to major general in November 1864. He conducted a successful raid on New Creek, West Virginia, taking hundreds of prisoners and seizing much need quantities of supplies. In January 1865, he took 300 men, crossed the mountains in deep snow and bitter cold, and surprised and captured two infantry regiments in their works at Beverly, West Virginia, taking 580 prisoners.
Rosser commanded a cavalry division during the Siege of Petersburg in the spring, fighting near Five Forks. It was here that Rosser hosted the “infamous” shad bake (fish feast) 2 miles (3.2 km) north of the battle lines preceding and during the primary Federal assault. Guests at this small affair included George Pickett and Fitzhugh Lee. Shelby Foote states that “Pickett only made it back to his division after over half his troops had been shot or captured..”. It is said that Lee never forgave Pickett for his absence from his post when the Federals broke the Confederate lines and carried the day at Five Forks.
Rosser was conspicuous during the Appomattox Campaign, capturing a Union general and rescuing a wagon train near Farmville. He led a daring early morning charge at Appomattox Court House on April 9, 1865, and escaped with his command as Lee surrendered the bulk of the Army of Northern Virginia. Under orders from the secretary of war, he began reorganizing the scattered remnants of Lee’s army in a vain attempt to join Joseph E. Johnston’s army in North Carolina. However, he surrendered at Staunton, Virginia, on May 4 and was paroled shortly afterwards.