In the late winter of 1863, both armies had settled down to defending their positions along the Rappahannock River in Virginia. Mostly the respective armies stationed units at the crossing fords, like Kelly’s Ford. On March 17, 1863, cavalry forces from both armies clashed at Kelly’s Ford in Culpeper County.
The Confederate cavalry had been harassing Union forces all winter. The Union command determined to respond and ordered Brig. Gen. William W. Averell‘s Union cavalry division to attack across Kelly’s Ford. Averell led 2,100 troopers, divided into three brigades plus accompanying artillery.
Facing them was Brig. Gen. Fitzhugh Lee with a brigade of about 800 men. Lee was the nephew of Robert E. Lee and was in Maj. Gen. J.E.B. Stuart’s command. The two men who were close friends from their West Point days had clashed on February 25th when Lee led a force of about 400 troopers in a raid near Hartwood Church in Stafford County, 9 miles northwest of Fredericksburg.
During this engagement about 150 troopers from Averell’s command were captured. Maj. Gen. Joseph Hooker, now the commander of the Army of the Potomac was furious and threatened to remove Maj. Gen. George Stoneman as commander of the Cavalry Corps. Stoneman reacted by ordering Averell to respond to the raids.
Meanwhile, Lee was sending his old classmate taunting messages across the Rappahannock. One such message read, “I wish you would put up your sword, leave my state, and go home. You ride a good horse, I ride a better. If you won’t go home, return my visit, and bring me a sack of coffee.”
When Union scouts came to Averell with the location of Lee’s force, he ordered his cavalry division to set off for Kelly’s Ford. He started with 3,000 troopers and 6 artillery pieces but after detaching men to guard his line of retreat, he arrived at the ford with 2,100. His three brigades were commanded by Col. Alfred N. Duffié, Col. John B. McIntosh, and Capt. Marcus Reno. The opposing Confederates were divided into 5 regiments and
When Averell’s advance forces arrived at Kelly’s Ford, they found the crossing fortified with felled trees and guarded by 60 Confederates. They made three attempts to force the crossing but all were unsuccessful. Finally, Averell’s chief of staff, Major Samuel E. Chamberlain, forced a crossing led by 20 men of the 1st Rhode Island Cavalry. Chamberlain was wounded in the head. Despite the minor casualties in this action, Averell proceeded cautiously, taking over two hours to cross his men over the swiftly running river.
Meanwhile, Lee had been alerted to the battle at the ford at about 7:30 AM. Assuming that it was a thrust against Brandy Station on the Orange and Alexandria Railroad, he ordered his force to block the Union advance. About 2 miles northwest of Kelly’s Ford, they ran into Averell’s deployed troopers. Duffié’s brigade was positioned on the left in a woodlot, McIntosh’s in the center, and Reno’s two regiments of regulars on the right, behind a stone fence.
Jeb Stuart who was at Culpeper Court House attending a court-martial decided to ride out and observe the battle. He was accompanied by his young artillery chief, Maj. John Pelham. When they arrived at the battlefield, Stuart realized that Lee’s force was not doing well. In fact, the 2nd Virginia cavalry had fled in the face of the enemy. The first time that this had happened in the war.
Lee ordered his five regiments to advance in a line abreast. As Pelham moved forward with Lee’s men he was hit and mortally wounded by a piece of shrapnel to the brain. He died several hours later. Duffié disobeyed Averell’s orders and ordered his men to charge, surprising the Confederates and forcing them back. However, he was unsupported by Reno who obeyed orders and was forced to withdraw.
After an all-day battle, Averell ordered his men to withdraw at about 5:30 PM. He left two Confederate officers who had been wounded and captured by Averell’s troops, a sack of coffee, and the following message: “Dear Fitz, Here’s your coffee. Here’s your visit. How do you like it?”
The Union advance had covered 2 miles over more than 12 hours and resulted in 78 casualties (6 killed, 50 wounded, 22 missing). The Confederates lost 133 (11 dead, 88 wounded, 34 captured); 71 Confederate horses were killed and 12 were captured.
For Stuart and the Army of Northern Virginia, the loss of Pelham, age 24, was devastating. Stuart wrote after the battle,“The gallant Pelham—so noble, so true—will be mourned by the nation.” Pelham’s death is commemorated to this day with a historical marker near the site of his death. It is the only such marker for the death of an individual soldier that I have ever seen in my travels around Virginia.
William Averell later went on to lead cavalry forces in Virginia and West Virginia until he was relieved of command after a dispute with Maj. Gen. Philip Sheridan after the Battle of Fisher’s Hill in 1864. In civilian life, he became well-known as the inventor of American asphalt pavement.
Fitzhugh Lee continued to lead cavalry forces throughout the war. He was the last commander of the cavalry forces of the Army of Northern Virginia but was among the troops surrendered by his uncle at Appomattox. In 1896, Lee was elected the 40th Governor of the Commonwealth of Virginia, a position that he held for 4 years.
Lee had the distinction of serving in the United States Army, the Confederate Army and again in the United States Army. He was one of three ex-Confederate general officers who were made major generals of United States Volunteers during the Spanish-American War.