Robert's father, Henry "Light Horse Harry" Lee, was a Revolutionary War cavalry commander who had earned fame for his daring raids against British forces and his close friendship with George Washington. However, Harry Lee's post-war career was marked by financial speculation, failed business ventures, and mounting debts that would eventually force him to flee his creditors and abandon his family. The contrast between his father's military glory and civilian failures would leave a lasting impression on young Robert, instilling in him both pride in military service and a deep aversion to financial irresponsibility.
The burden of raising Robert and his siblings fell primarily on his mother, Ann Hill Carter Lee, whose own distinguished lineage connected the family to some of Virginia's wealthiest and most influential planter families. Ann Lee's character combined deep religious conviction with practical wisdom and emotional strength that enabled her to maintain the family's dignity despite their reduced circumstances. Her influence on Robert was profound and lasting, instilling in him the virtues of duty, honor, and self-sacrifice that would define his entire life. Her emphasis on Christian principles and moral conduct provided the foundation for his personal code of behavior that would guide him through the most difficult decisions of his career.