Smith attended Stafford Senior High School in Stafford County, Virginia, where he played basketball and football. He accepted a scholarship to the University of Maryland. In 2008, he set the Atlantic Coast Conference single-season kickoff return record with 1,089 yards after a 99-yard touchdown in the 2008 Humanitarian Bowl.
Torrey Smith was selected by the Baltimore Ravens in the second round of the 2011 NFL Draft. He was a member of the Ravens Super Bowl XLVII Championship team that defeated the San Francisco 49ers.
Torrey founded the non-profit Torrey Smith Foundation (TSF), whose mission is to provide support to at-risk youth with physical, educational and financial challenges, and to focus on the enhancement of lives of those that have been affected by domestic violence.