Vermeil graduated in 1959 with an M.A. from San Jose State, where he was the backup quarterback. After serving as Head Coach for San Jose's Del Mar High School football team for several years, Vermeil was hired as the NFL's first ever Special Teams coach by the Los Angeles Rams in 1969. He would remain with the Rams until 1974 when he was named as Head Coach by UCLA. Vermeil compiled a 15-5-3 record in two seasons (1974-1975) as head coach at UCLA, including a 9-2-1 record in 1975 when Vermeil led the Bruins to their first Pac 8 Championship, and a win in the Rose Bowl over an undefeated and number 1 ranked Ohio State team.

Vermeil began his NFL head coaching career in 1976 with the Philadelphia Eagles. He would coach the Eagles through the 1982 season. He was named the NFL's coach of the year in 1980, and led the Eagles into Super Bowl XV in 1981. He retired for the first time after the 1982 season citing burnout. For the next 15 years, Vermeil spent time working as a sports announcer for the likes of CBS and ABC.

Vermeil returned to coaching with the St. Louis Rams in 1997 and led the Rams to their first ever Super Bowl victory in Super Bowl XXXIV later that season with a win over the Tennessee Titans. He was also named NFL Coach of the Year for his second time for the 1999 season. He walked away from coaching again after the Rams' Super Bowl victory.

Soon after retiring in 2000, Vermeil realized how much passion he still had for football, and decided to return to coaching yet again and coached the Kansas City Chiefs until he retired in 2005. Since his retirement, Vermeil has worked as a game analyst for college football and NFL games on NFL Network.