Weis launched his professional coaching career in 1990 when he was named offensive assistant and assistant special teams coach for the New York Giants. The Giants went on to win Super Bowl XXV at the end of that season, beating the Buffalo Bills by a score of 20–19. In 1991, Weis stayed on as the running back coach for two seasons, then began a four-year stint with the New England Patriots. For the first two years (1993–1994), Weis served as the tight end coach. In 1997, Weis became the offensive coordinator of the New York Jets in addition to duties as the team's primary receivers coach. In his second year as offensive coordinator, the Jets finished fourth in the National Football League in offense. Weis served as the team's offensive coordinator from 1997 to 1999.

Weis returned to New England Patriots as offensive coordinator from 2000 through 2004, installing the Erhardt-Perkins offensive system, and assisting the Patriots in three Super Bowl victories (XXXVI, XXXVIII, XXXIX).

In 2004, Notre Dame hired Weis, who was named Notre Dame's 28th head football coach on December 12, 2004. In his first season as head coach of the Fighting Irish, Weis took Notre Dame to the 2006 Fiesta Bowl in Tempe, Arizona, on January 2, 2006 which they lost to the Ohio State Buckeyes by a score of 34–20. The Irish finished ninth in the final AP Poll and eleventh in the Coaches Poll. His team's success on the field helped make Weis winner of the 2005 Eddie Robinson Coach of the Year Award. During the 2006 season, Weis led the Fighting Irish to a 10–2 regular season record, and a second straight BCS berth, this time losing 41–14 in the Sugar Bowl to the LSU Tigers. 

Weis was named offensive coordinator of the Kansas City Chiefs in 2010. In Weis's first year, the Chiefs improved from a record of 4–12 to 10–6 and returned to the playoffs after winning the AFC Western Division. With Weis as offensive coordinator, the Chiefs had the NFL's best rushing attack, averaging an impressive 165 yards per game on the ground. 

In 2011, Weis left the Chiefs to become the offensive coordinator for the University of Florida. On December 8, 2011, Weis was named the head football coach for the University of Kansas but was fired in 2014.