Although Johnson was recruited by several top-ranked colleges such as Indiana and UCLA, he decided to play close to home.[ He initially wanted to attend the University of Michigan, but decided on Michigan State University in East Lansing, after the school's basketball coach, Jud Heathcote, promised that Johnson could play the point guard position.
Johnson did not initially aspire to play professionally, focusing instead on his communication studies major and on his desire to become a television commentator. Playing with future NBA draftees Greg Kelser, Jay Vincent and Mike Brkovich, Johnson averaged 17.0 points, 7.9 rebounds, and 7.4 assists per game as a freshman, and led the Spartans to a 25–5 record, the Big Ten Conference title, and a berth in the 1978 NCAA Tournament. The Spartans reached the Elite Eight, but lost narrowly to eventual national champion Kentucky.
During the 1978–79 season, Michigan State again qualified for the NCAA Tournament, where they advanced to the championship game and faced Indiana State University, which was led by senior Larry Bird. In what was the most-watched college basketball game ever, Michigan State defeated Indiana State 75–64, and Johnson was voted Most Outstanding Player of the Final Four. After two years in college, during which he averaged 17.1 points, 7.6 rebounds, and 7.9 assists per game, Johnson declared himself eligible for the 1979 NBA Draft.