Shavuos became the time of the giving of Torah - When?

 There is no description in the Torah or Talmud that states Shavuous is celebrated as the time of the giving of the Torah. Nor could I find such a description in the Rambam

However Shulchan Aruch (OC 494)The fiftieth day of the count of the Omer is the holiday of Shavuot. The prayer service is like the holiday of Passover, rather we say "the Holiday of Shavuot, the time of the giving of our Torah". We say full Hallel. We take out two Torah scrolls and read in the first one of five [sections] from "The third month" until the end of the order. The Maftir is read in the second scroll, "On the day of the first fruits". As Haftorah we read the chariot of Ezekiel, and end with the verse "and the wind took me up."

The Magen Avraham asks that since everyone agrees that the Torah was given on the 51st day of the Omer - why do we celebrate Shavuous on the 5oth day?

Correction 

Pesachim (68b) Rabbi Elazar said: All agree with regard to Atzeret, the holiday of Shavuot, that we require that it be also “for you,” meaning that it is a mitzva to eat, drink, and rejoice on that day. What is the reason? It is the day on which the Torah was given, and one must celebrate the fact that the Torah was given to the Jewish people. 

However this contradicts two other gemoras as well as the Magen Avraham.