In 1904 they tried having the games in St. Louis, Missouri but athletes did not want to make the trip by boat across the ocean so attendance was significantly down from the 1900 games even though the St. Louis games proudly used the words Summer Olympics in nearly all of their events. But the 1904 games are recognized as the first official Summer Olympic competition and it started Britain on their streak of winning at least one gold medal in every Summer Olympics since the games were officially recognized at the 1904 event. There was a great backlash in Europe about holding the Summer Olympics in America and in 1906 there was an alternate set of games held in Athens, Greece. But to this day the International Olympic Committee, known as the IOC, has refused to recognize any results from those 1906 games as official Olympic results. To the IOC the next Summer Olympics were held in 1908 in Britain.
The 1908 London Summer Olympics saw the first real controversy of the games when Italian Dorando Pietri was initially awarded the gold medal for winning the marathon. But the American team protested because Pietri did enter the stadium first but he was in obvious difficulty as umpires had to continually point him in the right direction and then he kept falling down. It got so bad that the officials helped him up and helped him cross the finish line. The protest was accepted and American John Hayes was given the gold because he was the first person to cross the finish line after Pietri a full 30 seconds after Pietri had crossed.
The Summer Olympics were held in 1912 and the events of 1908 helped spur interest and attendance soared but the onset of World War I cancelled the 1916 games. When the games started up again in 1920 it was again very popular and in 1936 the Summer Olympics were so popular that Hitler used them to try and show the strength of the Nazi ideal of Aryan supremacy. Hitler was embarrassed when his Aryans lost in almost every event and the Summer Olympics went on to become the premier sporting event of the summer.