Fort Story became a military installation in 1914 when the Virginia General Assembly gave the land to the U.S. Government "to erect fortifications and for other military purposes." The War Department named this land in Cape Henry Fort Story after Gen. John Patton Story, one of the most noted coast artillerymen of his day.

As World War I gained momentum in Europe, Fort Story was integrated into the Coast Defenses of Chesapeake Bay, which included Fort Monroe (Headquarters) and Fort Wool. June 9, 1925, Fort Story was designated a Harbor Defense Command. As the events of World War I entered the pages of history, Fort Story began a period of post-war inactivity which lasted until the beginning of World War II.

As World War II approached, Fort Story began extensive development. In 1941, the Headquarters of the Harbor Defense Command was moved from Fort Monroe to Fort Story. Two additional harbor defense installations were added to the network in 1941.

By September 1944, Fort Story began a transition from a heavily fortified coast artillery garrison to a convalescent hospital for returning veterans of World War II. At the time of its closing March 15, 1946, the hospital had accommodated more than 13,472 patients.

At the close of World War II, Fort Story again changed its mission. The first amphibious training at Fort Story began in 1946 with the arrival of the 458th Amphibious Truck Company and the famous Army DUKWS. Fort Story was officially transferred to the Transportation Training Command, Fort Eustis. It was designated a Transportation Corps installation for use in training amphibious and terminal units in the conduct of Logistics-Over-The-Shore operations.

Fort Story was declared a permanent installation on December 5, 1961. On July 1, 1962, it was redesignated a Class I sub-installation of Fort Eustis.