The exciting history of the Turmberg (tower hill) dates back to the 11th or 12th century. According to Durlach's historian Fecht, the castle and its surrounding buildings were built by the Hohenberg family that settled in the Pfinzgau area in the 12th century. In 1274, the castles and the towns of Mühlburg, Grecingen and Durlach were captured by Rudolf of Habsburg.
Initially, the castle remained undamaged, but was plundered and put to the torch in 1279 by Konrad III of Liechtenstein, Bishop of Strasbourg. Margrave Rudolf I and his successors rebuilt the castle partly using the remaining walls. The Gothic tower dates back to this period.

According to a map dated 1565, the tower was then merely referred to as the "Durlach observatory". Fecht relates in his history of the Turmberg that in the 15th and 16th century the castle tower served as a garrison for his town and as a watchtower. During this period the 13-metre-high buttress was erected parallel to the tower. This served as a bridge-like platform for cannons. In 1689, French forces looted and laid waste to the town of Durlach. At the same time, the tower was set ablaze, although the walls withstood the flames and the attempt to demolish it using explosives. Over the years, the observatory became less important. Fortunately, the Turmberg was rediscovered in the 19th century as an historic monument and lookout point.