The castle goes back to Archbishop Balduin von Trier, who began building this castle under the castle of Schaumburg in 1319. The town was granted city rights in 1321 by Emperor Ludwig the Bavaria. Balduinstein Castle belongs to the type of spur castles and has an oval floor plan. The outer walls, which are up to 1.60 meters thick, have been partially preserved. The foundation walls of the three-storey residential building are still standing. In the southwestern corner of the castle, the remains of a small tower have been preserved. On the north side, the castle was protected by a trench, which is still well preserved to this day, and on the northwest side by a deeper dwelling. A keep is missing, typical of the castles of the late Middle Ages.