Near Elgin, was a motte-and-bailey castle and was in use from c.1140 to 1705.

During its occupation it underwent many alterations. The most fundamental was the destruction of the original wooden structure and its replacement with one of stone.

At the time of its establishment, it was one of the most secure fortifications in Scotland. At the death of the 2nd Lord Duffus in 1705, the castle had become totally unsuitable as a dwelling and so was abandoned.