Almeric built his castle of wood above the harbour but it is evident from a deed that by 1235 a new castle had been built on the present site in the middle of the fertile land. This again would have been built in wood. The earliest extant parts of the present structure date from the mid-fifteenth century.
The house has been extensively altered by succeeding generations to adapt it to their times, most notably in 1738, when the house took on its current appearance and again in 1911 when Sir Edwin Lutyens renovated and added to the house. It is still possible to see evidence of the alterations that have been made and infer what was there before. This gives a remarkable insight into how historic houses evolved in Ireland over the centuries.