Charles Dickens, King George II, Admiral Lord Nelson and The Beatles have been through its doors.
The three paintings are part of a set of five Cowper portraits painted in 1862 by Frederic Leighton.
Historic England is also considering a Certificate of Immunity from listing for the Sunderland site.
Historic England awards £65,000 for a survey of the old building in the hopes of preserving it.
The £235,000 Heritage Fund has been awarded to the charity, who are based in Gloucester.
Campaigners hope England's last remaining disinfecting station in Clapton can be repurposed.
"Centuries of modifications have taken their toll" on Walmer Castle, English Heritage says.
The building at Underbarrow is now Grade II* listed and work is ongoing to research its history.
The city council says urban explorers and children have been trespassing at the site.
