Researchers discovered a small proportion of men - one in 20 - who exhausted all other options, were alive and well, even after a clinical trial ended.
The study, published in the Journal of Clinical Oncology, said the most dramatic responses were seen in patients whose tumours had mutations in genes involved in repairing DNA.
Scientists are now looking at whether this group might especially benefit from immunotherapy, which is a type of treatment that uses the body's immune system to fight cancer by recognising and attacking cancer cells.
Johann de Bono, Regius Professor of cancer research at The Institute of Cancer Research, London, said: "We don't see much activity from the immune system in prostate tumours, so many oncologists thought immunotherapy wouldn't work for this cancer type.
Around 5% of the 258 men involved in the report saw their tumours shrink or disappear, while 19% showed some evidence of tumour response.
The average length of survival among a group of 166 patients with high levels of prostate-specific antigen (PSA) was just over eight months with pembrolizumab - with nine seeing their disease vanish or partly disappear on scans.
Four were super responders who remained on treatment at the end of the study follow-up, with reactions lasting for at least 22 months.
A second group who had lower levels of PSA but whose disease had spread to the bone lived for an average of 14.1 months.
Professor Paul Workman, chief executive of The Institute of Cancer Research, London, said: "It's encouraging to see testing for DNA repair mutations may identify some patients who are more likely to respond, and I'm keen to see how the new, larger trial in this group of patients plays out."