Bottle brush losing berries

This is So. Calif coastal.
I put in a bottle brush about 2 months ago. It started making "berries", but they all fell off before many of them even got red.
The nursery said it might be due to the unusual cold spell we had about that time.
Any input from gardeners in my area?
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
Add image file
Upload is a website for homeowners and building and maintenance pros. It is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.