Blue porterweed getting too many ants!

Hi, I love in zone 10. My blue porterweed plant is attracting so many ants. I have sprayed it twice with insecticidal soap (made with neem oil) but th e ants just come back in a couple of hours. The plant is doing fine but I a m afraid the ant will damage the buds. Also some leaves on the plant are li ght in colour and are getting dry at the edges. What's going on with it?
Add pictures here
<% if( /^image/.test(type) ){ %>
<% } %>
Add image file
Upload is a website for homeowners and building and maintenance pros. It is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.