Currently when I kill a drone agent manually, the autoscaler does not provision another agent. Is this a know behavior or autoscaler configuration error?
yes, this is the expected behavior. If you want to manually terminate an instance you should do so with the drone server destroy command so that the autoscaler is aware.
Also, we would like the drone agents to automatically update security patches. Since we are using Amazon Linux 2 instances for the agents, all we need to do is decommission older agents and the autoscaler provision new ones
The autoscaler tries to remove older instances first, which means you should always have newer (more up to date) instances replacing the older instances. I would be surprised if you ever had instances that were more than a few days old, although this does depend on usage patterns.
You could also setup a simple cron job that removes instances based on creation date, to ensure they are replaced with newer instances. The autoscaler exposes a full API that should help with this. I would not recommend scheduling such a job during peak hours.