Controlled by your voice can also be a problem. So, if a burglar came to your front door and said “Hey Google” unlock the front door, would that door unlock? It is particularly important that for any device such as a thermostat, door lock, alarm system, etc. that they can only be controlled by your voice or the voices that you authorized. Another way would be to set up a secondary authentication like a security code that the smart controller would have to hear before activating. Most of these have the capability to learn voices and only respond to those voices.
Voice activated locks are an evolving industry, as they get smarter, they will become more secure and will be able to more closely communicate with your smart controller. You will also see (AI) artificial intelligence play a bigger role in household security.