You can set a meeting policy so anyone can join without the leader, see Automatically admit users at


It does create some concern, for example if you are providing toll-free access at your cost then people could use your conference ID to avoid call charges, but if you have audio conferencing on a per user basis with toll lines there isn't a commercial risk,