This is because if they are guests of that team you conduct the meeting in they have access to the chat. In channel meetings the meeting chat is basically a conversation in that channel. If they join the meeting not as guests, they shouldn’t have access to channel conversations.

If this is an issue I would suggest setting up a private meeting instead of a channel meeting. Then everyone can access the chat and no one have to think about what account they connect to the meeting with.
The downside is that the chat will be a separate entity from the team itself