Check out Michael Crump's latest blog walking you through connecting virtual machines through the Azure portal by using Azure Bastion.
Evaluate Azure Bastion. Deploy Azure Bastion to securely connect to Azure virtual machines directly within the Azure portal, to effectively replace an existing jumpbox solution. Monitor remote sessions by using diagnostic logs. Manage remote sessions by disconnecting a user session.