Ventoy

Today I got introduced to a nice little utility called Ventoy (http://www.ventoy.net/en/download.html) that allow you to use the same USB stick as a bootable media for different OS ISOs. 

Here is where I heard this from and a little video showing you how it works.

VirtualBox USB and Ubuntu

I finally manage to get the USB working inside VirtualBox. I followed the instructions from this site https://help.ubuntu.com/community/VirtualBox/USB, restarted my machine, and like magics, my USB works inside my guest system now!