1. Run: pactl list short sinks
2. Note the device name you want to use as default
3. Try to run: pactl set-default-sink
This should work without giving you an error message.
4. Open the application "Startup Applications" (Should be preinstalled on Ubuntu)
5. Click on "Add"
6. Give your startup item a name
7. Copy your command from above into the command field:
pactl set-default-sink 'Your-Device-Name'
8. Click on "Add".