You need an ISO of Ubuntu having USB creator installed in it. If you don't want to burn a CD, have 2 USB keys, both formatted to Fat32, one with the setup described here:
Make a bootable USB stick by hand with Grub legacyand have the Ubuntu ISO you want to put on USB stick with persistent mode in the hard drive in the machine where you will boot.
If you have had a bootloader on the second USB stick the process might fail when the USB Creator will try to install a bootloader. In this case, reformat it to Fat32 again with Gparted.
Boot to the stick done by hand just before. Once you reach the desktop, insert the second key. Mount the hard drive's volume where you have the ISO. Start the Ubuntu Creator program from within the system menu. In the window, browse from there to the directory of the hard drive where the ISO is located. Then choose the second USB stick as destination. Choose the size for the persistent directory. Validate your choice, and go make yourself a tea or a coffee while the program works for you.
I tested today, rebooted to the newly created bootable USB, created a file, changed the configurations, shutdown, rebooted to another machine to check the changes had sticked : that worked.
