Sunday, November 26, 2017

boot - How to *actually* make a multi USB bootable disk in a reliable way. Without ruining its USB mass storage capability

I have tried countless bootable USB makers. When they work, they work pretty well.


Here's a list of tools I've tried.



  • YUMI

  • Unetbootin

  • Rufus


The problem is that they don't always work. The mode of failure isn't necessarily obvious. Perhaps the drive itself isn't compatible with the method used. Sometimes the drive isn't prepared properly for the tool ahead of time, and the tool doesn't even provide documentation that specifies what you need to do with the drive in order for it to work properly.


At any rate, I will attempt to describe the ideal situation, in increasing levels of system administrator user experience.


Mandatory: The USB drive shall be bootable in systems which support booting from USB. This should be the majority of computers. The USB drive shall NOT require a working operating system to exist on a working hard disk which is detected by the system, indeed the USB drive shall boot on a machine with no storage attached other than the USB drive in question.


Very much desired: The USB drive, after preparation, shall be mountable and writable and readable under Linux, OS X, and Windows, for use as an emergency mass storage device for storing < 2GB files as free space allows. Some directory in the root of the filesystem may be specified such that any files placed in there are LEFT ALONE by the preparator program as necessary.


Important: The USB drive, after preparation, can reliably be extended with a GRUB or similar menu to pick and load more ISO images. I want it to be EASY, not next to impossible, to make a 128GB bootable USB drive with a bunch of linux liveCDs, a bunch of Windows install ISOs, a bunch of diagnostic tool ISOs.


Extra credit: The USB drive is prepared once initially. It will be capable of dynamically scanning itself for ISOs loaded onto it via mass storage, and provide the GRUB picker or equivalent, dynamically, in which ISOs which were loaded onto the drive can be picked and booted. This means I can prepare the drive, then take it to other computers which have ISOs, load them on, and then take them to yet other computers to boot these new ISOs...


Is this too much to ask? Is this even possible, theoretically? Yes FAT32 won't let you put large ISO files onto the drive. That's not a good enough reason, though. This is goddamn 2016... the drive could just load something that can read exFAT to get at the ISOs stored on it...


These days I find that I am reformatting and rewriting images onto USB drives over and over and there is no reasonable way to maintain these things. Most of the tools that let you load multiple disks will unpack the image and drop files into directories, and eventually when booting files won't be in the right place or something and things fall apart. once this happens, the content on the disk is basically a lost cause.


So an alternative approach is to e.g. on a Mac hdutil convert -format UDRW and then dd to basically raw copy the iso to the USB drive. This has a very good chance of creating a solid USB drive, but now this USB drive is unreadable inside an OS and cannot be used as an emergency mass storage device. Neither can you ever have more than one image to boot this way, rendering the majority of the capacity of the drive useless.


For inspiration see the isostick. I actually have one of these. I haven't been using it much because it's freaking slow, and is USB 2.0 only. I have some USB 3 sticks I've been using but I'm really being fed up with loading images back onto my sticks over and over. At this point if I can have a solution that I can count on to work, I'll be happy waiting the few extra minutes eating my sandwich, rather than babysitting these AWFUL image loader tools. It's really not an ideal solution though, because 128GB USB3 drives with 130MB/s read speed are had for under 50 USD. Our software is really behind the curve at this point. Please just give me the speed and the flexibility.


I really hope someone who knows how this syslinux/grub stuff works can explain like I'm five why the hell I can't have this all done in software.

No comments:

Post a Comment

hard drive - Leaving bad sectors in unformatted partition?

Laptop was acting really weird, and copy and seek times were really slow, so I decided to scan the hard drive surface. I have a couple hundr...