tl;dr
SuperDuper! v3.10 B5 now available with minimal changes.
Medium Hanging Fruit
I discussed the speed of drives in the last post, not just to suggest that people use SSDs (which you should), but also to bring up the issue of "stalling": situations where it looks like SuperDuper! isn't doing anything, because accessing the drive is going slowly.
We've been thinking about this situation for some time. Coming up with a reasonable heuristic to detect what the user might consider "stalling" is a particular problem, as is doing something "sensible" when a stall is detected.
When this happens in a support case, I'll often provide a user with a few Terminal commands that interrogate the file system calls to show what we're doing, both to reassure the user that something is, indeed, happening...but also to provide a path that indicates what we're working on.
That process is clumsy and can be confusing, though. So, it was time to take a baby step towards a solution.
Minimal But Useful
To that end, rather than change the status view when a stall is detected (which, again, isn't easy to detect in a useful way), we've decided to add an "on-demand" bit of logging that I've named "WTFIH", in honor of "What's Going to Happen".
Basically, if the user opens the log window (thus asking the question "What's going on?"), we ask the copy engine what it's doing and add a line to the log that indicates where we're operating "right now".
We'll do this every time they open the log window, and that entry remains part of the log for the copy.
In addition, if the user ever asks the copy process to stop, we log the same info.
Information is Power
This new logging gives the user information about what's happening on demand and adds detail to the log that gives me more data without requiring them to run a Terminal command...and with that, assurance that things are operating as desired.
Every Little Thing
There are other small changes in this release, most of which you probably won't notice. As is our way, as we get close to an official release.
Download Away
And with that, it's time to
Download SuperDuper! v3.10 B5
As usual, this update will be offered to people who already have a previous Beta version installed. When the final release is made available, that will also be offered to Beta users, after which you'll only be offered normal updates until you manually install a future Beta.
So that's it! Let me know if you have any issues, and thanks for using SuperDuper!
tl;dr—Beta 4 of v3.10 is now available, and can be downloaded with the link at the bottom of this post, or via the auto-updater inside SuperDuper itself.
Progress Report
So, our last update was a big one, internally, and the results, so far, are quite promising, with nearly 100% boot success.
People who are having trouble are typically using slow destination devices. If they're too slow, not only does the boot process take a very long time, it can fail due to kernel watchdog timeout errors. (It's quite clear that these watchdogs are expecting fast storage operation.)
Quantification
Every so often, we run into a user who is seeing extended "stalls" when working with large folders of data. Inevitably, the folders have a lot of files in them, and they're using an HDD either for the source (for much older Macs) or the destination.
For example, let's take a some cases of a million or so objects. (This happens more than you might think.) We have three folders - one with just files, one with folders, and one with files with basic metadata (Safari attributes and Quarantine).
Unpacking a tar
archive with those folders to an SSD takes a few minutes. Unpacking the same archive to an HDD takes a few...hours (in fact, more than a few - we're talking over four).
Both freshly formatted, both fast USB-C devices.
Minutes vs. hours.
That's just creating the test case. Now, just to use pure, unadulterated, standard Mac commands that make use of the standard iteration calls, we've tried doing a timed ls
(list files) on those folders and counting the result:
SSD
time ls -1 dirWithAlotOfEntries | wc -l
1000000
real 0m11.152s
user 0m3.152s
sys 0m2.774s
time ls -1 dirWithAlotOfFolders | wc -l
1000000
real Om9.420s
user 0m2.713s
sys Om2.822s
time ls -1 dirWithAlot0fMetadataFiles | wo -l
1098579
real Om11.254s
user Om5.371s
sys 0m2.296s
HDD
time ls -1 dirWithAlotOfEntries | wc -l
1000000
real Om54.112s
user Om2.898s
sys Om3.473s
time ls -1 dirWithAlotOfFolders | wc -l
1000000
real 13m6.616s
user Om2.512s
sys Om4.594s
time ls -1 dirWithAlot0fMetadataFiles | wo -l
1098579
real 42m1.864s
user Om5.359s
sys Om11.220s
Interpret That For Me, Please
Just take the metadata example: to list the contents of the folder stored on the SSD, it takes about 11 seconds. With the HDD, it takes 42 minutes. And that's just to list files. We obviously have to do more than that.
Conclusion
At this point, I think it's wise to use an external SSD for your backups. They're affordable, reliable, and significantly faster. They're not flawless, and fail just like everything does, but you'll likely be a much happier camper with one.
Small Changes (Are Still Changes)
This new Beta is mostly a "cleanup" update.
We noticed during aspects of testing, in certain combinations, the "What's going to happen?" section of the main window didn't always give quite the right information in some edge cases (the sizes would be missing and stuff like that).
So, it's been extensively reworked internally, and all the various bits and pieces should appear in the expected way, in the expected order, and at the expected time.
Or so we expect.
Cough/Hack
Here at SPHQ, I've been a bit under the weather with my first (!) Covid-19 case (ah, I thought I was a lucky, lucky unicorn, but it seems not). Definitely out of the "woods", so to speak, with only general tiredness and an annoying cough remaining, but if support requests seem a little slower and/or loopier than normal, let's just chalk it up to that and not my senescence, eh?
Have At It
That's about it! We've got some plans for some other things for the next significant update, but for this beta, this should do.
As always, if you already have a Beta installed, this will be offered as an update. If not, and you download and install the linked version, it will update to the next beta (etc) until the "final release" of v3.10 is out.
At that point, once that final is installed, you'll be on the "regular" update unless you manually install a future Beta version.
Download v3.10 Beta 4