Shirt Pocket

You Got Culpa on My Mea! Saturday, January 21, 2023

One of the problems with supporting versions of macOS going back to 10.10 is that it becomes harder and harder to test older versions...and that's complicated further by Apple silicon, since you can't run an Intel VM on Apple silicon...Rosetta won't work.

Unfortunately, in v3.7.3 and v3.7.4, this has caused a problem. It's something we didn't anticipate, and, alas, it caused some minor attribute issues on older OS versions that are fixed in v3.7.5.

These problems will self-correct the next Smart Update, and never put data at risk, but they're embarassing anyway.

What Happened

During the last few months, we'd had a report that the "Date Added" attribute of files wasn't getting updated. We didn't remember exactly why that attribute wasn't copied, and when we checked it under current OS versions it seemed like it could be copied, so we implemented that in our copy engine and distributed that to some external testers.

During that process, we found a way to copy attributes that allowed us to eliminate a read operation. Basically, the fts API has a field that was populated with things we were reading separately, and we changed the copy engine to use those instead.

External testing showed why we didn't copy "Date Added" in the first place: setting it is not supported by some file systems and some versions of macOS. So, before shipping v3.7.3, we backed out that change (with a typo that caused the relase of v3.7.4), but we left in the optimization.

Unfortunately, post-v3.7.4, we received a few reports of folders that had become inacessible without elevating permissions. On investigation, this was due to the optimization: not all versions of macOS populate that field properly, and that was causing the problem.

The Solution

The solution to this was to back out the optimization, which we've done in v3.7.5, released today. Any incorrect attributes will be automatically updated next Smart Update.

Things to Improve

We wanted to turn around v3.7.4 quickly once we found the problem, which we did, but since we were backing out a change (rather than implementing a fix), we didn't put it through a full external test.

That was a mistake, especially since it involved the copy engine. And while it didn't cause any harm as such (save for embarassment), it's something we'll endeavor to not do again.

The Future

This may mean we will have to phase out "new" support of older macOS versions in future releases: beyond the testing problems, it has become hard to even set the "target" version of SuperDuper builds to 10.10.

That wouldn't mean we won't "support" older macOS versions (after all, we offer versions of SuperDuper that work with macOS all the way back to the Power PC days), but it would mean that new versions of SuperDuper may not support as many versions "back" as we'd like.

Have at It

SuperDuper v3.7.5 is available now for auto-upgrade and as a download. Thanks, as always, for your support, reports and for using SuperDuper: we appreciate it.

Wait, Hold On! Thursday, January 12, 2023

You know that thing where you put out a release, that fixes two minor crashes, but then a typo causes a different failure?

Well, welcome to SuperDuper! v3.7.3/v3.7.4, the "Can I Have a Do-Over" release.

v3.7.3 was released to correct two rare problems: both of which had to do with launching scheduled copies.

The first was a weird race condition that caused occasional crashes right at launch, but only with scheduled/scripted copies launched without loading settings. It was strange, we couldn't reproduce it internally, but it happened to a few users and we think we've run it down.

The second caused copies to terminate successfully, typically after copying all the data, before running the cleanup phase of the copy operation. This was another one we couldn't reproduce internally, but seems to be related to a singleton that was getting re-initialized in some situations. And if that doesn't mean anything to you, don't worry about it, beacuse it's fixed and all is right with the world.

Of course, all was right with the world, until we released v3.7.3 this morning, and wihin an hour a user got an error.

Really? Oh, Man...

We thought we were in pretty ideal shape. v3.7.3 had been given to quite a few people over the last month or two, and it was working great both internally and in external testing. But, right after release, a user had a copy error on Catalina, and it was quite weird so we ran it down.

During development, we were doing some investigation into copying of some uncopyable attributes, and at one point we found an optimization that allowed us to skip a read operation, since we already had the data.

But, when implmented, there was an expansion-related mistake in one access to the variable, which caused an error in some rare circumstances.

Although, again, we couldn't reproduce the problem internally (I hate that, since it makes it hard to generate a test case to guard against this kind of thing in the future), we nevertheless ran that to ground quickly after v3.7.3's release (ah, v3.7.3, we never really knew ya), and out the door went v3.7.4.

So there you go! Two quick releases, two small fixes, one panicked change of underwear, and one other fix. A good way to spend a Thursday.

Enjoy!

Analyzed, Statically Saturday, December 17, 2022

Well, v3.7.1 didn't last very long, did it. >sigh<

Happy to announce SuperDuper! v3.7.2, the "give me a fix, one fix only" release.

Ready for Launch

Here's the deal: when Applescript starts SuperDuper with a specific Apple Event ("launch"), SuperDuper starts and doesn't load the default settings. Instead, it waits for the application scripting it to load the settings it wants, and proceeds from there.

Late in the development of v3.7.1, we ran SuperDuper! through the Xcode static analyzer, and it mentioned that one specific thing in our launch sequence could hang. So, it was moved off the main thread to a background thread.

Initial tests showed things worked, and the analyzer was happy, so we went with it.

If a Bug Happens in the Forest...

Well...I noticed that I received a weird error panel during a scheduled copy that showed that a volume wasn't mounted (which was true, but didn't matter). The copy happened fine, and it was kind of harmless, but it was weird.

That, though, was the result of the moved code...the launch event was loading settings it shouldn't, and if the drives for those settings were missing, you'd get an error.

Now, no users reported the problem, but we found it, fixed it, and that's v3.7.2.

Code is Served

That's about it! I'd love to say "100% Faster!" or "Tastes Great, Less Filling!" but really it's just "a bug was fixed".

And now SuperDuper! v3.7.2 is available for automatic update. Enjoy your weekend!

Ventura Highway Friday, December 16, 2022

I'm happy to announce the immediate availability of SuperDuper! v3.7.1, an update that will go down in history as being released today. >fanfare<

So, what's different?

As is tradition around these parts, after the release of a major OS, and our corresponding major update to support it, there's always something that can be improved. Even with extensive private and public testing, there are just so many different Macs, software configurations, peripherals, drives, drive firmware versions, docks, raw drive units...something always pops up.

And so it has. In Ventura, on some systems, we've seen some cases where, post-replication ("Erase, then copy" in Big Sur and later), the destination volumes wouldn't always re-mount. Sometimes an error would occur (referencing the 'bsd' info), sometimes not. When these failures occur, Apple's replicator has also replicated the source volume name, and due to the error, we didn't get a chance to rename it back to what's expected.

Anyway, it was annoying to you and (because we hate things like this) us. So we've been working for the last month or so to try to find a way to fix this...and I'm happy to say we have.

But the excitement doesn't stop there.

What's in a name?

A recent Ventura release has also started notifying people about "Startup Items" that were installed. Of course, as you might expect, when you set up a schedule, we have to install some items to get those schedules to, you know, run and stuff.

So, Ventura starts telling you that it happened. But - rather than use the Application name, it used the "Development Team Name"...and so people who see that "Bruce Lacey" had added some startup items...confusing everyone. (Bruce, as you'll see in the About box, is my partner in this endeavor).

Of course, this confused people, they turned the startup items off, and then schedules didn't work.

We've found the key needed to get the startup items to say something more sensible, and so now they'll say "SuperDuper!"—please don't turn them off! If you do, your schedules will not work.

No Emulation Needed

Finally, we've found a way to get the updater to not demand the use of Rosetta: something entirely unnecessary, given the SuperDuper! application works natively on Apple silicon. This'll make the update seamless for users on Apple silicon without Rosetta installed.

Whee?

Yeah, not terribly exciting, I know. But, important behind-the-scenes improvements that should make things better for everyone.

As always, thanks for being a SuperDuper! user, and enjoy the new update!

Silly Season Monday, October 17, 2022

Welcome back to Silly Season, where we all wait to see what the Apple gods have wrought with the new release of the OS that'll come this month.

As most of you already know, since Big Sur, programs that want to copy the OS must rely upon Apple's replicator ('asr') to perform the task. There's no choice: this particular copying ball, and the ability to boot from the result, are entirely in Apple's court.

And at least so far, we're seeing a larger percentage of failures both in replication (typically error 49244, which when decoded and byte stripped is error 92. That means there's non-fatal -- except to the replicator -- corruption) and in boot.

These are rather frustrating for us and, of course, for you. For us, because apart from reporting the issues to Apple, which we have done, we can do nothing about them. Apple has to fix the problems; we cannot work around them.

For you, because until those problems are fixed by Apple, you won't be able to boot from a backup.

Still Works!

That said, your Smart Updates are going to work fine, and remain fully restorable. As explained previously, you need only reinstall the OS and point at the backup when prompted to restore during first boot. This all works great...and when Apple fixes the issues with the replicator and with boot, that will "magically" start working again too.

Meanwhile...

In the meantime, we've been improving SuperDuper. We've worked around some issues with Google Drive (which was incorrectly protecting folders on drives it's not operating on), significantly improved some corner-cases in Smart Update, and made changes needed to work well under Ventura.

Dinner is Served

We don't expect any major problems with this new release, and it has been working well in limited field release. Given that, it's time to roll out a wide public beta, and so here we are.

As always, to use the Beta, download and install the version linked in this post.

Once you're running a Beta release, any subsequent Betas in this series will be offered to you as updates, as will the final, production release.

Once the production release is installed, no further Beta releases will be offered automatically until you opt-in by installing a future Beta.

Enjoy, and get in touch if you have any problems!

Download SuperDuper! 3.7 B1

The “Roll Up Some Fixes” Release! Sunday, March 13, 2022

Things have been relatively quiet here at the blog, but not because I've just been enjoying listening to records.

Rather, we've been working on a series of fixes that we're releasing today as SuperDuper v3.6.

This release is especially important for Apple silicon users. macOS 12.3 will break SuperDuper! v3.5 due to problems with asr (Apple Software Restore), the tool that must be used to copy the OS, and we've worked around that issue in our new update.

I'm Sure I've Used #blessed Before

On Big Sur and later, as I've written before, asr must be used to produce a bootable copy. We can update everything but the OS with Smart Update, but if the OS needs to be copied, we need to use asr to do it.

As part of its operation, asr blesses the copy. That basically means it does the final steps necessary to set the drive up for boot.

Under Big Sur and later, however, that's not sufficient: the copy has to be authorized for boot, which is done either with the Startup Disk Preference Pane (if it appears there, which it sometimes doesn't) or via the "boot choice" menu at start (which is accessed during startup via Option on Intel Macs, or by holding down Power on Apple Silicon Macs).

On macOS 12.3, a change in bless breaks asr, and it returns an error, which causes SuperDuper! v3.5 to fail the copy. (At this point, everything's been copied, and no data will be lost, but a scary red bar appears, often followed by a change of underwear, a support email, and a request for dry cleaning reimbursement.)

We've changed SuperDuper! to work around this problem, and as long as you've updated to v3.6, things should work fine.

Better, Even!

But that's not all. We've also improved performance, especially when cleaning up folders with fewer files in them during Smart Update; updated our handling of "Cloud" files on Monterey; fixed a crash with new-style serial numbers on macOS versions before 10.14; worked around some security prompts when running from non-Admin accounts...a list of all the updates is in the release notes, as usual.

Status Report

As I'm sure you've noticed, the status bar in an active phase is, well, blue-on-blue. Over the years, Apple has changed both the size and color of their progress bar control so that it's become closer and closer to our color.

When the window is active, that means there's minimal contrast. When inactive, though, it's quite visible...which I often joke is a subtle hint to not watch the pot boil.

We didn't want to create our own control here, since we'd constantly be chasing Apple's visual updates. In the past we've tried applying a hue filter to the control, but that caused some older Macs to crash, and since we support the OS a long way back, and a lot of old Macs, that proved to be unworkable.

And yes, we could change our bar color, but since Apple keeps changing their color, that isn't a good choice either.

All is not lost, though. You can change the color to suit your taste: in System Preferences's General section, the Highlight or Accent selection changes the control color. So, if it bothers you, there you go—go crazy and choose a bar color of your own!

Wake Up!

As I wrote before, if you get a "Resource Busy" error during an Erase, then copy error under Big Sur and later, it's likely your Mac fell asleep during the copy, even though we asked it to stay awake (and even if it looks like it didn't). You can usually fix this by installing Coca from the App Store (it's free!) and using it to keep the Mac awake during this process. Ensure you "Activate" it using its menu extra.

Wrap Up

We've been quite happy with the way SuperDuper! v3.5 has been working in the field, with lots of good feedback, and this version should take care of the few issues we've found.

Thanks to everyone who's using it, and who has written in. Until next time, back to the code mines (which I think is a Pretenders song)!

It’s a Trip to the Moon, Not a Marathon Monday, October 25, 2021

tl;dr

At long last, v3.5 is out, with

  • Big Sur and Monterey Bootable Backups
  • Support for macOS 10.10 and later
  • Apple silicon support
  • Various fixes and improvements (see the Revision History in the Help menu)


Under Big Sur and later, the OS is only copied if you use Erase, then copy. Everything else is updated when you use Smart Update later, of course. The What's going to happen? section will tell you what's going on.

Users on Apple silicon systems with older (pre-2019 or so) licenses will need to either purchase a new license or run under Rosetta.

Auto-update now available, or

Download SuperDuper! v3.5

Long, Strange Trip

If you're interested, the history of this version is documented in detail in earlier blog posts over the last year or so. It's taken longer than we'd hoped, but through the various Beta releases we've got a lot of testing miles under our belts, and we're pretty pleased with the way it came out.

There remain some challenges, of course. Bootable backups under macOS 11 (Big Sur) and later are a bit strange, because they don't show up immediately in the Startup Disk Preference Pane. You'll find they do show up in either Option+Boot (Intel) or Power+Boot (Apple silicon) boot menus, though. And once authorized, they boot nicely.

General Process

So, what's the process for making an initial bootable backup under Big Sur and later?

  1. First, the destination has to be formatted as APFS. To do a nice, full format, you should:
    • Open Disk Utility
    • Choose "Show All Devices" from the View menu
    • Select the destination drive hardware (above the existing volume)
    • Click Erase
    • Choose the "GUID" partition scheme (2nd pop-up), THEN "APFS" formatting (1st pop-up) and name appropriately
    • Click Erase
  2. Then, make an initial Erase, then copy backup. Erase, then copy is selected in Options under During copy.
  3. That's it!
  4. No, really, that's it.

And Then What?

To maintain your backup, use "Backup - all files" with Smart Update (again, in Options under During copy). That will update all "your stuff" (that is, everything except the operating system), and things should still boot normally even if the OS is updated.

When you want to update the version of macOS on the backup, you can make an Erase, then copy backup again, as above.

Does My Backup Have to be Bootable?

It does not. If you start with a fresh drive or APFS volume, as above, a "Backup - all files" with Smart Update will only copy "your stuff" (that is, the macOS "Data" volume).

This can still be restored via migration after a clean OS install, of course...nothing is missing save for the OS, and you can download that any time.

My Backup Isn't in Startup Disk Preferences!

Often the backup won't show up in the Startup Disk Preference pane. That's OK! As mentioned in a previous post, you should use Option+Boot (on an Intel Mac) or Power+Boot (on an Apple silicon Mac), and it should show up there.

What About Encryption?

A bootable backup cannot start out encrypted, due to macOS rules about creating volume groups. So, after you create a bootable backup, start up from it, turn on FileVault (you don't have to wait for it to finish), then boot back to your regular drive.

Ensure the password is added to the keychain when prompted, and that's all there is to it. Subsequent Smart Updates will maintain the encryption.

Regulations Require My Data To Always Be Encrypted

No problem. While you can't have a bootable backup without the brief gap between backing up and encrypting, a non-bootable backup, as above, can start with a volume formatted encrypted. Just use "Backup - all files" with Smart Update.

Anything Else I Need to Know?

If you get a "Resource Busy" error during an Erase, then copy error under Big Sur and later, it's likely your Mac fell asleep during the copy, even though we asked it to stay awake. You can usually fix this by installing Coca from the App Store (it's free!) and using it to keep the Mac awake during this process.

You may notice that, if an Erase, then copy backup fails under Big Sur and later, the backup drive has been renamed to the name of the source drive.

This is because those backups are done with asr, a system tool that does a low-level replication of the drive. Along with the data comes the drive name. After a successful copy, we rename the drive to its original name, but if asr fails, the drive never gets renamed. Fully erasing, as above, will fix it.

Nap Time

So, that's about it, save for whatever I've neglected to mention! Thanks for your patience as we worked through this release - your support and encouragement is sincerely appreciated. Not much left, except to

Download SuperDuper v3.5

Tweaking Sunday, October 24, 2021

tl;dr - new beta release (B5) now available. Very minor changes, including additional prompts for old licenses on Apple silicon. Download away.

Download SuperDuper! v3.5 B5

Signposting

So, B4's rollout went well, with only a few people accusing us of trying to "put one over" on people by invalidating licenses.

Honestly, that wasn't the intent. If we wanted to invalidate licenses and force people to buy new ones, we would have just done so.

Unfortunately, those users didn't read the post that explained the situation, and so we've added another notification about old license use at startup. Hopefully that'll help guide people either to Rosetta or a new license with less mystery and intrigue.

Disk...full?

Somehow, one of the sections of the What's going to happen? section that pointed out when a source might not fit on the destination seemed to be missing.

After an investigation, including combing the beaches to see if it had just headed to a vacation without telling us, we figured out the problem.

Under Catalina and Big Sur, the check of a volume group's size returns the wrong value - it only gives the value of the system volume (which is typically 15GB or so) rather than the whole group (which is System + Data + Preboot + Recovery, etc).

Apple fixed this under Monterey, and so under Monterey and later, the warning will appear as expected.

Emulation

We found another weird problem where the system wasn't always paying attention to the "Use Rosetta" setting the user selected.

As such, under Rosetta, scheduled copies would fail in some cases, because when SuperDuper! was launched to do the copy, it wouldn't use emulation, and thus old serial numbers (see above) would fail to validate.

We now override the behavior to force emulation, on schedules, for users with old serial numbers. And, of course, if you want to run natively, and have an old, pre-2019 license, you can simply purchase a new license.

Wrapping up

That's really about it for this release. We're anticipating the release version quite soon, so speak now...and thanks to everyone who used the Beta versions and sent in feedback!

Download SuperDuper! v3.5 B5

Native Apple Silicon, Big Sur & Monterey Sunday, October 17, 2021

Executive Summary

v3.5 B4 is now out, is fully Apple silicon native, and produces bootable backups for both Big Sur and Monterey. It will be offered as an automatic update for existing v3.5 Beta users, or you can download it here.

Download SuperDuper! v3.5 B4

Windier Version with Details

So far, the Beta has been going great. SuperDuper itself has been performing well, and the only real issues have, as expected, been with Apple's asr tool, which remains a black box that occasionally fails and spits out obscure errors.

Part of rolling these Betas slowly involves trying to figure out how to help users when these failures happen: basically, it's a learning process, and when we can't refer to the code, and the errors are (at best) a number and a word or two, well, it takes some time.

At this point, though, I'm relatively confident I understand how to work around most of asr's failures.

Architectural History

Years ago, when we released v3.2.5, we announced that our old e-commerce provider, eSellerate, had shut down, and they took their license recovery site with them...along with the ability to generate new serial numbers.

Despite asking quite a few times, they were unwilling to provide us with the code necessary to generate or validate eSellerate serial numbers, beyond what we already had, which was an SDK library developed for Power PC and Intel.

You'll note one significant missing chip family there.

And no, it's not 68K.

At that point, we moved to Paddle and our own licenses, which has gone well, but with the upcoming release of v3.5, we found ourselves in an interesting situation: if we wanted to release an Apple silicon-native version of SuperDuper!, we couldn't include the eSellerate license validation, since it doesn't natively support Apple silicon.

This Sounds Ominous

It'll be OK.

Here's what it means: native Apple silicon execution require new-style licenses purchased after June of 2019, when our new license system was released. Old licenses will not work natively... and if you want to run natively on an Apple silicon Mac with registered features, you'll need a new license.

What Do You Mean by "Natively"?

This does not mean you have to buy a new license! If you run with Rosetta 2 (that is, Intel emulation, set in Get Info in Finder), your existing license will work fine. All the same features are available.

But if you want the additional speed of Apple silicon (and want to future-proof yourself should Apple remove Rosetta 2 the way they removed Rosetta), or think, maybe, it's been a while since you registered, you'll want to purchase a new license.

So, Native Apple Silicon Support?

Yes!

What that all means is that this version now fully supports Apple silicon on Macs with those chips. Not much to say about that, other than we took our time and carefully tested to make sure things worked as expected, given the new chip architecture.

And they do.

Monterey Support

This version also supports Monterey, as of the current Beta release (Beta 10). It seems likely that's close to, if not the actual release version of Monterey, which I expect next week sometime, given recent announcements. They don't tell us in advance, though, so things might change.

Data-Only Encryption Support

Previous Beta versions of v3.5 didn't support encrypted data volumes, due to some tests that were required in previous OS releases. We now support pre-encrypted volumes for Data Volume Only copies (that is, copies of your stuff, with no OS, achieved with Smart Update).

Additional Improvements

We've made some changes to various bits and pieces to try to make operation under Big Sur and Monterey clearer and even more reliable.

One thing I didn't expect to be a problem is that, under Big Sur, SuperDuper doesn't explicitly say it's making the backup bootable.

This has confused people, because they think that if it didn't say it was making the backup bootable, then, well, it's not.

But it is.

Here's the deal. The asr tool, mentioned above, actually performs the bless task that makes the backup bootable. Since it was already done, we changed the wording of the "bootable" step to say it was "finalizing" the backup, since we weren't actually making it bootable any more, and I thought it was better to be accurate.

But, given the confusion, I think inaccurate-but-reassuring is a better approach.

In Sum

It's a long time coming, I know, but the final release is in sight. The feedback on the Beta versions has been almost entirely positive, and that's encouraging. So, enjoy, and let me know if you have any problems.

Download SuperDuper! v3.5 B4

Like Riding a Bike Wednesday, May 05, 2021

You know it's been a while since you last released a Beta when you don't quite remember the rhythms of the process.

But, it doesn't take long for muscle memory to kick in... and I'm happy to say that it's going really well. A lot of that is due to you: thanks for your support, encouragement and reports of both success and, occasionally, failure.

To deal with the latter (and celebrate the former), here's v3.5 B2.

A few things to mention as we continue this process:

  • Make sure you have "Check for Updates" turned on in our Preferences. You may have turned it off if you were using the v3.2.5 workaround: it's time to turn it on again, since that's how you'll get Beta updates.
  • Some users were confused by the macOS 11.4 Beta requirements. 11.4 is only required for M1 Macs. Intel Macs work fine with SuperDuper under all versions of Big Sur.
  • Startup volumes will often not show up in the Startup Disk Preference Pane in System Preferences under Big Sur. This does not mean the drive isn't bootable—it's a weird bug in Big Sur.

    To start up from the backup, use Option+Power On on an Intel Mac, or hold down the Power button when powering on with an M1 Mac.

    If you do this, you'll note that the Mac's regular drive doesn't appear in the startup disk preference pane...but the backup drive does! It's weird. But the backup is OK! You can breathe!

  • If you only made a Smart Update, and didn't start with an Erase, then copy backup, you'll note that your backup drive seems "empty". That's because you're looking at the "empty" System volume...and the Data volume is mounted by the OS as "no browse"—basically, it doesn't show up readily in Finder.

    It's still there, though. Use Finder's Go To Folder, in the Go Menu, to open /Volumes and you'll see your Data volume.

    For less stress—and who doesn't want less stress?—start with an Erase, then copy backup.

  • We've actually figured out how to get at least a little useful information out of the weird errors system tools produce:

For Beta 2 we've addressed a few things and polished some others: it's a relatively minor update. Details are in the Revision History, but to highlight a few things:

  • Backup on Connect no longer runs over and over if you use Erase, then copy on Big Sur. This had to do with some very low-level changes to some Big Sur APIs that messed with our code that detected whether we were already running a copy for that volume.

    The bug was useful for testing a zillion back-to-back Erase, then copies, though, so there's that. Yeah! We meant to do that! Sure!

  • Erase, then copy was using asr to copy Catalina, which was unintentional. It now does the same thing v3.3.1 did.
  • Send to Shirt Pocket was failing for users in all versions of SuperDuper due to an locally cached response of a value that had been updated on our server. This turned out to be rather interesting, and frustrating, because we don't have a way of "fixing" old copies...although they will "fix themselves" over time...anyway, it won't happen any more in v3.5 B2 and later.

That'll about do it. Enjoy the new version, and thanks for the emails and tweets!

Download SuperDuper! v3.5 B2

Thanks, again, for using SuperDuper...we look forward to your comments.

Page 1 of 20 pages  1 2 3 >  Last »