Repairing a corrupse sparse disk image

I recently came upon an excellent and detailed account of how to recover a damaged sparse disk image.

I use encrypted sparse images all the time to securely store files. Sparse images have the advantage of automatically resizing as you add more files, so that you don’t have to devote a huge amount of disk space for the image until it’s really needed.

Unfortunately, it’s not uncommon for these images to become corrupt if the resizing process is cancelled before it’s completed (for example if if the Finder crashes while the image is resizing). As sparse images are also used for Filevault-protected Macs, this could potentially lead to a catastrophic loss of data.

So this technique, while somewhat involved, is a welcome addition to my disk repair arsenal.

Written on April 27, 2007

Google directory assistance and SEO

I tried 1-800-GOOG-411 for the first time today and was blown away with the results. It’s directory assistance powered by Google’s local/maps search. I got the info I wanted faster and easier than I would from T-Mobile, and the cost was nearly infinitely lower.

In this world of ubiquitous search, I’m amazed that T-Mobile, Qwest and their ilk can still get away with charging more than a quarter for this service, let alone $1.50. Way to go, Google!

I was so pleased by it, that I decided to do some more searches and try to find my friends’ small businesses, but I couldn’t find them. Same goes for the company that cleans my house and for my favorite masseuse.


Well, they don’t rank very high in a local search. Either they don’t have an address on their site, or the address isn’t on the first page or otherwise placed for good Google ranking. (Yet another reason to not have a splash page without a sitemap to point Google to the right place!)

Businesses really need to get their act together if they expect folks to find them online. Google (and Yahoo et. al.) are quickly replacing the yellow and white pages. It’s not just local businesses, either – people will search for your business or your office online and in Google Maps even if you’re international.

I’ve talked to some small business owners who don’t post an address because they work from home and don’t want folks showing up, or because they want to look more international and bigger. Well, you don’t see Microsoft or P&G hiding their corporate address just because they want to look like a big time company, do you?

Update: One of my small-biz-owning friends noted that his business address is right there on his web page. So’s my other friend’s business address, actually (all three of them). Turns out it helps to register your business with Google in order to get it to show up reliably in local searches.

Written on April 25, 2007

Linux is the nerd who did your homework for you in high school while you were dating the quarterback

I recently helped a friend with a dead Dell recover his files with the help of Knoppix, a Linux distro that boots right off a CD. I shared this success story with some friends of mine, one of whom wittily commented:

> thank you linux… thank you for booting our dead windows computers so we can reinstall windows. You’re a really great… friend.

> I know.. i know.. don’t feel sad.. we’ll always be friends.. I just.. don’t like you that way. Ya know?

> Linux is the nerd who did your homework for you in high school while you were dating the quarterback.

So true…

Written on April 18, 2007

Access authenticated feeds in Google Reader using Yahoo Pipes

As I’ve mentioned before, I’m a Google Reader fanatic. Nothing else handles RSS as smoothly as it does.

The one gigantic failing of Google Reader is that you cannot receive password-protected feeds that request user authentication. I think that sucks. A lot.

So I created a Yahoo Pipe to work around this failing.

Just go to the appropriate pipe and run it. It will prompt you for the feed’s URL, your username and your password. Enter all that information, and you’ll get your own private URL you can use in Google Reader (or any other http:auth-challenged RSS reader).

Warning! Feeds accessed in this way are open to everybody, so if you’re using NewsGator or Bloglines or some other service that shares your feeds with the world, be sure to set the feed un-shared. Otherwise you might get your private access summarily cut off when it’s determined that your personal password is effectively being broadcast to the world. Also, your Yahoo Pipes URL will contain your login and password, so even if you’re okay with giving the world access to premium content, you may not be so pleased about sharing your password with the same unwashed masses. You have been warned. Caveat RSS reader.

Written on March 28, 2007

Making Markdown + Textmate export just a little bit nicer

I tend to use Markdown syntax for most of my document creation needs. Then I use TextMate’s built-in Markdown support to convert to a Word document, RTF, PDF, etc… as necessary.

The one problem I have is that a lot of the so-called MultiMarkdown commands in TextMate leave me opening a file in my /tmp folder. If I forget to do a Save As… my document is lost forever. No good!

Here’s how you fix it…

I’ve added the following command to all the relevant MultiMarkdown commands:

/Developer/Tools/SetFile -a T "$DST.doc"

(Adjust as necessary changing .doc to .rtf or whatever you need, same for the $DST file path variable if that’s different in a custom command)

This uses the SetFile command (free with Apple’s developer tools) to change the converted document into a stationary pad. When you open up a stationary pad, it will create a new, untitled, document, which will prompt you to save to a location the first time you hit save.

Here’s my custom Markdown bundle to convert to a Word document:

# first figure out a name for the result
`| >"$DST.html"`
# textutil is included with Tiger
require_cmd textutil
textutil -convert doc -extension doc "$DST.html"
`# set the file as stationary so it opens as a new doc` `/Developer/Tools/SetFile -a T "$DST.doc"`
# Open it up in Word
open -a "Microsoft Word" "$DST.doc"

[b]Special Bonus Tip:[/b] If you choose “Auto Format” from Word’s Format menu, it will nicely convert the headers into styled headers so that you can keep your document formatted with style sheets, and will also clean up lists in the same fashion! Efficient!

Written on March 21, 2007

Help raise awareness of how great Mac software is!

A while ago, I set up a Squidoo lens to let folks vote on which software for the Mac was best. To my great surprise, it was really popular, and quickly shot up into the top 50 Squidoo lenses, and was nominated as a Lens of the Day by Squidoo.

Well, it’s popularity has diminished slightly, but now the lens is in the running for the Squidoo Lens of the Year!

How cool would it be if a lens promoting the very BEST stuff for the Mac was awarded this prestigious title?

If you want to push for that, go vote on the lens of the year and give your vote to “The Top MacOS X Software. Period.” and let’s see what happens. (It’s hanging out around #30-40 out of nearly 200 lenses as I write this.)

Note that I make NO money and get NOTHING if it’s nominated. Any ad revenue the lens gets goes straight to charity. I just think it’s a great opportunity to show off some of the awesome software that we enjoy using every day. So go vote already!

Written on March 20, 2007

Copying and Metadata Redux, or Apple's Folly

This week, I’ve taken a very close look at how various Mac utilities deal with the plethora of metadata available in MacOS X. I took a look at file archiving and compression programs, backup and synchronization software, and also released a set of test files for other people to verify my own tests and run their own.

In addition to what I’ve published, I’ve corresponded with various closed and open source developers to learn more about what’s going on under the hood, and have come to a few conclusions which I think are worth sharing.

Transferring and backing up

As of the writing of this article, I am aware of only three utilities that will successfully maintain all your file’s information upon copy. The Finder, SuperDuper! and ChronoSync. No other utility that I’ve tested will maintain all your metadata, extended attributes, and resource forks when copying or transferring your files!

Furthermore, when it comes to compressing and archiving your documents on foreign file systems, your options become even more limited. Only one encoding format will keep your file intact when decoded, and that’s the Interarchy Backup Format. All other compression and archiving schemes, including many created specifically for the Mac, lose some measure of your files’ metadata.

The good news is that the Interarchy Backup Format is free and open-source. If you want your own copy (and don’t want to build it out of the source code), you can just download a demo of Interarchy, and you can find it in the application’s bundle in Interarchy/Contents/Resources/FileConverters/Backup and can move that wherever you like to use it for your other files.

I’d be very pleased to see some aspiring programmers give that format a little extra love and turn it into a more robust tool than it is today.

It is also worth noting that there are no real backup programs that can restore your files reliably and also track rolling archives, versions, etc. All we have are sync utilities.

Metadata is hard to keep track of, but vitally important to save

All of this metadata is very well hidden from the user. Aside from certain Finder flags, dates, and Spotlight Comments, the user can go about their day blithely unaware of the resource forks, BSD flags, extended attributes, and complex permission systems working under the surface.

However, when some of that data is lost, the results can range from highly confusing to catastrophic, especially for the active geek!

If you are accustomed to using smart folders, tagging your files with third party utilities, using the Spotlight Comments to further refine your searches and smart folder hierarchies, you will find that nearly everything you do to your files from the command-line will break your delicate system. Even less-geeky users will find thumbnail previews missing from some of their files if the resource fork is lost, or will be unable to find that article they wrote two years ago (because it now says it was created last week).

And third parties are already making heavy use of extended attributes to add an extra level of polish and ease of use to their applications. I’m sure we aren’t far off from seeing “real” information stored in those attributes, rather than just system-specific metadata.

Apple really screwed up, bad

The worst part is that Apple built an OS with this rich metadata layer, and they’re also responsible in large part for how fragile and therefore useless it is.

First off, there’s just too many ways to store this sort of information. Old-style resource forks, new style resource forks, invisible .DS_Store files, extended attributes, type/creator codes, and more. Apple really needs to pick one scheme for metadata and stick to it. (My choice would be Extended Attributes, as they’re by far the most flexible.)

Secondly, Apple needs to provide developers APIs to use to manipulate files that just plain work. As it is, not a single API nor command-line-utility that’s provided by Apple will successfully copy a file and keep its metadata intact. Apple brags that a whole host of command-line programs have been updated to support resource forks and other HFS+ goodies, but in truth, none of them work as well as a drag & drop copy in the Finder. And comments… don’t get me started! Apple hasn’t even provided an API to manage comments at all!

So those few developers who put in the effort to be respectful of our data and move/copy it with precision are forced to roll their own file copying algorithms. These algorithms are slower than a normal copy, and are necessarily idiosyncratic and have to be updated as Apple adds features and functionality to the operating system.


In the end, I’m putting this all on Apple’s doorstep (not that they’re listening to their customers on these points – rsync has been broken to the point of uselessness for the past 3 or 4 point updates of MacOS X). They need to set the standard and give us all the tools to reliably move our files and benefit from the functionality this metadata provides. It’s unreasonable to force users to choose among tools based on which one is most respectful of their files. This should be easy, and it isn’t.

Until then, I hope this guide is informative as you look to the right tools to do what you need.

Written on March 16, 2007