Working with GitHub

August 22nd, 2010

This blog post is intended to be a reference not only for you but for me, because I always forget how it works. “It”, in this case, is Git, or more specifically, GitHub. GitHub is a web site that hosts public Git repositories. It allows anyone in the world to collaborate on any GitHub project without having to worry about repository commit privileges or patch files. If you want to work on a GitHub project, you simply fork the project’s GitHub repository, creating a new GitHub repository in your name. After you make changes to your GitHub repository, you can send pull requests to other GitHub users to notify them of the changes, and they can pull your changes into their own repositories.

While GitHub does make social coding convenient, there is still a significant learning curve. I won’t discuss the process of installing Git or signing up for GitHub account; you can find documentation for those elsewhere. The most confusing aspect of working with GitHub, in my opinion, is repository management, and that’s what I’ll explain here. My explanation will give you the steps of the GitHub workflow using an example project. I’ve chosen ClickToFlash as my example, because people are familiar with it, and I’ve contributed code to the project.

Mystery surrounds the origin of ClickToFlash. The project first appeared on Google Code, posted by an anonymous donor. Not long thereafter, the project disappeared without a trace. We still don’t know the identity of ClickToFlash’s author. (I suspect Holtzman, Holtzmann, or Holzmann.) Fortunately, several developers including ‘Wolf’ Rentzsch preserved the source code, and Rentzsch’s GitHub repository has become ‘official’. Other GitHub repositories such as my own and Simone Manganelli’s are forked from Rentzsch’s. That’s the place to start.

If you haven’t already forked ClickToFlash, you’ll see a “Fork” button on http://github.com/rentzsch/clicktoflash. When I clicked that button, it created a fork of the project at http://github.com/lapcat/clicktoflash. The fork is my public repository, which GitHub users can pull from. The catch is that I can’t work directly on my public repository, because I don’t have shell access to GitHub’s servers where the repository resides. Besides, I wouldn’t be able to run Xcode anyway. Thus, I have to clone the repository on my own local Mac. The URL of my public repository is listed on http://github.com/lapcat/clicktoflash. Actually, there are multiple URLs, but you’ll want to make sure the clone the SSH version, which is read-write. If you clone the read-only version, then you won’t be able to push changes back to the public repository.

$ git clone git@github.com:lapcat/clicktoflash.git

Your private, local clone automatically has a master branch that matches the master branch of your public, remote repository.

$ cd clicktoflash
$ git branch
* master
$ git status
# On branch master
nothing to commit (working directory clean)

Your remote cloned repository is given the special name origin by your local clone repository.

$ git remote
origin

The local master branch also tracks the remote repository, so that git fetch, git pull, and git push automatically apply to origin when run with master checked out.

$ git remote show origin
* remote origin
  Fetch URL: git@github.com:lapcat/clicktoflash.git
  Push  URL: git@github.com:lapcat/clicktoflash.git
  HEAD branch: master
  Remote branches:
    cutting-edge tracked
    master       tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)

Despite the fact that the local repository is a clone of origin, and origin is a fork of rentzsch, the local repository knows nothing of rentzsch. [Expletives censored.] If you want to pull changes from rentzsch, you need to add it as a remote repository. In this case, you can use the read-only URL, because you can’t push changes to his repository.

$ git remote add rentzsch git://github.com/rentzsch/clicktoflash.git
$ git remote
origin
rentzsch

Some people suggest upstream for the name of the remote repository, but I find this needlessly confusing. The name rentzsch tells me exactly where the changes are coming from. Unlike upstream, it’s not abstract or subject to misinterpretation with origin.

Note that unless you use the -f option, the rentzsch remote is not automatically fetched, so you’ll need to fetch it manually. I also find this needlessly confusing and wish the default behavior were to fetch rather than not fetch. You might find yourself perplexed, for example, if you try to create a new branch from the remote.

$ git branch rentzsch-master rentzsch/master
fatal: Not a valid object name: 'rentzsch/master'.
$ git branch -r
  origin/HEAD -> origin/master
  origin/cutting-edge
  origin/master
$ git fetch rentzsch
$ git branch -r
  origin/HEAD -> origin/master
  origin/cutting-edge
  origin/master
  rentzsch/1.4.2-64bit
  rentzsch/cutting-edge
  rentzsch/gh-pages
  rentzsch/master
$ git branch rentzsch-master rentzsch/master
Branch rentzsch-master set up to track remote branch master from rentzsch.

I recommend that you create a branch specifically to track the forked repository, as I do in the last instruction above. Then no matter what changes you make, you can still look at the ‘official’ version of the project by checking out the rentzsch-master branch. If you use the remote branch rentzsch/master as the starting point for the local branch rentzsch-master, the local branch automatically tracks the remote repository rentzsch, just as the local master automatically tracks the remote origin.

$ git remote show rentzsch
* remote rentzsch
  Fetch URL: git://github.com/rentzsch/clicktoflash.git
  Push  URL: git://github.com/rentzsch/clicktoflash.git
  HEAD branch: master
  Remote branches:
    1.4.2-64bit  tracked
    cutting-edge tracked
    gh-pages     tracked
    master       tracked
  Local branch configured for 'git pull':
    rentzsch-master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (fast-forwardable)

When changes occur in the master branch of the remote rentzsch repository, here is the procedure for merging them:

$ git checkout rentzsch-master
$ git fetch
$ git merge rentzsch/master
$ git checkout master
$ git merge rentzsch-master
$ git push

You could use the one step git pull instead of the two steps git fetch and git merge rentzsch/master. However, I’ve heard it suggested that git pull sometimes causes problems, though that issue is beyond the scope of this blog post. Anyway, what you’re doing with these steps is first merging the remote rentzsch repository changes into the local rentzsch-master branch, then merging the local rentzsch-master branch into the local master branch, and finally pushes the local changes to the remote origin repository. The somewhat convoluted procedure is necessary because you cannot directly pull the remote rentzsch changes into origin, they have to go through the local repository.

The key to successful repository management, I believe, is to never write code on the local master branch. I’ve learned this important lesson by trial and error. In particular, if you try to merge changes from a remote repository into master while you have local changes on master that haven’t yet been pushed to origin, everything can blow up. It’s best to keep master as pure as possible. In fact, it’s best to keep all your tracking branches as pure as possible. With Git, branches are cheap. When you want to make local changes, always create and check out a new branch, and then merge the changes back into the tracking branch when you want to push.

As far as I can tell, origin by default will contain the same branches that existed in the rentzsch repository at the time you forked it. Consequently, http://github.com/lapcat/clicktoflash only has 2 branches, whereas Rentzsch’s GitHub repository has 4. In any case, the local clone only has master by default. New branches created in the local repository with local starting points are not automatically pushed to origin. This means you can safely hack on local code changes in a branch without exposing your mess to the public. If you want to work on another public ClickToFlash branch, such as rentzsch/cutting-edge instead of rentzsch/master, you’ll need to create a new local branch.

$ git branch rentzsch-cutting-edge rentzsch/cutting-edge
Branch rentzsch-cutting-edge set up to track remote branch cutting-edge from rentzsch.
$ git branch cutting-edge rentzsch-cutting-edge
$ git checkout cutting-edge
Switched to branch 'cutting-edge'

Again, we have both a branch rentzsch-cutting-edge that is a duplicate of rentzsch/cutting-edge and a branch cutting-edge that includes your changes. This mirrors the arrangement of the branches rentzsch-master and master.

If origin already contains a cutting-edge branch, then git push should be sufficient to push your local changes. (Beware: in another maddening default behavior, git push will push all branches that exist on origin, i.e., master and cutting-edge, not just the currently checked out branch.) On the other hand, if origin does not yet contain a cutting-edge branch, you’ll need to use git push origin cutting-edge to create the branch on origin.

I hope this mini tutorial helps you to work with GitHub more efficiently and with fewer headaches (from banging your head against the wall). If you have further questions, feel free to ask … someone else, because I don’t know the answer.

A world without Twitter

July 9th, 2010

This post has been rolling around in my mind for a while — along with my bank balance, the song “Gee, Officer Krupke”, and Jennifer Aniston’s … nevermind. I was inspired to finally commit it to paper, or whatever the heck this big rectangular white thing is, by a recent tweet from @danielpunkass, AKA Daniel Jalkut, AKA Mr. MarsEdit, AKA The Boston Strangler. As John Lennon might say if he were alive today, and unable to come up with any new ideas in the last 40 years: imagine there’s no Twitter. It’s easy if you try. Command-r, command-r.

The credit, or blame, goes to @rentzsch for luring me onto Twitter. Not once, but twice. The first time was as a requirement for attending the C4[2] conference. My initial run on Twitter came to a halt later when I famously (for some definition of famous) took a 5-month hiatus. At C4[3], however, ‘Wolf’ (as he is known in the porn industry) persuaded me to return to Twitter. You can see why C4 was evil and had to be abolished.

For a whale, err, while, Twitter was fairly stable, relatively speaking. Lately, though, the uptime has been absolutely horrid, even when there’s no World Cup game. The only thing that never fails is the status blog. (Why don’t they make Twitter out of the status blog? Or the black box?) The Twitter developers don’t really inspire confidence; they keep pushing code changes that are supposed to improve reliability but end up causing outages. Furthermore, after all this time they’ve still failed to implement any convenient way to follow conversation threads on Twitter. The @reply is only the most rudimentary step in this direction. To me, that suggests the developers have completely missed what is important and special about Twitter, the social aspect.

Another threat to Twitter’s future, besides the continued technical suckage and danger of collapsing under its own weight, is the need to make money. We’ve grown accustomed to receiving this service for free, but Twitter is not a charity, it’s a for-profit corporation. We’re starting to see this manifested in ‘promoted’ trends. Third-party spammers have existed on Twitter from the beginning, of course, but the question is whether commercialization will transform Twitter itself into something repulsive (cf. iAds). On the other hand, there’s a possibility that Twitter cannot make itself commercial enough. If Twitter doesn’t generate sufficient revenue to sustain itself and make a profit over the long term, it may have to close up shop and go out of business. It certainly wouldn’t be the first dot-com service to disappear. I believe that if Google wanted, they could create a serious competitor to Twitter, backed with Google’s superior knowledge, reliability, and bandwidth. Indeed, I’m surprised that Google hasn’t attempted this already. Maybe they don’t see any profit in it. But that doesn’t bode well for Twitter either.

As good programmers, we always have a backup. (Right? Right? Redundant and off-site, right?) The purpose of this post is to consider, what is our backup to Twitter? If Twitter dies, or if we’re compelled to leave, where do we go? We could go back to our pre-Twitter existence, back to our spouses, children, neighbors, and back to writing long blog posts instead of pithy 140- character tweets. We could, but blech! The un-tweeted life is not worth living. There would always be something missing. Twitter allows us to connect regularly with many friends that we don’t get to see very often (and to make arrangements when we do see them). It introduces us to people that we’d never meet or be able to talk to otherwise. It provides a unmoderated, unstructured forum for sharing crucial tidbits of information and knowledge with people we trust and respect (as well as everyone else, for better or worse). Twitter is like Cheers for the internet, except without Sam Malone. Oh wait, it’s got Sam Malone. The problem is, we don’t have a good backup. Without Twitter, we’ve got nothing. (Please don’t even mention Gary’s Old Towne Tavern.)

Admittedly, there are some alternatives to Twitter, a few services that are similar, but they all suffer from the same inherent limitations as Twitter. They present a single point of failure. If Twitter’s servers are overloaded, then the service becomes unusable. Everyone gets the fail whale. In order to reliably handle the traffic, a centralized service require a large amount of resources, and thus money. So the only hope for a reliable, centralized Twitter-like service is heavily commercialization, and it’s not clear that a heavily commercialized service is one we’d want to spend our time on. Would a profit train be recognizable as the little Twitter we know and (sometimes) love? Besides, any centralized service is still prone to catastrophic failure. Even the mighty Amazon goes down.

So I had this idea. In the shower. (With Jennifer Aniston.) What if we made a distributed Twitter? Like DVCS. All the hip young programmers today use DVCS, whether that’s Git, or Mercurial, or … yeah, Git or Mercurial. The key to DVCS is that there’s no single point of failure, no centralized repository. Each working copy is its own repository, with the entire commit history. And you can commit locally! If one repository goes offline or disappears entirely, the other repositories can continue operating indefinitely without it. Nothing is lost. Similarly, each user of Dwitter (clever, huh?) would install Dwitter software on a web server, which would keep a record not only of the user’s own dweets but also the dweets of everyone the user follows.

When you follow someone, your Dwitter server would send a request containing information such as your name and server address to the Dwitter server of the person you follow. The followee’s Dwitter server stores this information, and then when that person dweets, his/her/its/their Dwitter server would send the dweet to the Dwitter servers of all followers. To prevent connection overload, the rate of sending out dweets to followers would be limited, perhaps to something like 1.5 followers per second. This would allow a dweet to be propagated to 5000 followers in less than an hour. And if you compose another dweet while your followers are still getting notified of the previous dweet, the dweet notifications would get consolidated for the followers who haven’t yet been notified. Furthermore, any Dwitter user directly mentioned in a dweet would receive immediate notification of the dweet, prior to anyone else, to facilitate quick conversation.

Obviously, this system won’t scale for someone who has a million Twitter followers. However, the only people who have a million Twitter followers are celebrities. There are plenty of other, better outlets for celebrity news, we don’t need Twitter for that. Note that celebrities did not make Twitter popular. Rather, celebrities came to Twitter because it was already popular. What makes the presence of celebrities on Twitter mostly useless is that they rarely participate in the culture of Twitter, they don’t foster conversation. You can’t have a conversation with a million followers, because that doesn’t scale either.

A potentially more serious problem for the distributed version of Twitter is the difficulty of use. Running your own Dwitter server requires a fair bit of technical knowledge. It’s much more difficult than going to a web site and signing up for an account. This may exclude mom and pop from joining Dwitter. However, I personally don’t consider the loss of mom and pop to be worse than the loss of celebrities. Frankly, I don’t want my mom following me on Twitter. Have you read my tweets? She’d make me wash my mouth out with soap! Almost all of the people I follow are programmers. They should be fully capable of setting up a Dwitter server on their own internet domain. Even though Twitter is completely public (except for protected tweets, you bastards), the irony is that it’s best suited for insulated groups such as programmers. 140 characters is not enough to teach outsiders your terminology and concepts. When you tweet, you’re forced to leave out a lot and to use shorthand that only likeminded people will understand. Tweeting is kind of like sending out coded messages. (@IwayAmwayAwayUssianrayYspay)

Even with the distributed system, there may be options for less technical users. WordPress operates under a similar model. You can install WordPress software on your own web server and run your blog yourself, or you can sign up to have a hosted blog on wordpress.com, where they take care of the technical aspects for you. The key is that even if one hosted Dwitter service fails, that would only prevent new dweets from the hosted accounts. The rest of the Dwitterverse would go on as usual, and the archives of the hosted dweet accounts would continue to exist on the Dwitter servers of their followers.

I don’t personally have the expertise to design a distributed Twitter-like service. I have some web knowledge, but I’m mainly a desktop programmer. Nonetheless, I will take all glory and riches arising from this idea. All I ask in return for all the glory and riches is that the designers of the new service don’t create a half-assed API. I’m tired of crappy, fatally flawed designs becoming popular by virtue of being first to market. Indeed, this is how Twitter itself became popular. Please, do it right the first time. Or I’ll be looking for an alternative to you too.

Safari extension: autocomplete

June 10th, 2010

Safari has a feature called form autocompletion, or AutoFill, that reads the username and password you type into a web form, saves them to your keychain, and automatically fills them in from the keychain the next time you visit the web form. This feature is completely opt-in: you can enable and disable it in Safari’s preferences, and even when it’s enabled, Safari will ask you before saving the username and password from each web form. Many other web browsers have a similar feature; it appears to have been introduced by Internet Explorer.

Unfortunately, a number of web sites (including my bank, for example) choose to disable autocompletion in a misguided attempt at security. Autocompletion can be disabled by using the attribute autocomplete=off in a web form. The idea behind disabling autocompletion seems to be that it leaves the account holder vulnerable to someone else accessing the computer and logging into the account. I believe that this is misguided for at least two reasons. First, autocompletion is opt-in, so the user can decide whether to save passwords on a particular machine. Anyone who chooses to save their passwords on a public terminal is an idiot. Indeed, anyone who logs in to their bank account on a public terminal deserves to be hacked and lose all their money, because who knows what manner of keyloggers or other malware could be running on the machine? I feel safe turning on AutoFill on my computer because I’m the only person who ever has access to it.

Another reason that disabling autocompletion is misguided is that it encourages the use of weak passwords. For security, I generate very long, random passwords for web sites and save them to my keychain. There’s no way I could memorize even one of my web site passwords, much less all of them. It’s difficult for almost anyone to memorize a bunch of web site passwords. Disabling autocompletion forces the user to type them in manually every time, and this encourages the use of short, easy to remember passwords. Worse, it encourages password sharing among different web sites. Thus, if an attacker can guess or brute-force one password, the attacker suddenly has access of all of a persons’s web site accounts. That’s terrible security.

WebKit, the web engine underlying Safari, respects the autocomplete attribute, and there’s no preference or API to make WebKit ignore the attribute. However, I discovered an excellent script written by Michael Kisor called Autocomplete Always On! that actually patches the WebKit framework itself on your system so that it ignores autocomplete. It works by changing one byte in the file

/System/Library/Frameworks/WebKit.framework/Versions/A/Frameworks/WebCore.framework/Versions/A/WebCore

transforming the string autocomplete into xutocomplete. After that change, WebKit looks for xutocomplete=off but never finds it in the web form, which means autocomplete never gets disabled. WebKit is open source, so we can verify the consequences of the patch.

The only downside of the Autocomplete Always On! script is that it needs to be re-run after any software update to the WebKit framework on your system. I’ve been using the script for years with no trouble … until Safari 5. After installing Safari 5, I discovered that the script was no longer effective in re-enabling autocompletion. This was no flaw in the script, however. Autocomplete Always On! still works as designed on the version of WebKit shipped with Safari 5. The problem is that the Safari 5 binary itself seems to include a new check for the autocomplete attribute in web forms. I’ve verified this behavior in the debugger. If the form contains autocomplete=off, then Safari 5 never calls the (private) WebKit method -[WebHTMLRepresentation elementDoesAutoComplete:], so Safari doesn’t even ask WebKit whether autocomplete is enabled for the web form element. It is possible to patch the Safari binary just like the WebCore binary, after which Safari 5 will call -[WebHTMLRepresentation elementDoesAutoComplete:], making the WebCore patch effective again. Unfortunately, patching the Safari binary breaks codesigning for the application, and the keychain uses codesigning to determine whether an application can access saved passwords.

That’s the bad news. The good news is that Safari 5 also introduced extensions. A Safari extension is like a plug-in, because it can run code inside the browser, but unlike a plug-in, an extension doesn’t run native C or Objective-C code but rather HTML or Javascript code. For example, an extension can specify some Javascript to run on page load. I found a script written by Andreas Huber that removes any autocomplete attributes from a web form, and I cleaned it up a bit for inclusion in a Safari extension. With my autocomplete extension installed, you don’t have to patch WebKit or Safari, because the autocomplete attributes are simply removed from the web page before the browser checks for their existence.

I’m making my autocomplete Safari extension available — in September. No, today! You can download the extension here. To install the extension on your computer, you first need to check “Show Develop menu in menu bar” at the bottom of Safari’s Advanced preferences. Then in the Develop menu, you need to check “Enable Extensions”. After extensions are enabled, you can simply double-click the Safari extension in Finder to install.

Enjoy! In lieu of donations, I am accepting sexual favors. Or an icon.

Beware DIY component replacement

February 17th, 2010

I own a 17-inch 2.33GHz Intel Core 2 Duo MacBook Pro. When I purchased it, I chose the largest hard drive available, the 200GB 4200rpm option. Over the years, that hard drive had become quite full, and it’s also very slow compared to newer hard drive models. Thus, I decided recently to replace it with a 500GB 7200rpm Seagate Momentus hard drive.

My original plan was to pay a local computer repair shop to swap the drives. I’m a software guy, not a hardware guy, and my time and sanity are (somewhat) valuable. However, some people who shall remain nameless shamed me into doing it myself, arguing that a proper computer geek should be able to replace components easily. (Sure, right, just like the midplane was a ‘user-serviceable’ part of the iMac G5.)

I found several online videos with instructions for replacing my MacBook Pro’s hard drive. For example, Other World Computing has a video here, which specifically mentions my model: MacBookPro2,1. After studying the instructions carefully, I set about to do it myself

The first problem was that the OWC video mentions the requirement of a #00 Phillips screwdriver, which I did not have but which I acquired for the purpose of this hard drive replacement. Nonetheless, it turns out that the #00 was not actually the right size for the screws. Fortunately, I did already happen to have a screwdriver that fit the screws in my MacBook Pro. Otherwise, I would have been forced to scrub the replacement at the start.

This screwdriver mixup was but a minor blip compared to the next and worst problem. The videos completely failed to show or mention that there was a very short and easily snapped wire on the far left side of the machine, running from the bottom board to the top case. I did not discover that this wire existed until I opened the top case, and after a few seconds of attempting to lift the top case, the wire indeed snapped.

Here’s a photo of the broken black wire:
Photo of broken black wire

And here’s a photo of where it was attached on the top case:
Photo of top case

At the time, I had no idea what the wire was for. I had to take the machine in to the genius bar at my local Apple Store. The genius examined it and informed me, to my great relief, that the wire was for the built-in microphone. My microphone is now broken and inoperative, but it could have been much worse.

Another thing the videos failed to mention is that the metal tabs with screw holes on the top case are extremely fragile. When I put the top case back on, one of the tabs broke off. It’s not a big deal, there are enough other screws around the case to keep it securely in place, but it’s annoying, and there is a little area of the bottom case on the right side that is sticking out slightly and bent.

In the end, I’m happy with my new hard drive. My advice, though, is to pay a professional to perform the replacement, don’t try to do it yourself. Ignore the DIY demons whispering in your ear. I don’t think so, Tim.

Local variables are free

December 19th, 2009

This is part II of my irregularly scheduled series on compiler optimization. In part I, I explained how the compiler can optimize away return statements, resulting in missed breakpoints. My given workaround to that problem, though effective, was very ugly and architecture-dependent, much like Cowboys Stadium.

(gdb) break *0x00001fc5 if $eax != 0

Although there’s not much we can do to prevent the compiler optimization, we can greatly simplify our conditional breakpoint. I had suggested rewriting the source code, which was awe-inspiringly prescient, because that’s what I’m going to do now. Here’s the original code:

8	if (ShouldReturn())
9		return;

And here’s the revised code:

8	int localVar = ShouldReturn();
9	if (localVar)
10		return;

The return at line 10 will still be optimized away. However, the revised code allows us to set a simple breakpoint at line 9 that will stop when we want:

(gdb) break 9 if localVar != 0

No knowledge of the architecture, machine registers, or assembly language is required.

From the beginning of time (January 1970, of course), programmers have struggled over coding style. Objective-C programmers, for example, expend undue effort arranging their brackets. (I have [NSMutableArray array] going to the Final Four.) For some, bracket-making becomes a kind of game or contest.

[[[[[[[[[[[[[See how] many] method] calls] we] can] fit] on] one] line] of] source] code];

I’ve changed my coding style over the years, but I’ve settled on one fundamental principle: write your code so that it’s easy to debug. All your fancy margin-aligning isn’t going to help when you need to figure out why your app keeps exploding. If you have nested method calls on one line of code, you can’t easily set a breakpoint in the middle. That’s why I prefer as much as possible to have only one method call per line of code, and create a local variable to store the return value.

There is a misconception that local variables are expensive, in terms of either computation or memory. The truth is that local variables are very cheap, the value meals of the computing world. (Would you like trans fat with your saturated fat?) It only takes one machine instruction to store a pointer address to a local variable. One machine instruction is really quite fast, about as fast as you can get — at least with restrictor plates. With regard to memory, local variables only take up stack space. To create a local variable, you simply move the stack a little. When the method or function returns, the stack is moved back, and thereby the space reserved for local variables is automatically recovered. Of course, you don’t want to create large C arrays on the stack, but a pointer to an Objective-C object only takes 4 bytes on the stack for 32-bit, 8 bytes for 64-bit. The default 32-bit stack size is 8MB, so you’re not going to run out of space unless you have deeply recursive calls.

Even these small costs are only relevant in the context of your app’s unoptimized, debug configuration. For your customers, on the other hand, local variables are free. As in Mumia, or Bird. When you compile your app using the release configuration, the local variables disappear, the compiler optimizes them away. (By the way, this is one of the reasons that debugging the release build of your app can be a frustrating and/or wacky experience.) To see the optimization in action, let’s consider some sample code:

1  #import <Foundation/Foundation.h>
2
3  @interface MyObject : NSObject {}
4  @end
5
6  @implementation MyObject
7
8  -(NSString *)myDirectProcessName {
9  	return [[[NSProcessInfo processInfo] processName] lowercaseString];
10 }
11
12 -(NSString *)myRoundaboutProcessName {
13 	NSString *myRoundaboutProcessName = nil;
14 	NSProcessInfo *processInfo = [NSProcessInfo processInfo];
15 	NSString *processName = [processInfo processName];
16 	NSString *lowercaseString = [processName lowercaseString];
17 	myRoundaboutProcessName = lowercaseString;
18 	return myRoundaboutProcessName;
19 }
20
21 @end
22
23 int main(int argc, const char *argv[]) {
24 	NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
25 	MyObject *myObject = [[[MyObject alloc] init] autorelease];
26 	NSLog(@"My direct process name: %@", [myObject myDirectProcessName]);
27 	NSLog(@"My roundabout process name: %@", [myObject myRoundaboutProcessName]);
28 	[pool release];
29 	return 0;
30 }

The above code is obviously contrived and useless. It only has value for explanatory purposes, and perhaps in the app store for $0.99. The methods -myRoundaboutProcessName and -myDirectProcessName do the same thing, the former with and the latter without local variables. Here’s the i386 disassembly for the methods when compiled using the debug configuration:

-[MyObject myDirectProcessName]:
00001d2a	nop
00001d2b	nop
00001d2c	nop
00001d2d	nop
00001d2e	nop
00001d2f	nop
00001d30	pushl	%ebp
00001d31	movl	%esp,%ebp
00001d33	pushl	%ebx
00001d34	subl	$0x14,%esp
00001d37	calll	0x00001d3c
00001d3c	popl	%ebx
00001d3d	leal	0x000012e8(%ebx),%eax
00001d43	movl	(%eax),%eax
00001d45	movl	%eax,%edx
00001d47	leal	0x000012e4(%ebx),%eax
00001d4d	movl	(%eax),%eax
00001d4f	movl	%eax,0x04(%esp)
00001d53	movl	%edx,(%esp)
00001d56	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001d5b	movl	%eax,%edx
00001d5d	leal	0x000012e0(%ebx),%eax
00001d63	movl	(%eax),%eax
00001d65	movl	%eax,0x04(%esp)
00001d69	movl	%edx,(%esp)
00001d6c	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001d71	movl	%eax,%edx
00001d73	leal	0x000012dc(%ebx),%eax
00001d79	movl	(%eax),%eax
00001d7b	movl	%eax,0x04(%esp)
00001d7f	movl	%edx,(%esp)
00001d82	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001d87	addl	$0x14,%esp
00001d8a	popl	%ebx
00001d8b	leave
00001d8c	ret
-[MyObject myRoundaboutProcessName]:
00001d8d	nop
00001d8e	nop
00001d8f	nop
00001d90	nop
00001d91	nop
00001d92	nop
00001d93	pushl	%ebp
00001d94	movl	%esp,%ebp
00001d96	pushl	%ebx
00001d97	subl	$0x24,%esp
00001d9a	calll	0x00001d9f
00001d9f	popl	%ebx
00001da0	movl	$0x00000000,0xe8(%ebp)
00001da7	leal	0x00001285(%ebx),%eax
00001dad	movl	(%eax),%eax
00001daf	movl	%eax,%edx
00001db1	leal	0x00001281(%ebx),%eax
00001db7	movl	(%eax),%eax
00001db9	movl	%eax,0x04(%esp)
00001dbd	movl	%edx,(%esp)
00001dc0	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001dc5	movl	%eax,0xec(%ebp)
00001dc8	movl	0xec(%ebp),%edx
00001dcb	leal	0x0000127d(%ebx),%eax
00001dd1	movl	(%eax),%eax
00001dd3	movl	%eax,0x04(%esp)
00001dd7	movl	%edx,(%esp)
00001dda	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001ddf	movl	%eax,0xf0(%ebp)
00001de2	movl	0xf0(%ebp),%edx
00001de5	leal	0x00001279(%ebx),%eax
00001deb	movl	(%eax),%eax
00001ded	movl	%eax,0x04(%esp)
00001df1	movl	%edx,(%esp)
00001df4	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001df9	movl	%eax,0xf4(%ebp)
00001dfc	movl	0xf4(%ebp),%eax
00001dff	movl	%eax,0xe8(%ebp)
00001e02	movl	0xe8(%ebp),%eax
00001e05	addl	$0x24,%esp
00001e08	popl	%ebx
00001e09	leave
00001e0a	ret

As expected, -myRoundaboutProcessName makes more room on the stack than -myDirectProcessName:

00001d34	subl	$0x14,%esp
00001d97	subl	$0x24,%esp

At 00001da0, -myRoundaboutProcessName sets the value of the local variable to nil, as in line 13 of the source code. The interesting differences, though, are immediately after the calls to objc_msgSend(). By the standard ABI, the register eax contains the return value of objc_msgSend(). In -myDirectProcessName, the value in eax is simply moved to the register edx:

00001d5b	movl	%eax,%edx

In contrast, -myRoundaboutProcessName first stores the value on the stack before moving it to edx. The address on the stack is the space reserved for the local variable:

00001dc5	movl	%eax,0xec(%ebp)
00001dc8	movl	0xec(%ebp),%edx

After the final objc_msgSend() call, -myDirectProcessName doesn’t bother to do much, because the return value in eax will become the return value of the whole method. In -myRoundaboutProcessName, it needs to store values in local variables as in lines 16 and 17 of the source code:

00001df9	movl	%eax,0xf4(%ebp)
00001dfc	movl	0xf4(%ebp),%eax
00001dff	movl	%eax,0xe8(%ebp)
00001e02	movl	0xe8(%ebp),%eax

So that’s how the methods differ in the unoptimized build. Now let’s see what happens when we use the release configuration. Here’s the optimized disassembly for -myDirectProcessName:

-[MyObject myDirectProcessName]:
00001dce	pushl	%ebp
00001dcf	movl	%esp,%ebp
00001dd1	subl	$0x18,%esp
00001dd4	movl	0x00003000,%eax
00001dd9	movl	%eax,0x04(%esp)
00001ddd	movl	0x0000302c,%eax
00001de2	movl	%eax,(%esp)
00001de5	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001dea	movl	0x00003004,%edx
00001df0	movl	%edx,0x04(%esp)
00001df4	movl	%eax,(%esp)
00001df7	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001dfc	movl	0x00003008,%edx
00001e02	movl	%edx,0x0c(%ebp)
00001e05	movl	%eax,0x08(%ebp)
00001e08	leave
00001e09	jmpl	0x0000400a	; symbol stub for: _objc_msgSend

The optimized method is significantly shorter, as expected from the compiler option -Os. First, you’ll notice that all those pesky nop instructions have been deleted. Stallman put them in unoptimized builds just to annoy us. (Or they may have been for Fix and Continue, but I always assume the worst.) There are additional optimizations as well that I won’t belabor here, because I’m eager to get to the climax. (Sorry, dear.) For your enlightenment and enjoyment, here’s the optimized disassembly for -myRoundaboutProcessName:

-[MyObject myRoundaboutProcessName]:
00001e0e	pushl	%ebp
00001e0f	movl	%esp,%ebp
00001e11	subl	$0x18,%esp
00001e14	movl	0x00003000,%eax
00001e19	movl	%eax,0x04(%esp)
00001e1d	movl	0x0000302c,%eax
00001e22	movl	%eax,(%esp)
00001e25	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001e2a	movl	0x00003004,%edx
00001e30	movl	%edx,0x04(%esp)
00001e34	movl	%eax,(%esp)
00001e37	calll	0x0000400a	; symbol stub for: _objc_msgSend
00001e3c	movl	0x00003008,%edx
00001e42	movl	%edx,0x0c(%ebp)
00001e45	movl	%eax,0x08(%ebp)
00001e48	leave
00001e49	jmpl	0x0000400a	; symbol stub for: _objc_msgSend

Identical! Ah, that’s nice. Smoke ‘em if you got ‘em.

In conclusion, feel free to sprinkle, pepper, dash, or even drown your code with local variables. And with the engineering hours of debugging time you save, get me a nice (not free) present. I’m partial to flavored coffee and unflavored MacBooks.

Why did my breakpoint not get hit?

November 16th, 2009

This is part I of a II+ (take that, trademark trolls) part series on compiler optimization. For the gcc compiler, you can specify the level of optimization with various -O options. The default for compiling is -O0, which means do not optimize. As we shall see, however, the compiler always optimizes to an extent. That is to say, gcc -O0, you lie!

The primary reason for using the -O0 option (besides to avoid compiler optimization bugs) is to facilitate debugging of your code. With higher levels of optimization, the compiler is given more freedom to ‘ignore’ your source code in writing machine instructions, as long as the results are the same. Although it is possible to debug optimized binaries, the experience is often confusing and unhelpful for the programmer (much like reading cocoa-dev). Turning off optimization gives the closest correlation between source code and machines instructions. Yet even with no optimization, the correlation is not perfect, and this can lead to debugging problems.

Let’s consider a simple example:

$ cat > returnbreak.c
#include <stdio.h>

int ShouldReturn(void) {
	return 1;
}

void HelloWorld(void) {
	if (ShouldReturn())
		return;

	printf("Hello, World!\n");
}

int main(int argc, const char *argv[]) {
	HelloWorld();
	return 0;
}
$ gcc -g -O0 -o returnbreak returnbreak.c
$ gdb returnbreak
GNU gdb 6.3.50-20050815 (Apple version gdb-966) (Tue Mar 10 02:43:13 UTC 2009)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ... done

(gdb) list HelloWorld
2
3	int ShouldReturn(void) {
4		return 1;
5	}
6
7	void HelloWorld(void) {
8		if (ShouldReturn())
9			return;
10
11		printf("Hello, World!\n");
(gdb) break 9
Breakpoint 1 at 0x1fc9: file returnbreak.c, line 9.
(gdb) run
Starting program: /Users/jeff/Desktop/returnbreak
Reading symbols for shared libraries ++. done

Program exited normally.

WTF?!? Why did my breakpoint not get hit?

(gdb) info break
Num Type           Disp Enb Address    What
1   breakpoint     keep y   0x00001fc9 in HelloWorld at returnbreak.c:9

Hmm, that seems ok. Let’s try something else.

(gdb) break HelloWorld
Breakpoint 2 at 0x1fc0: file returnbreak.c, line 8.
(gdb) info break
Num Type           Disp Enb Address    What
1   breakpoint     keep y   0x00001fc9 in HelloWorld at returnbreak.c:9
2   breakpoint     keep y   0x00001fc0 in HelloWorld at returnbreak.c:8
(gdb) run
Starting program: /Users/jeff/Desktop/returnbreak 

Breakpoint 2, HelloWorld () at returnbreak.c:8
8		if (ShouldReturn())
(gdb) c
Continuing.

Program exited normally.

Odd, it hits the breakpoint at line 8 but not at line 9. The breakpoint on line 9 is at address 0x00001fc9, so let’s look at the (i386) disassembly for that:

(gdb) disassemble 0x00001fc9
Dump of assembler code for function HelloWorld:
0x00001fb3 <HelloWorld+0>:	push   %ebp
0x00001fb4 <HelloWorld+1>:	mov    %esp,%ebp
0x00001fb6 <HelloWorld+3>:	push   %ebx
0x00001fb7 <HelloWorld+4>:	sub    $0x14,%esp
0x00001fba <HelloWorld+7>:	call   0x1fbf <HelloWorld+12>
0x00001fbf <HelloWorld+12>:	pop    %ebx
0x00001fc0 <HelloWorld+13>:	call   0x1fa6 <ShouldReturn>
0x00001fc5 <HelloWorld+18>:	test   %eax,%eax
0x00001fc7 <HelloWorld+20>:	jne    0x1fd7 <HelloWorld+36>
0x00001fc9 <HelloWorld+22>:	lea    0x30(%ebx),%eax
0x00001fcf <HelloWorld+28>:	mov    %eax,(%esp)
0x00001fd2 <HelloWorld+31>:	call   0x3005 <dyld_stub_puts>
0x00001fd7 <HelloWorld+36>:	add    $0x14,%esp
0x00001fda <HelloWorld+39>:	pop    %ebx
0x00001fdb <HelloWorld+40>:	leave
0x00001fdc <HelloWorld+41>:	ret
End of assembler dump.

When ShouldReturn() returns, the return value is in the register eax. The test instruction at 0x00001fc5 performs a bitwise AND of the two operands — which in this case are the same. If the result is non-zero — and in this case the result is 1 — the Zero Flag in the EFLAGS register is set to 0. This instruction corresponds to evaluating the conditional on line 8 of our source code. Then the jne instruction at 0x00001fc7 jumps to a certain address if the Zero Flag is 0. In our source code, the flow of control should move to the return statement on line 9 when the conditional evaluates to non-zero. According to the machine instructions, on the other hand, it jumps to 0x1fd7 when the conditional evaluates to non-zero. This address is the beginning of the standard function epilog, which restores the stack and registers to their previous state before returning.

The problem here is that while the function HelloWorld() has two exit points in our source code, it only has one exit point in the machine instructions. In essence, the compiler has optimized for size, despite our use of the -O0 option. Given the generated machine instructions, there is nowhere to put a breakpoint that will only be hit when the conditional at line 8 evaluates to non-zero. A breakpoint at 0x00001fc5 or 0x00001fc7 would be hit whenever the conditional is evaluated, which is always. A breakpoint at 0x00001fd7 would be hit whenever the function returns, which is always as well. Unfortunately, gdb places the breakpoint at 0x00001fc9, which is actually the opposite of what we intended, because it only gets hit when the conditional evaluates to zero. This is why the program exits normally without ever hitting the breakpoint. I consider this to be a bug in gdb; it would be better, I think, if it would just fail and give an error when we try to set the breakpoint. Of course, it may be a bug in gcc that it optimizes away our multiple exit points with optimization off. But hey, what do you expect from free software?

There are several workarounds for this problem. One would be to re-write your source code. (No, that’s not a joke. See Part II of this series.) Another workaround, if you only want to break on the result of a conditional, is to use a conditional breakpoint:

(gdb) delete break
Delete all breakpoints? (y or n) y
(gdb) break *0x00001fc5 if $eax != 0
Breakpoint 1 at 0x1fc5: file returnbreak.c, line 8.
(gdb) info break
Num Type           Disp Enb Address    What
1   breakpoint     keep y   0x00001fc5 in HelloWorld at returnbreak.c:8
	stop only if $eax != 0
(gdb) run
Starting program: /Users/jeff/Desktop/returnbreak 

Breakpoint 1, 0x00001fc5 in HelloWorld () at returnbreak.c:8
8		if (ShouldReturn())
(gdb) c
Continuing.

Program exited normally.

To summarize, if you find that your breakpoints are not getting hit, you now know who to blame. Namely, yourself. It’s almost certain that your Xcode project settings are wrong.

Gruber does a snow job on Snow Leopard

September 5th, 2009

John Gruber holds Apple blameless for shipping Mac OS X 10.6.0 with an outdated version of Adobe Flash Player, a version with known vulnerabilities that have been exploited in the wild. The essence of his absolution is the following timeline: “Adobe released version 10.0.32.18 of Flash on July 30. Snow Leopard went GM on Friday August 7″.

It is true that Adobe released Flash Player 10.0.32.18 to the public on July 30. However, with regard to the version of Flash Player included with Snow Leopard, that date is largely irrelevant. On July 21, Adobe released a security bulletin to the public: “Adobe is aware of reports of a potential vulnerability in Adobe Reader and Acrobat 9.1.2 and Adobe Flash Player 9 and 10.” The next day, there was a public follow-up: “A critical vulnerability exists in the current versions of Flash Player (v9.0.159.0 and v10.0.22.87) for Windows, Macintosh and Linux operating systems, and the authplay.dll component that ships with Adobe Reader and Acrobat v9.x for Windows, Macintosh and UNIX operating systems. This vulnerability (CVE-2009-1862) could cause a crash and potentially allow an attacker to take control of the affected system. There are reports that this vulnerability is being actively exploited in the wild via limited, targeted attacks against Adobe Reader v9 on Windows. We are in the process of developing a fix for the issue, and expect to provide an update for Flash Player v9 and v10 for Windows, Macintosh, and Linux by July 30, 2009″.

It is crucial to note that these were all public releases. Since Apple ships Adobe Flash Player with Mac OS X, Apple and Adobe undoubtedly have a private relationship. One would presume that Adobe privately discloses security vulnerabilities in the OS X version of Flash Player to Apple in advance of public announcements and that Adobe privately provides Apple with versions of Flash Player to test on OS X prior to public release. If these things do not occur privately, then both companies ought to be blamed for failing to follow industry best practices.

I’m not sure where Gruber gets the August 7 date. (Perhaps an email from Phil?) Snow Leopard build 10A432 was seeded to eligible ADC members on August 12. If Apple had already declared build 10A432 the GM before seeding it to developers for testing, that would be completely irresponsible (though sadly, not unprecedented). In any case, if the 10A432 seed had turned up a show-stopping bug, Apple could have un-declared it GM. Is allowing an attacker to take control of a system via a web browser not a show-stopper? Gruber asks, “Should Apple have postponed Snow Leopard for another month?” Despite the rhetorical nature of the question, I’ll answer: maybe they should have. Or at least, they should have postponed it 4 days. At WWDC, Apple told us that Snow Leopard would be released in September. Last time I checked, August 28 is not in September. Even delaying Snow Leopard a month would have allowed Apple to ship on time, in September.

Setting aside these more relevant dates, let’s just accept Gruber’s 8-day window for the sake of argument. “Does anyone really think that Apple should have replaced the single-crashiest piece of software in Mac OS X with a new untested version just eight days before going GM?” Yes, I do. We’re not talking about a major update here — obviously Apple should not switch from Flash 9 to Flash 10 eight days before GM. Apple had already pulled the trigger on Flash 10 for Snow Leopard. When security vulnerabilities come to light, fixes must be released quickly. Eight days of testing the update from Flash Player 10.0.22.87 to 10.0.32.18 really should have been sufficient for Apple. If a critical vulnerability was discovered in Mac OS X, Apple should be able to ship a fix within 8 days, if not sooner. Indeed, there were 8 days between Adobe’s security bulletin and the release of the updated Flash Player.

Apple’s record for shipping timely fixes to security vulnerabilities is poor. For example, Apple ‘distinguished’ themselves by being the only vendor in the world to fail to join the coordinated effort to release a fix for the Kaminsky DNS vulnerability on all platforms on the same day. Instead, Apple took its own sweet time, which was particularly egregious for Mac OS X Server customers. (Though I use OS X on my personal computers, I’m glad my web host does not use OS X on their servers.) I personally was aware of the ‘Safari RSS’ vulnerability for months while it remained unpatched and could have easily exploited any visitor to my web site if I had wanted. Fortunately for you, I’m not malicious. Not criminally malicious, anyway.

Apple hot-swapped Mac OS X 10.5.8

September 1st, 2009

There has been some confusion in the net-o-sphere over the existence of two Mac OS X 10.5.8 builds: 9L30 and 9L31a. I think it’s time to clear up that confusion, now that Max OS X 10.6.0 has been released and nobody cares anymore about 10.5.x.

The Max OS X build version is stored in the following file on your hard drive:

/System/Library/CoreServices/SystemVersion.plist

On my machine, the build is 9L30. I installed 10.5.8, as usual, via the combo updater (because I’m a paranoid superfreak who also repairs permissions and offers sacrifices to Demeter). This was a prudent three days after 10.5.8 was released, though the combo updater was downloaded two days after release.

Lately (in the Holocene epoch), the Mac OS X installers have come in ‘flat’ package format (i.e, “smooth and even; without marked lumps or indentations” or “lacking interest or emotion; dull and lifeless”). This makes them opaque to the casual observer. Fortunately, I am Klondike Kat. The installer .pkg file is actually a xar archive that can be read and extracted with /usr/bin/xar. To extract the package contents into the current working directory:

$ xar -xf /Volumes/Mac\ OS\ X\ Update\ Combined/MacOSXUpdCombo10.5.8.pkg

This gives us an ‘old-style’ (Pleistocene) .pkg file whose contents we can view in Finder. The package contains, among other things, a Payload. Be careful not to ignite it, otherwise you may require intensive care, if not AppleCare. The Payload is a gzip archive, so I slapped a .gz extension on the file and gunzip‘ed it. After extracting that archive, you’re left with … yet another archive. (Apparently the Matryoshka method of software distribution.) The new Payload can be read and extracted with /bin/pax:

$ pax -f Payload -r *SystemVersion.plist

The SystemVersion.plist from my original installer is for build 9L30, but the one from the installer I downloaded today is for build 9L31a. Thus, we have to conclude that Apple ‘hot swapped’ Mac OS X 10.5.8. That is, they switched Mac OS X builds after release without bumping the version number.

Why Apple did this remains a mystery. Usually software developers do it when they discover an issue shortly after release but don’t want to go to the trouble of making a public announcement of a new version. What was the issue, and do those of us who have build 9L30 installed still suffer from the issue? For the answer to those questions, you’ll have to read the release notes. ;-)

Snow Leopard hidden Dock preference

August 28th, 2009

In Mac OS X 10.5 and earlier, clicking and holding on an item in the Dock would bring up a contextual menu for that Dock item. In Mac OS X 10.6, popularly known as Snow Leopard, unpopularly known as Leopard Service Pack 1 or “What am I supposed to do with my Power Mac G5 Quad?”, this behavior has changed. One of the new non-Exchange non-features in Snow Leopard is Dock Exposé. Clicking and holding on an application icon in the Snow Leopard Dock invokes Exposé for that application. This is the same effect you see when pressing the F10 key in Tiger and later.

If for some bizarre reason you prefer the old behavior (you backward, reactionary, Obama-hating Luddite), it is possible to bring it back. This is a Lap Cat Software exclusive — you heard it here first, folks! Launch the Terminal application and enter the following:

defaults write com.apple.Dock show-expose-menus -bool no; killall Dock

You’re welcome. Remember me fondly in your will.

It’s over

August 6th, 2009

I figured I’d cruise, at least through the Spring. However, the wheels on the bus go round and round.

rdar://problem/7125338

I am still master of my domain. Although I need to renew before it expires in three weeks.