I'm planning to write a series of blogs posts about my recent adventures with text views, i.e., NSTextView and UITextView. The current plan is 3 parts. However, if the box office receipts are high enough, I may continue to milk the franchise indefinitely. This post, Part 1, describes a bug in NSTextView and how to work around it.
You might be surprised to learn that text views are not just for text. They also support text attachments, which are arbitrary data or files inserted into the document. If you save a Rich Text document with TextEdit app, it uses the file extension .rtf, but if you drag a file into the document and save, it uses the file extension .rtfd, which is a Rich Text Format Directory. An .rtfd file is a bundle containing an .rtf file along with the text attachments in the document. Text attachments can include any type of data, without restriction. Typically you would include images with the text, but believe it or not, a rich text document can also include movies. Moreover, NSTextView will show the movies inline! If Zawinski's Law is that every program attempts to expand until it can read mail, then Johnson's Law is that every program attempts to expand until it can play video.
In macOS 10.12 Sierra and earlier, NSTextView used QuickTime to show inline movies. In macOS 10.13 High Sierra and later, it switched to AVKit by default. According to the AppKit Release Notes for macOS 10.13, "The built-in media support from NSTextAttachment now covers all file types supported by AVFoundation." Unfortunately, this switch brought with it several problems, one of which I'll discuss here. To demonstrate, I've created a sample Xcode project: LeakyTextMovie. Just download the project, run the app, and follow the app's instructions. After the text view is reset, you should see a "zombie movie" remain in the window. Scary!
According to Apple's NSTextAttachment documentation, "In macOS, the text attachment also uses a cell object conforming to the NSTextAttachmentCell protocol to draw and handle mouse events." Not entirely true! Look at the workaround code:
id <NSTextAttachmentCell> attachmentCell = [(NSTextAttachment *)value attachmentCell];
if (attachmentCell == nil)
[attachments addObject:value];NSTextAttachment does still use a cell to display images, but in macOS 10.13 and later it doesn't use a cell for displaying movies, it uses AVPlayerView. This is our zombie. How did I know it was AVPlayerView? Funny you should ask! First, I possess mad debugging skills. Second, when I paste a movie in the text view I see an "Unable to simultaneously satisfy constraints" warning in the Xcode debugger console that mentions AVPlayerView. Mad skills.
The curious thing about the zombie bug was that it doesn't occur when I select all and delete in the text view instead of calling -[NSTextStorage setAttributedString:]. This was the key to debugging the issue. I needed to discover what causes the AVPlayerView to be removed from the window on delete. I decided to set a breakpoint on -[AVPlayerView dealloc]. The breakpoint hit in an autorelease pool, though, so that was no help. Next I tried -[AVPlayerView viewWillMoveToWindow:], which was much more informative. This breakpoint hit in UIFoundation`-[NSLayoutManager(NSPrivate) _clearTemporaryAttributesForCharacterRange:changeInLength:]. At the break, the code seemed to be checking whether an object responds to the selector removeView and then calling that selector. Thus, I set a new symbolic breakpoint on removeView. Note that you can set a breakpoint on the method name without specifying the class. Bingo! Jackpot! Yahtzee! I found the culprit in UIFoundation`-[NSTextAttachmentViewProvider removeView]. Here's the disassembly:
0x7fff62ead0f0 <+0>: pushq %rbp
0x7fff62ead0f1 <+1>: movq %rsp, %rbp
0x7fff62ead0f4 <+4>: pushq %r15
0x7fff62ead0f6 <+6>: pushq %r14
0x7fff62ead0f8 <+8>: pushq %rbx
0x7fff62ead0f9 <+9>: pushq %rax
0x7fff62ead0fa <+10>: movq %rdi, %r14
0x7fff62ead0fd <+13>: movq 0x3bfbe25c(%rip), %rsi ; "view"
0x7fff62ead104 <+20>: callq *0x3bf80356(%rip) ; (void *)0x00007fff6647be80: objc_msgSend
0x7fff62ead10a <+26>: movq %rax, %rbx
0x7fff62ead10d <+29>: testq %rbx, %rbx
0x7fff62ead110 <+32>: je 0x7fff62ead155 ; <+101>
0x7fff62ead112 <+34>: movq 0x3bfbfac7(%rip), %rsi ; "textAttachment"
0x7fff62ead119 <+41>: movq 0x3bf80340(%rip), %r15 ; (void *)0x00007fff6647be80: objc_msgSend
0x7fff62ead120 <+48>: movq %r14, %rdi
0x7fff62ead123 <+51>: callq *%r15
0x7fff62ead126 <+54>: movq %rax, %r14
0x7fff62ead129 <+57>: movq 0x3bfbdbf8(%rip), %rsi ; "superview"
0x7fff62ead130 <+64>: movq %rbx, %rdi
0x7fff62ead133 <+67>: callq *%r15
0x7fff62ead136 <+70>: movq 0x3bfbf96b(%rip), %rsi ; "detachView:fromParentView:"
0x7fff62ead13d <+77>: movq %r14, %rdi
0x7fff62ead140 <+80>: movq %rbx, %rdx
0x7fff62ead143 <+83>: movq %rax, %rcx
0x7fff62ead146 <+86>: movq %r15, %rax
0x7fff62ead149 <+89>: addq $0x8, %rsp
0x7fff62ead14d <+93>: popq %rbx
0x7fff62ead14e <+94>: popq %r14
0x7fff62ead150 <+96>: popq %r15
0x7fff62ead152 <+98>: popq %rbp
0x7fff62ead153 <+99>: jmpq *%rax
0x7fff62ead155 <+101>: addq $0x8, %rsp
0x7fff62ead159 <+105>: popq %rbx
0x7fff62ead15a <+106>: popq %r14
0x7fff62ead15c <+108>: popq %r15
0x7fff62ead15e <+110>: popq %rbp
0x7fff62ead15f <+111>: retqIt calls -[NSTextAttachment detachView:fromParentView:], which removes the AVPlayerView from the window. So we know how the movie gets removed in the case of select all and delete. Now let's go back and look at our buggy case of -[NSTextStorage setAttributedString:]. In that case, -[NSTextAttachmentViewProvider removeView] gets called as expected. However, -[NSTextAttachment detachView:fromParentView:] does not get called! Why not?
I was puzzled because it does in fact call objc_msgSend with the selector detachView:fromParentView:. But as you ancient language experts may know, an Objective-C message sent to nil does nothing (except return nil). In the buggy case, -[NSTextAttachmentViewProvider textAttachment] returns nil, which means -[NSTextAttachment detachView:fromParentView:] is a NOP. Let's look at the disassembly for -[NSTextAttachmentViewProvider textAttachment]:
0x7fff62ead261 <+0>: pushq %rbp
0x7fff62ead262 <+1>: movq %rsp, %rbp
0x7fff62ead265 <+4>: addq 0x3bfc3da4(%rip), %rdi ; NSTextAttachmentViewProvider._textAttachment
0x7fff62ead26c <+11>: popq %rbp
0x7fff62ead26d <+12>: jmp 0x7fff62f5cc02 ; symbol stub for: objc_loadWeakOh my. It's a weak reference. With Automatic Reference Counting (ARC), a zeroing weak reference automatically becomes nil when the object is deallocated. If we set a breakpoint for -[NSTextAttachment dealloc], we can see that it gets called in setAttributedString: before -[NSTextAttachmentViewProvider removeView], which explains our mystery nil value and consequently, our zombie movie.
If the NSTextAttachment gets deallocated prematurely in our buggy case, why doesn't it get deallocated in the case of select all and delete? Some friends suggested that I try the Xcode memory debugger to find the answer. The memory debugger showed that if I call -[NSTextView delete:], the NSTextAttachment is still held in an autorelease pool when -[NSTextAttachmentViewProvider removeView] gets called. This inspired a workaround for the bug: I just needed to keep a strong reference to the NSTextAttachment around until after -[NSTextStorage setAttributedString:] is done.
We know that the bug occurs when an NSTextAttachment lacks an attachment cell. You can see in my workaround code that I add every such NSTextAttachment to an NSMutableArray, which keeps a strong reference to its objects. The trick is that I specify the lifetime of the array by NS_VALID_UNTIL_END_OF_SCOPE, which guarantees that it will not be deallocated until the end of the block, after setAttributedString: returns. Note that NS_VALID_UNTIL_END_OF_SCOPE must be declared outside the scope of the if (shouldWorkaround) conditional, because the end of the conditional's scope is before setAttributedString: gets called. You'll need to quit and relaunch the sample app in order to test the workaround, because the workaround won't remove any views that had already leaked beforehand.
This is the end of Part 1 of my text adventures. I hope you enjoyed it, because if not, you're really not going to enjoy the subsequent parts.