Apple's documentation for the class NSURLSession (or URLSession for Swift coders) contains a warning marked "Important":
The session object keeps a strong reference to the delegate until your app exits or explicitly invalidates the session. If you don’t invalidate the session, your app leaks memory until the app terminates.
This warning doesn't tell the whole story, though. It's incomplete. What it doesn't tell you is that if you don't invalidate the session (via finishTasksAndInvalidate or invalidateAndCancel), then the internet connection created by the session remains open until the app terminates, even after the delegate method URLSession:task:didCompleteWithError: has been called, and even after the app's code no longer has a strong reference to the session. It's more than just a potential memory leak.
I discovered this connection leak the same way I discover a lot of things about how macOS works: by accident, with Little Snitch. I confirmed it with a sample Xcode project and packet traces. (Note that in my sample app the NSURLSession delegate is also the NSApplication delegate, which is expected to remain instantiated for the lifetime of an app, so there's no worry about a memory leak.)
My sample app runs two consecutive ephemeral sessions with https://www.reddit.com, a URL chosen mostly arbitrarily. I did want something other than apple.com in order to stand out in packet traces that can also include system processes phoning home to Cupertino. I ran two sessions to see whether NSURLSession would reuse the connection, but it did not; the two ports were different.
The packet traces show that when my app calls invalidateAndCancel after the URLSession:task:didCompleteWithError: delegate method, my Mac immediately sends a TCP FIN packet to Reddit, closing the connection. On the other hand, if invalidateAndCancel isn't called, then… nothing happens. There's no further traffic on the connection, but it remains open. Indeed, both connections from both sessions remain open. This can be verified for example with the netstat command-line tool.
My Mac finally sends TCP FIN packets on both connections, simultaneously, after applicationWillTerminate: is called, so it becomes obvious that the connections were leaked for the lifetime of the app process.
The NSURLSession API seems peculiar, because you would expect URLSession:task:didCompleteWithError: to be, you know, the end. Shouldn't you be able to freely (pun intended) dispose of the connection at that point? The reality, however, is that you need to invalidate every used session. So now I know, and now you know.
You can see in Xcode's "View Memory Graph Hierarchy" that the system framework via URLProtocol still has references to the two NSURLSession objects even though the app no longer has references to them. The docs do not mention this second memory leak, only the first memory leak of the NSURLSession keeping a reference to its delegate (in this case AppDelegate).