The Part You Throw Away