I think I’ve settled on the latter. Disagreement is maybe best communicated by the absence of an upvote? And downvotes work best when they signal something that is just off base, and while not reportable, is not appreciated at a broad cultural level.
Like if you downvote everything you don’t upvote, it almost doesn’t have an effect because all those things are equally bearing one new downvote and hence don’t change sort precedence with each other.