• 0 Posts
  • 49 Comments
Joined 1 year ago
cake
Cake day: June 29th, 2023

help-circle





  • Probably not. I mean the whole movie was a psudo-satire about the war on terror. But there were lots of internal contradictions that made the second half age terribly. The main message of the second half was that Iraq definitely does have WMDs and America might be annoying and brash and dumb but someone has to invade the middle east to fight evil!

    A sequel would either have to admit that there were no WMDs and the war on terror was open colonialism, or ignore the entire message of the first movie and pretend it never happened, or double down on saying actually it’s cool for the US to invade other countries.















  • People sometimes act like the models can only reproduce their training data, which is what I’m saying is wrong. They do generalise.

    During training the models are trained to predict the next word, but after training the network is always effectively interpolating between the training examples it has memorised. But this interpolation doesn’t happen in text space but in a very high dimensional abstract semantic representation space, a ‘concept space’.

    Now imagine that you have memorised two paragraphs that occupy two points in concept space. And then you interpolate between them. This gives you a new point, potentially unseen during training, a new concept, that is in some ways analogous to the two paragraphs you memorised, but still fundamentally different, and potentially novel.