Here’s a provocative question: Are democracies really always better than dictatorships or other forms of power organization? Does living in a democracy make people happy? If you have a ruler who makes responsible decisions for the good of the people, is it even necessary to involve the people in these decisions?
I have no idea how alignment works. I can well imagine that it might make sense to train artificial intelligence using values that have been determined through democratic consultation with large population groups. But let’s assume that the AI is already perfectly aligned. Could it be that it knows much better than humans themselves what is good for humans? If you ask a small child what it wants, it might say: a mountain of chocolate. But the parents would know very well that this mountain of chocolate is not good for the child. Another question, of course, is under what circumstances one could trust such an AI.
You admit then, that at least initially, you would need a democratic system to train the AIs alignment on. And if human values change gradually over time, as they have so far, this would need to be a regular process of retraining and realigning the AI.
Yes, I agree with you. I would also like to see the process of AI alignment be a democratic process that is regularly adjusted to reflect people’s values.