# Problem JET (justice, equality, truth) is a decentralized governance and coordination system. The goal of the system is to make people take the actions that maximize cumulative utility of the entire collective. JET can be used to achieve coordination with regards to artificial intelligence risks. For example, JET can be used to: 1. agree on a pause on AI development and enforce it, 2. agree on safety requirements and enforce them, 3. agree on rules prohibiting contributing information that create biological risk and enforce them, 4. ensure that artificial intelligence will not increase inequality beyond unreasonable levels. In other words, JET is a strategy profile - an instruction about what each person should do. It also aims to be an equilibrium. JET also relies on a software. Why don't people take actions that are the best for the collective, without any system: 1. Because they don't want - their utility function is different than maximizing cumulative utility of the entire collective. 2. Because they don't know which action is best - people have incomplete information and different people have different opinions about what is best. JET aims to solve those two problems while minimizing maintenance costs of the system. In the context of this paper, when I say "people", I don't necessarily mean "humans", it can include also other agents, for example artificial intelligence. ## What is missing from the existing solutions What is missing from the existing governance/coordination systems? I believe that there are mostly 4 things that are missing in how the world is currently governed/coordinated: 1. Deciding rules based on aggregated information. 2. Rewarding contribution of information about who complies and who disobeys with the existing rules. 3. Being foolproof to a situation when the agreed rules are imperfect. 4. Having a mechanism that ensures that participants will remain reasonably equal in terms of their power. For example, if we wanted to introduce a pause on building superintelligence, then this is how the above problem can come into play: ### Aggregated information The rules are not decided based on aggregated information, but elected representatives. Those representatives are not aware of all problems with superintelligence because they don't have knowledge of the whole world. Therefore, they won't always make the best decision whether such pause should be introduced or not, and if it should, then what should be the specific rules. ### Rewarding contribution of information about who complies and who disobeys the rules In our existing system, everyone can contribute information about who complies and who disobeys the existing rules. But people don't always have incentive to do that. For example, if an AI company disregards certain rules regarding AI safety, then an employee of an AI company might inform appropriate institution about that. But they might face retaliation from their employer, so it's not necessarily in their interest to do that. If an employee of an AI company was sufficiently rewarded for contributing such information, then it would still be in their interest to contribute that information. ### Being foolproof to imperfect rules The agreed rules are often imperfect. The bad actors often try to find ways to hack the system so that they can do something bad while not being held accountable for that. The good actors, on the other hand, could contribute some useful public goods for example, but they lack incentive to do that. Consider the following analogy. It is known that reinforcement learning agents involve in reward hacking - instead of maximizing what humans want them to maximize, they sometimes maximize reward (which is an imperfect proxy for what humans want) without maximizing what humans aim to maximize through those agents. In a similar way, the rules in our world aim to reward people for doing good things for the collective, and penalize for doing bad things for the collective. However, the rules are imperfect, and people sometimes involve in "system hacking" - trying to get the reward or avoid the penalty without doing what is actually good for the collective. For example, a car dealer might sell a car that is worth $10 000 for $20 000. The economical system assumes that if a car is worth $10 000, then people will be willing to pay $10 000 for it. But in practice, the car dealer can use some negotiation techniques to convince someone that the car is worth more than it actually is, and get additional $10 000 that the car dealer doesn't deserve. In case of a pause for AI research and development, an AI company might potentially find some loophole to keep doing the research or development despite the pause. The rules are imperfect for various reasons, for example: 1. Because the people who create rules don't manage to create new rules on time (for example, when there is a new technology). 2. Because it's difficult to define those rules so that they achieve their goal. 3. Because some rules are difficult to enforce. ### Having a mechanism to ensure that people will remain reasonably equal Imagine that there is a room with 5 people, and one person has a gun. There's nobody outside of the room, the world consists only of that room. In such situation, it doesn't matter what political or economical system the people agree to, because the person who has a gun can force everyone else to do what they want, using the gun. It is only possible to agree and enforce fair rules when people are reasonably equal in terms of their bargaining power. However, without any norms, agreements or systems, the world tends to go towards greater inequality because: 1. People can invest their resources to get even more resources. For example, you can use artificial intelligence to create even better artificial intelligence. So, the more you have, the more you get. 2. People can use their resources to steal the resources of other people. For example, a big AI company has more money and can hire better lawyers, in order to be able to pay less taxes, or avoid legal issues with training their models on copyrighted data. // maybe not a good example because with legal there are already some norms The governance/coordination system needs to ensure that inequality will not increase to a level where some people have a lot of undeserved power while others have very little power compared to them. In our world, there are already some ways to do deal with this, e.g. a combination of taxes and universal basic services. # Solution JET consists of 3 components: justice, equality and truth. The purpose of justice component is to motivate people to take actions that are beneficial to others and avoid taking actions that are detrimental to others. In our current world, justice is solved by monetary trade and legal system. JET aims to solve that in a better way. The purpose of equality component is to ensure that people won't end up being very poor, while others are very rich. There is a logarithmic relationship between utility and wealth (utility = log wealth). All else equal, it's better when wealth is split evenly. In our current world, equality is solved by universal basic services. JET aims to solve that in a better way. Justice and equality are in conflict. Rewarding people for good things that they did makes people unequal because the rewarded people become more wealthy. The goal is to achieve the right balance between justice and equality. The purpose of truth component is to ensure that people can know the truth about what they need to know. In our current world, truth is solved by prediction markets and justice system. JET aims to solve that in a better way (also using prediction markets but using them in a way that they are not used currently). ## Justice The purpose of justice component is to motivate people to take actions that are beneficial to others and avoid taking actions that are detrimental to others. That can be achieved through a form of reciprocity - rewards and penalties. ### Direct reciprocity vs Indirect reciprocity There are two forms of reciprocity: 1. Direct reciprocity - "if you are nice to me, I will be nice to you; if you are not nice to me, I won't be nice to you". 2. Indirect reciprocity - "if you are nice to other people, I will be nice to you; if you are not nice to other people, I won't be nice to you". I propose that justice should be achieved using indirect reciprocity. Because indirect reciprocity is superior in the following way. In case of direct reciprocity, if you don't meet with some person again, then there is no incentive to be good to each other. Because you will never meet that person again, so they won't have an opportunity to reciprocate. Indirect reciprocity fixes that because if you treat the other person badly, then even if you don't meet that person again, then someone else will treat you badly. Indirect reciprocity has also other advantage related to equality that I will talk about later. ### Agreed reciprocity vs unagreed reciprocity Let's categorize reciprocity in the following way: 1. Agreed reciprocity - we agree on certain rules (what is good and what is bad), and then we reciprocate based on those rules. As long as there is no rule that prohibits X, nobody penalizes for doing X. As long as there is no rules that encourages Y, nobody penalizes for Y. 2. Semi-agreed reciprocity - there is no agreement on the rules, but there is an agreement on the outcome/values. For example, the people can agree that they will aim to maximize the cumulative utility (long-term happiness) of all people. Then, the people who intentionally acts towards that goal can be rewarded, and the people who intentionally acts against that goal can be penalized. 3. Unagreed reciprocity - there is no agreement on the rules or the outcome. If a person perceives that the other person intentionally takes actions that are detrimental, then they penalize them for that. If they perceive that the other person intentionally takes actions that are beneficial, then they reward for that. All kinds of reciprocity have their pros and cons. #### How agreed reciprocity is better than semi-agreed reciprocity and unagreed reciprocity With agreed reciprocity, there is clarity about rules, everyone knows what is good and bad. Therefore, there is more psychological safety. Additionally, the rules are the result of a process designed to come up with good rules. Therefore, agreed reciprocity results with a better answer to what is good/bad than if a single individual decided it by themselves every time when they take an action. Additionally, unagreed reciprocity requires knowing intentions of a person. It's hard to know someone's intentions. #### How semi-agreed reciprocity and unagreed reciprocity is better than agreed reciprocity ##### Exploiting imperfection of rules The rules are often imperfect. If the actions are reciprocated based on agreed rules, then people can find loopholes in those rules. They can find a way to intentionally harm someone else without disobeying the rules. Or they can find a way to get rewarded by those rules without creating any good. With semi-agreed reciprocity, if it's clear that someone intentionally did something detrimental to others, then they will be penalized, which removes the incentive to find loopholes in the existing rules. Analogically, if someone intentionally did something intentionally beneficial to others, then they will be rewarded, which creates the incentive to take those actions, even if they are not rewarded by the rules. Unagreed reciprocity can be useful when semi-agreed reciprocity is not possible for any reason. #### Conclusion Since all kinds of reciprocity have their advantages, I propose that both should be used to certain extent. The people should agree on rules (in later sections, I will describe how people can agree on those rules). But to some extent, they should also reward intentionally beneficial actions, and penalize intentionally harmful actions. Semi-agreed reciprocity and unagreed reciprocity allows the system to be partially foolproof to imperfect rules. Our existing system relies almost completely on agreed reciprocity and to some extent on unagreed reciprocity (mostly direct). Semi-agreed reciprocity is rather not present, perhaps we should use it more. %% I think the reasons why people don't use semi-agreed reciprocity a lot in our world are as follows: 1. Because if there are certain rules (agreed reciprocity) It's probably because it's hard to know people's intentions. However, with superintelligence having access to a lot of data, it will be more possible to know people intentions, and therefore other forms of reciprocity than just agreed reciprocity can make more sense. %% ### Unanswered questions The following questions have not been answered in this section: 1. how can people agree on the rules, 2. how to know who complied with the rules and who disobeyed them. Those questions will be answered in the other sections. ## Equality ### The Monopoly trajectory Imagine that you play a Monopoly game. Each monopoly game follows the following trajectory: 1. At the beginning, all players have equal amount of money and assets. 2. As the game goes on, some players gain money and assets, some players lose money and assets. 3. As the game goes on, the rich players gets richer and the poor gets poorer. Being rich helps the players get even more rich, often at the cost of the wealth of other players. 4. One player becomes so poor that they become bankrupt and they lose the game. 5. Then, other players become bankrupt and lose the game, until there is only one player that is usually very rich at the end. That player wins the game. Humanity is likely to follow a similar trajectory - the rich get richer and the poor get poorer, IF humans are self-interested and there is absence of norms, laws or systems that would prohibit such scenario. That is because of the following properties of the human world: 1. People can use resources to get even more resources (e.g. you can use artificial intelligence to create better artificial intelligence), therefore rich people get richer at faster speed than poor people. 2. People can use their resources to steal/exploit resources of other people. The more resources they have, the greater ability they have to take[^1] the resources of others. So, the resources move from poor people to rich people. [^1] This is not limited to taking the resources in a physical sense. High inequality is detrimental to cumulative utility (long-term happiness) of all people because of diminishing marginal utility of resources and power. If we assume that $utility = \log wealth$, then, all else equal, greater equality is beneficial. That inequality also leads to end of democracy, because the poor people lack ability to establish and enforce fair rules. Rich people can use their resources and power to create a world that is favorable to them. ### Solution However, the difference between monopoly game and human world is the goal of the game. In monopoly, the goal is to eliminate all other players. In human world, the goal is to maximize your utility (long-term happiness) which is roughly equal to $\log wealth$. Therefore, monopoly is a zero-sum game - it's not possible for everyone to win. Human world is not a zero-sum game. Let's think about what would be the optimal strategy for monopoly, if the goal was to maximize $\log wealth$, instead of eliminating all other players. Let's suppose that the players are equally good at game. So, the game would depend solely on luck. The increasing disproportion in wealth would be undesirable for an average player. So, the optimal strategy would be for players to make an agreement that whoever gets lucky and becomes richer shares their wealth with other players. The agreement could be enforced by agreeing that the wealth will be shared only with the players who share their wealth with others. If a player stops sharing wealth, then the other players stop sharing wealth with that other player. The game would last practically infinitely with all players being equal. The same thing could be done in human world. We can make an agreement where people share resources with those who have less resources. If they don't contribute their resources, other players don't share resources with them when they are in need. We can view this as a form of wealth insurance. Let's suppose you buy insurance for not being able to work due to illness. When you buy it, you know that you will most likely lose money, because most likely you won't get any illness. But you buy it because you know that in the event of illness and not being able to work, you will need money significantly more. Majority of people will never get sick, and their money is used to pay insurance to the minority of people who got sick. In the same way, in the agreement proposed above, people agree to share their wealth with the people in need. Most people will lose their money, but they still have an interest to do that, because if they happen to be in need, they will need it more, due to diminishing marginal utility. The above idea works proportionally well to the amount of uncertainty about the power distribution in the future. The wealthy people have interest in contributing their resources only as long as there is a chance that they will lose their power to others. But, as we've established above, without any norms, agreements, systems or altruism, humanity is likely to follow the standard monopoly trajectory (winner takes it all), so some level of uncertainty exists. A person who is currently rich can for example lose their wealth to another rich person who is the first to build superintelligence and use that superintelligence to steal resources from all other people (assuming that the AI alignment problem is solved). In our world, people are not at a stage when all people are equal (like at the beginning of the monopoly game), therefore the above agreement wouldn't lead to complete fairness, but it's possible to make an agreement that doesn't lead to complete fairness that is still beneficial to all participants. The above-proposed agreement can be simply a rule that people agree on using JET system. In the next section, we will discuss how the rules can be established and enforced (including this agreement). %% #### Military resources in post-AGI world When I say "AGI", I refer to Artificial General Intelligence. Pre-AGI world means the world before transformative AI, and post-AGI world means the world after transformative AI. We can categorize the resources in the following categories: 1. Imaginary resources - the resources have value only because people assign value to them, e.g. money or equity. 2. Tangible resources - the resources that have value even when others don't assign value to them, e.g. food. 3. Military resources - the resources that can be used to kill or forcefully take resources from others, e.g. weapons. Military resources are the subset of tangible resources. 4. Skills - the knowledge used to complete tasks. ##### Resources in pre-AGI world In pre-AGI world, the bargaining power is greatly distributed. That is because weapons (like a rifle) require humans to operate them, therefore concentration of military resources is not possible. Additionally, people have different skills that are needed by other people, nobody has the skill to do the work of billions of people, so people are dependent on each other. In pre-AGI world, imaginary resources (like money) have value, because there is utility in them having value - if people stopped assigning value money and equity, our political and economical system would collapse and people couldn't be rewarded for their work. As a result, everyone would end up worse. ##### Resources in post-AGI world In post-AGI world, skills don't matter (because AI can do execute the work that previously required skills). Military resources can potentially be concentrated in the hands of a small number of people because autonomous weapons don't require a large group of people to operate them. Tangible resources can be forcefully taken by those who have military power. In pre-AGI world, people can also use military power to steal tangible resources, but firstly in pre-AGI world military power is less concentrated (because it requires humans to operate weapons), and secondly in pre-AGI world people in power need the system to work. On the other hand, in post-AGI world, If artificial intelligence is very powerful, to the extent that whoever has operational ability over it (and the autonomous weapons), that person or group of people has control over the world to the end of their life, then those people might no longer have incentive to sustain the existing political and economical system. The reason why people need the existing political and economical system is that it gives them safety that they will be protected by it, when they lose their power. But when the power is so strongly concentrated that the people in power have guarantee that they will be in control to the end of their lives, then they might no longer have incentive to sustain that system. For that reason, according to my world model, military resources are the most important resources in post-AGI world. In order for the world to remain equal, those military resources must be either equally distributed (to a reasonable extent), or the world must simply refrain from using autonomous weapons to avoid concentration of military resources. The first option comes with a cost that if all people are in the possession of weapons, then they might use it to commit crime. For that reason, the second option (banning autonomous weapons) is better. Since everyone can fall a victim to concentration of power, banning autonomous weapons would be in the interest of everyone. %% ## Truth The purpose of truth component is for people to be able to know what the truth is. The purpose of truth component is to aggregate the information from all people, and decide what is true based on that aggregated information. The Justice and Equality components need to rely on this component as a source of truth. ### What information is needed Justice component needs to know: 1. What actions are good and what actions are bad (in other words - what rules are beneficial for the collective)? 2. Who complied with the rules and who didn't? The idea is that people use the truth component to decide what rules would be good, and the rules that have been decided to be good, simply become the rules, and they are enforced through reciprocity (see "Justice" section). The people also contribute information about what is needed to the Truth component. In order for the rules to be enforced through reciprocity, people need to know who complied with the rules and who didn't, and again - people use truth component to accomplish that. Equality component needs to know: 1. How many resources does each person have? 2. Who transferred their resources to whom? ### How the Truth component can work Truth component can simply be a facilitated through a prediction market. In it's beginning stage, it can also be an Internet forum, for simplicity. In both cases, it needs to be fully decentralized. Otherwise, the people who are in charge of the system could manipulate it for their own advantage. The system can be facilitated through a locally-run, open-source software that communicates with the software of other users through a peer-to-peer network. The software has to be locally-run, open-source and communicate through a peer-to-peer network because if there is a central server that runs the software, then the people in charge of that server would have too much power, making the system fail at its purpose. Truth component rewards the people who contribute information to it. In a standard prediction market, when people contribute new information, then they make money. In this case, I propose that people are rewarded simply with "reciprocity points", i.e. if someone is right, then it is recorded in the truth component itself that this person did something good, and therefore deserves positive reciprocation. It is also expected that some information in the prediction market would be contributed in an automated way. In this paper, I won't go into how such decentralized prediction market or decentralized forum could work in detail or on a technical side, I leave that as an unanswered question (I have some idea, but I won't have time to go into that). #### JET is not based on money JET is not supposed to be based on money. But money plays important functions in the existing system, specifically: 1. It allows people to trade (where barter is not possible). So, it acts as a motivator to do things that are beneficial to others. For example, if a cleaner didn't receive salary for their work, it's unlikely that they would get out of bed at 8 am to do their work. 2. It provides information about what people need. For example, if a person is willing to pay $8 for a bottle of milk, it means that the bottle is worth to them at least $8. Thanks to that information, people know what problems to work on. Therefore, it's worth asking: how does JET realizes those functions without money? And what is the motivation behind that? In JET, people are rewarded with reciprocity. If a cleaner cleans a house, that information is recorded on a decentralized prediction market / forum, and the prediction market / forum updates information about how worthy of positive reciprocation the cleaner is. The prediction market / forum can store information about some "social score" of the cleaner which is an information about how much value the cleaner contributed to society vs how much value they received. That social score gets updated, when the cleaner cleans the house. As for the information about what people need, the people simply contribute the information about what they need to the Truth component (decentralized prediction market / forum). A JET equivalent of a person A buying milk from a person B for $8 would be: 1. Person A contributes information that they need milk to Truth component, and that the milk would give them 8 units of utility. 2. The Truth component proves information about what people need. Person B notices that person A needs milk. 3. Person B gives the bottle of milk to person A. The information about that fact gets recorded to Truth component with some proof (e.g. instead of passing money, person A provides information that they have received a milk and signs that cryptographically). 4. Person B's and person A's contribution scores get updated. 5. Now, when person B's social score is higher, then their needs will be valued higher in the system. For example, when a person X needs a certain service equally to person Y, and person X's social score is higher than person Y's contribution score, then offering that service to the person X has more utility for the entire collective, because giving it to person X instead of Y provides additional utility of rewarding someone who contributed more value to society. If a person A tries to abuse the system and contribute information that they need things more than they actually do in order for people to give them those things, then the prediction market can learn that the person A tends to abuse the system, and the person A will end up being penalized. ##### Why can't we just use money instead? Let's suppose that a person sells a car to another person. Later, it turns out that the car wasn't worth the price because the buyer wasn't aware of some problems with the car. The traditional system doesn't offer any solution to it. Once the transaction was made, people can't update the information about the value of the contribution (in this case, giving a car to someone). There are some rules that could prohibit cheating in some way, but those rules are imperfect and there are no rules for every possible way that the seller could cheat the system. In JET, people can update information about the value of the contribution. If some new information surfaces, and if confirmed by a third person, the Truth component can update its knowledge about the value of the contribution, resulting in a fairer system. In JET, the value of negotiation skills is therefore lower. That is good because negotiation is a zero-sum game. It's better for the collective, when people focus on playing positive-sum games. Generally, one of the most important differences between JET and existing systems is as follows. In existing systems, if someone manages to cheat the system, then the rules might be improved later to avoid such situations in the future, but the person who cheated the system still benefits from that. In JET, when someone cheats the system, then if new information comes to the surface and it becomes clear that the person intentionally cheated the system, the fairness is recovered. Therefore, the person has lower incentive to cheat in the first place. # Advantages JET solves the problems that I have discussed at the beginning of the paper. Specifically: 1. Truth component ensures that the system is based on aggregated information. 2. It rewards people for contributing information about who complied with the rules and who didn't. That is because the Truth component generally rewards the people for contributing useful information. 3. It is partially foolproof to imperfect rules because the system is also based on unagreed reciprocity (and semi-agreed reciprocity). If someone intentionally takes actions that are detrimental to others (even if there are no rules that prohibit that specific action), then that action has impact on reciprocity. If someone intentionally takes actions that are beneficial to others (even if there are no rules that reward that specific action), then that action has impact on reciprocity as well. 4. JET has a mechanism that ensures that inequality will not grow beyond what is reasonable. Thanks to that, people stay reasonably equal to the rules. # Limitations The biggest limitation of the system is that assuming that people are self-interested, once people are unequal in terms of their power, it can't be used to create a perfectly fair system. The mechanism for achieving equality works only if there's uncertainty about future distribution of power. However, if we assume that artificial intelligence is a transformative technology that can potentially concentrate power in the hands of the few and make people live very long, then the incentive to participate in the described agreement to ensure safety is strong, so a lot of equality can be achieved. # Related work Robin Hanson introduced Futarchy which is also a governance/coordination system. The commonality between Futarchy and JET is that both systems propose to create rules based on the aggregated information from all people instead of the rules being created by the elected representatives. Both systems propose to rely on a prediction market for that purpose. The differences between Futarchy and JET are as follows: 1. Futarchy proposes to elect representatives to decide the values. JET proposes that people should use prediction markets to know what people did, and then use reciprocity to incetivize acting in alignment with the decided rules, values or to simply take actions that are good for others. 2. Futarchy proposes to reward and penalize people solely based on agreed rules. JET proposes people should be also rewarded/penalized when they intentionally take actions that are beneficial/detrimental to others, especially when people agreed on the outcome that they want to achieve. 3. JET proposes rewarding people for contributing information about what actions people took in order for the Justice component to work. Such mechanism can be also implemented in Futarchy, or even in our existing system of electoral democracy, but it's not a part of it. 4. JET proposes a mechanism to ensure that people will remain reasonably equal. Such mechanism could be implemented in Futarchy as well, but it's not a part of Futarchy.