Predicting G7 performance on compliance
The G7 Compliance Simulator makes it possible to identify which G7 members are the least likely to fulfil their commitments – and commit the necessary resources to help them meet their obligations
For almost 50 years, the G7 has served as an international forum for promoting the liberal and democratic ideals championed by its members. To do so, leaders at their annual summits have collectively made more than 6,000 commitments to tackle issues such as development, climate change, health and financial regulation. However, only 62% of these commitments have been met in full, according to G7 Research Group analysis. For the G7 to succeed in its mission, it is critically important that its members comply with the commitments that their leaders make.
For the first time, research on commitment outcomes conducted by the G7 Research Group has made it possible to produce data-driven estimates of the probability that each member will fulfil the commitments made at G7 summits.
The simulator
Research has shown that the underlying factors behind whether G7 members will comply with summit commitments primarily relate to immutable properties such as the economic position of the member or past compliance with similar commitments. These factors cannot be changed in the short term and thus cannot be leveraged to improve the probability of compliance at the next summit. However, by building a model to predict compliance, it becomes possible to identify which members are least likely to meet their G7 commitments – enabling resources to be devoted to assisting them to fulfil their obligations. This could improve the overall effectiveness of the G7.
The G7 Compliance Simulator is such a model. The tool is freely accessible online (see below) so users can enter relevant information to identify which G7 members may need assistance in meeting their summit obligations.
How it works
Using each G7 member’s commitment outcomes for 655 individual commitments assessed for compliance (n = 5,994), the model fits the impact of various summit and commitment characteristics onto a binomial regression model.
This revealed that several key characteristics of summits and commitments were associated with a significant increase in the probability of a given member complying with a specific G7 commitment (see figure 1). Those characteristics are references to democracy or human rights, holding ministerial meetings before G7 summits, forming official G7 bodies on the relevant issue and hosting the summit.
Similarly, holding the presidency appeared to influence levels of compliance. Compliance levels were higher when the United Kingdom hosted the G7 summit and lower when France and Canada hosted (see figure 2).
There was also variation in the probability of compliance across G7 members. The European Union and United Kingdom were more likely to fulfil commitments; France, Japan, and Italy were less likely to do so (see figure 3).
The probability of compliance varied by the area of focus as well. Compliance was more likely for commitments regarding social policy, international cooperation, information and communication technologies and digitalisation, labour and employment, and energy. Compliance was less likely for commitments on education and gender (see figure 4).
Caveats
Despite the many significant variables, the overall explanatory power of the model is very low. Even when all summit characteristics, members’ properties and commitment features are considered together, only 7.3% of the variance in G7 compliance could be explained (McFadden’s pseudo R2). This suggests that most G7 commitment outcomes may be determined by unknown factors or may be simply random.
Nonetheless, there is hope for increasing G7 effectiveness. All the significant variables examined can be combined into a model to predict future compliance. The binomial logistic regression model can predict compliance in a holdout set with 67% accuracy, and a random forest classifier model trained on the same data can predict compliance with 70% accuracy. Although not perfect, this second model performs much better than chance, enabling potential compliance issues to be detected with the simulation tool as soon as commitments are made so that resources can be directed accordingly.
Conclusion
Increasing G7 effectiveness by leveraging factors associated with higher probabilities of compliance is extremely difficult, as most compliance is likely determined by factors outside the control of the G7. However, by using available data to predict future compliance, it may be possible to direct resources to assist members at higher risk of failing to meet their obligations – thus improving the overall ability of the G7 to achieve its goals.
Note: The compliance simulator can be accessed at https://g7-utoronto.shinyapps.io/compliance-tool. Full data and code are available at https://github.com/rapsoj/g7-compliance
