When the world’s most high-profile system of ranking sustainability depends on companies’ own declarations, its validity must be called into question
The annual update of the Dow Jones Sustainability Indices (DJSI) took place in September. Among the winners that gained a place or returned to the DJSI world index were Bank of America (which had been de-listed in 2014), French bank Société Générale, Rolls-Royce and mining giant BHP Billiton. Among the losers culled from the index were Cisco Systems, PepsiCo and Royal Bank of Canada.
Guido Giese, head of indices for sustainability analyst RobecoSAM, which provides the data for the DJSI, says the indices are the “gold standard for corporate sustainability”. This might be an exaggeration. The DJSI measures sustainability in relative, rather than absolute, terms. Companies are evaluated in comparison with their sector peers based on sector-specific questionnaires. Therefore, some companies in the DJSI might be leagues ahead of their direct competitors on sustainability, but well behind even the third or fourth best from other sectors.
The DJSI is focused on the world's largest 3,400 companies by market capitalisation and is derived to a great extent from the responses companies provide to questionnaires. This raises the possibility that the leading DJSI companies are not necessarily the best on sustainability, but might be the best at filling in questionnaires. Daniel Johnson, research hub manager for the non-profit Institute for Business Ethics says: “Any index has limitations. There is often a gap between what a company says it is doing and what it is actually doing.”
Theory v practice
It certainly seems that companies are more sustainable if judged by their policies and codes of conduct than if judged by their operational performance. According to RobecoSAM, the DJSI Corporate Sustainability Assessment (CSA) criteria on which companies scored best in 2015 were compliance systems, corporate governance and environmental management systems. The lowest CSA scores were earned when it came to actually behaving sustainably: for “operational eco-efficiency”, human capital development and corporate citizenship.
The gap between words and action also showed up in the DJSI this year over Volkswagen. When the 2015 DJSI update was published on 10 September, Volkswagen was ranked the industry leader for “automobiles and components”. By late September, after revelations about Volkswagen's cheating in air pollution emissions tests, the company had been ejected from the DJSI.
This puts a question mark over the DJSI research methodology. In addition to questionnaires, the calculation of sustainability scores involves analysis of “publicly available information from consumer organisations, NGOs, governments or international organisations”, according to RobecoSAM. “They do take wider stakeholder considerations into account,” says Johnson.
But the media and stakeholder analysis must have missed the many reports issued over several years by groups such as Transport & Environment and the International Council on Clean Transportation highlighting auto-industry gaming of emissions tests. Volkswagen might have been caught using clandestine software to cheat tests, but the sustainability performance of many carmakers is called into question by large gaps between air pollution scores achieved in laboratories and emissions in the real world.
And Volkswagen is not the first. In November 2014, Hyundai and Kia paid a $100m penalty in the US for fixing greenhouse gas emissions tests in order to achieve unrealistically good results. But Hyundai remains on the DJSI World index, while Kia is on the DJSI Korea index.
S&P Dow Jones Indices says that after Volkswagen, it wants to “closely monitor the automotive industry before making any further decisions on this sector”. Overall, though, it does not believe the DJSI methodology needs to change. “We depend on the information we receive from a company as well as the publicly available information we are able to gather ourselves,” says spokeswoman Haw-Yan Man. “If the information is wrong, the problem is with the information and not with our methodology.”DJSI Volkswagen Hyundai emissions cars