TECHNOLOGY & HUMAN RIGHTS

“Killing two birds with one stone?” The Cashless COVID Welfare Payments Aimed at Boosting Consumption

In launching its COVID-19 relief payments scheme, the South Korean government had two goals: providing a safety net for its citizens and boosting consumption for the economy. It therefore provided cashless payments, issuing credit card points rather than cash. However, this had serious implications for the vulnerable.

In May 2020, South Korea’s government distributed its COVID-19 emergency relief payments to all households through cashless channels. Recipients predominantly received points on credit cards rather than cash transfers. From the outset, the government stated explicitly that this universal transfer scheme had two goals: it was not only intended to mitigate the devastating impacts of the pandemic on people’s livelihoods, but also explicitly aimed at simultaneously boosting consumption in the South Korean economy. Providing cash would not necessarily boost consumption as it could be placed in savings accounts. Therefore, credit card points were offered instead to require recipients to spend the relief. But in trying to “kill two birds with one stone” by promoting consumption through the relief program, the government jeopardized the welfare aim of this program.

Once the payouts began, the government boasted that the delivery of the relief funds was timely and efficient. The relief program had been launched based on business agreements with credit card companies for “rapid and smooth” payment, and indeed, it was true that the card-based channel enabled distribution which was much faster than in other countries. Although “offline” applications for the relief program could be made in-person at banks, the scheme was designed around the submission of applications through credit-card companies’ websites or apps. The relief funds were then deposited onto recipients’ credit card or debit card in the form of points—which were separated from normal credit card points—within two days after applying. In September 2021, during the second round of universal relief payments known as the “COVID-19 Win-Win National Relief Fund,” 90% of expected recipients received their payments within 12 days.

Restricting spending to boost spending

However, paying recipients in credit card points meant restricting their access to cash. While low-income households received the relief fund in cash during the first round of COVID-19 relief, they had to apply for the payment in the second round and could only choose among cashless methods which included credit cards and debit cards. To make matters worse, the policy placed constraints on where points could be used, in the name of encouraging consumption and growing the local economy. The points could only be used in designated places, and could not be used to pay for utility bills, repay a mortgage, nor for online shopping. They could not be transferred to others’ bank accounts or withdrawn as cash. Therefore, recipients had no choice but to use their relief funds in certain local restaurants, markets, or clothing stores, etc. If the points had not been used approximately 3-4 months after disbursement, then they were returned to the national treasury. All of these conditions were the outcome of the fact that the policy specifically aimed at boosting consumption.

Jeopardizing the welfare aim

These restrictions had significant repercussions on people in poverty, in two key ways. First, the relief fund failed to fulfill the right to social protection of vulnerable people at risk. As utility bills, telecommunication fees, and even health insurance fees could not be paid with the points, many were left unable to pay for the things they needed to pay for, while much-needed funds remained effectively stranded on the card. What use is a card meant only for restaurants and shops when one is in arrears on utility bills, health insurance fees, and at risk of electricity supply and health insurance benefits being cut off? Those who needed cash immediately sometimes handed their credit cards to other people to use, and then requested payment back in cash below the value. It was also reported that a number of people bought products at stores where relief fund points could be used, and then sold the products at a lower price on the second-hand online market to obtain cash. Although the government warned that it would crack down on such “illegal transactions,” the demand for cash could not be controlled.

Second, the right to housing of vulnerable populations was not sufficiently protected through this scheme. Homeless persons, who needed the most help, were severely affected because the cashless relief funds could not function as a payment method for monthly rent. Homeless people and slice-room dwellers were the group which most strongly agreed that “the COVID-19 relief fund should be distributed in cash” in a survey. Further, given that low-income people spent a higher proportion of their income on rent than those from other social classes, the fact that the relief funds could not be used on rent also significantly affected low-income households. A number of temporary or informal workers who lost their jobs due to the pandemic were on the verge of being pushed into poorer conditions because they could not afford their rent. The relief program could not help these groups cover some of their most urgent expenditures—housing costs—at all.

Boosting consumption can be expected as an indirect effect of government relief funds, but it must not be adopted as a specific goal of such programs. Attempting to achieve this consumption-oriented goal through the relief payments resulted in the scheme’s design imposing limitations on the use of funds, thereby undermining the scheme’s ability to help those in the most extreme need. As the government set boosting consumption as one of the aims of the program and seemingly prioritized it over the welfare aim, the delivery of the payments was devised in an inappropriate way that did not take the most vulnerable into account.

Killing two birds with one stone?

The Korea Development Institute (KDI) found that only about 30% of the first emergency relief funds led to an increase in consumption, while the remaining 70% led to household debt repayment or savings. In the end, it seemed that the cashless relief stipend did not successfully increase consumption, all while it caused the weakening of its social security function.
Such schemes aimed at “killing two birds with one stone” were doomed to fail from the beginning because these two goals come into tension with one another in the program’s design. The consumption aim is likely to harm the welfare aim through pushing for cashless, controlled, and restricted use. The sole purpose of emergency relief funds in a crisis should be to provide assistance for the most vulnerable. Such schemes should be delivered in a way that will best fulfill this aim, they should be focused on providing a safety net, and should be designed from the perspective of right-holders, and not of consumers.

April 19, 2022. Bo Eun Kwon, LLM program, NYU School of Law whose interests include international human rights law, economic and social rights, and digital governance. She has worked at the National Human Rights Commission of Korea.

TECHNOLOGY & HUMAN RIGHTS

Experimental automation in the UK immigration system

The UK government is experimenting with automated immigration systems. The promised benefits of automation are inevitably attractive, but these experiments routinely expose people—including some of the most vulnerable—to unacceptable risks of harm.

In April 2019, The Guardian reported that couples accused of sham marriages were increasingly being subjected to invasive investigations by the Home Office, the UK government body responsible for immigration policy. Couples reported having their wedding ceremonies interrupted to be quizzed about their sex life, being told they were not in a genuine relationship because they were wearing pajamas in bed, and being present while their intimate photos were shared between officials.

The official tactics reported are worrying enough, but it has since come to light through the efforts of a legal charity (the Public Law Project) and investigative journalists that an automated system is largely determining who gets investigated in the first place. An algorithm, hidden from public view, is sorting couples into “pass” and “fail” categories, based on eight unknown criteria.
Couples who “fail” this covert algorithmic test are subjected to intrusive investigations. They must attend an interview and hand over extensive evidence about their relationship, a process which has been described as “insulting” and “grueling.” These investigations can also prevent couples from getting married altogether. If the Home Office decides that a couple has failed to “comply” with an investigation—even if they are in a genuine relationship—the couple is denied a marriage certificate and forced to start the process all over again. One couple was reportedly ruled non-compliant for failing to provide six months of bank statements for an account that had only been open for four months. This makes it difficult for people to plan their weddings and their lives. And the investigation can lead to other immigration enforcement actions, such as visa cancellation, detention, and deportation. In one case, a sham marriage dawn raid led to a man being detained for four months, until the Home Office finally accepted that his relationship was genuine.

We know little about how this automated system operates in practice or its effectiveness in detecting sham marriages. The Home Office refuses to disclose or otherwise explain the eight criteria at the center of the system. There is a real risk that the system is racially discriminatory, however. The criteria were derived from historical data, which may well be skewed against certain nationalities. The Home Office’s own analysis shows that some nationalities, including Bulgarian, Greek, Romanian and Albanian people, receive “fail” ratings more frequently than others.

The sham marriages algorithm is, in many respects, a typical case of the deployment of automation in the UK immigration system. It is not difficult to understand why officials are seeking to automate immigration decision-making. Administering immigration policy is a tough job. Officials are often inexperienced and under pressure to process large volumes of decisions. Each decision will have profound effects for those subjected to it. This is not helped by the dense complexity of, and frequent changes in, immigration law and policy, which can bamboozle even the most hardened administrative lawyer. All of this, of course, takes place in an environment where migration remains one of the most vexed issues on the political agenda. Automation’s promised benefits of greater efficiency, lower costs, and increased consistency are, from the government’s perspective, inevitably attractive.

But in reality, a familiar pattern of risky experimentation and failure is already emerging. It begins with the Home Office deploying a novel automated system with the goal of cheaper, quicker, and more accurate decision-making. There is often little evidence to support the system’s effectiveness in delivering those goals and scant consideration of the risks of harm. Such systems are generally intended to benefit the government or the general, non-migrant population, rather than the people subject to them. When the system goes wrong and harms individuals, the Home Office fails to take adequate steps to address those harms. The justice system—with its principles and procedures developed in response to more traditional forms of public administration—is left to muddle through in trying to provide some form of redress. That redress, even where best efforts are made, is often unsatisfactory.

This is the story we seek to tell in our new book, Experiments in Automating Immigration Systems, through an exploration of three automated immigration systems in the UK: a voice recognition system used to detect fraud in English language testing; an algorithm for identifying “risky” visa applications; and automated decision-making in the process for EU citizens to apply to remain in the UK after Brexit. It is, at its core, a story of risky bureaucratic experimentation that routinely exposes people, including some of the most vulnerable, to unacceptable risks of harm. For example, some of the students caught up in the English language testing scandal were detained and deported, while others had to abandon their studies and fight for years through the courts to prove their innocence. While we focus on the UK experience, this story will no doubt be increasingly familiar in many countries around the world.

It is important to remember, however, that this story is just beginning. While it would be naïve to think that the tensions in public administration can ever be wholly overcome, the government must strive to reap the benefits of automation for all of society, in a way that is sensitive to and mitigates the attendant risks of injustice. That work is, of course, best led by the government itself.

But the collective work of journalists, charities, NGOs, lawyers, researchers, and others will continue to play a crucial role in ensuring, as far as possible, that automated administration is just and fair.

March 14, 2022. Joe Tomlinson and Jack Maxwell.
Dr. Joe Tomlinson is a Senior Lecturer in Public Law at the University of York.
Jack Maxwell is a barrister at the Victorian Bar.

TECHNOLOGY AND HUMAN RIGHTS

U.S. Government must adopt moratorium on mandatory use of biometric technologies in critical sectors, look to evidence abroad, urge human rights experts

As the White House Office of Science and Technology Policy (OSTP) embarks on an initiative to design a ‘Bill of Rights for an AI-Powered World,’ it must begin by immediately imposing a moratorium on the mandatory use of AI-enabled biometrics in critical sectors, such as health, social welfare programs, and education, argue a group of human rights experts at the Digital Welfare State & Human Rights Project (the DWS Project) at the Center for Human Rights and Global Justice at NYU School of Law, and the Institute for Law, Innovation & Technology (iLIT) at Temple University School of Law.

In a 10-page submission responding to OSTP’s Request for Information, the DWS Project and iLIT argue that biometric identification technologies such as facial recognition and fingerprint-based recognition pose existential threats to human rights, democracy, and the rule of law. Drawing on comparative research and consultation with some of the leading international experts on biometrics and human rights, the submission details evidence of some of the concerns raised in countries including Ireland, India, Uganda, and Kenya. It catalogues the often-catastrophic effects of biometric failure, of unwieldly administrative requirements imposed on public services, and the pervasive lack of legal remedies and basic transparency about use of biometrics in government.

“We now have a great deal of evidence about the ways that biometric identification can exclude and discriminate, denying entire groups access to basic social rights,” said Katelyn Cioffi, a Research Scholar at the DWS Project, “Under many biometric identification systems, you can be denied health care, access to education, or even a drivers’ license, if you are not able or willing to authenticate aspects of your identity biometrically.” An AI Bill of Rights that allows for equal enjoyment of rights must learn from comparative examples, the submission argues, and ensure that AI-enabled biometrics do not merely perpetuate systematic discrimination. This means looking beyond frequently-raised concerns about surveillance and privacy, to how biometric technologies affect social rights such as health, social security, education, housing, and employment.

A key factor of success for the initiative will be much-needed legal and regulatory reform across the United States federal system. “This initiative represents an opportunity for the U.S. government to examine the shortcomings of current laws and regulations, including equal protection, civil rights laws, and administrative law,” Laura Bingham, Executive Director of iLIT stated. “The protections that Americans depend on fail to provide the necessary legal tools to defend their rights and safeguard democratic institutions in a society that increasingly relies on digital technologies to make critical decisions.”

The submission also urges the White House to place constraints on the actions of the U.S. government and U.S. companies abroad. “The United States plays a major role in the development and uptake of biometric technologies globally, through its foreign investment, foreign policy, and development aid,” said Victoria Adelmant, a Research Scholar at the DWS Project. “As the government moves to regulate biometric technologies, it must not ignore U.S. companies’ roles in developing, selling, and promoting such technologies abroad, as well as the government’s own actions in spheres such as international development, defense, and migration.”

For the government to mount an effective response to these harms, the experts argue that it must also take heed of parallel efforts of other powerful political actors, including China and the European Union, which are currently attempting to regulate biometric technologies. However, it must also avoid a race to the bottom or jump into a perceived ‘arms race’ with countries like China, by pursuing an increasingly securitized biometric state and allowing the private sector to continue its unfettered ‘self-regulation’ and experimentation. Instead, the U.S. government should focus on acting as a global leader in enabling human rights-sustaining technological innovation.

The submission makes the following recommendations:

1. Impose an immediate moratorium on the use of biometric technologies in critical sectors: biometric identification should never be mandatory in critical sectors such as education, welfare benefits programs, or healthcare.
2. Propose and enact legislation to address the indirect and disparate impact of biometrics.
3. Engage in further review and study of the human rights impacts of biometric technologies as well as of different legal and regulatory approaches.
4. Build a comprehensive legal and regulatory approach that addresses the complex, systemic concerns raised by AI-enabled biometric identification technologies.
5. Ensure that any new laws, regulations, and policies are subject to a democratic, transparent, and open process.
6. Ensure that public education materials and any new laws, regulations, and policies are described and written in clear, non-technical, and easily accessible language.

This post was originally published as a press release on January 17, 2022.

The Digital Welfare State and Human Rights Project at the Center for Human Rights and Global Justice at NYU School of Law aims to investigate systems of social protection and assistance in countries worldwide that are increasingly driven by digital data and technologies.

The Temple University Institute for Law, Innovation & Technology (iLIT) at Beasley School of Law pursues action research, experiential instruction, and advocacy with a mission to deliver equity, bridge academic and practical boundaries, and inform new approaches to technological innovation in the public interest.

TECHNOLOGY AND HUMAN RIGHTS

Response to the White House Office of Science and Technology Policy’s Request for Information on Biometric Identification Technologies

In January 2022, the Digital Welfare State & Human Rights Project team at the Center together with their partners at the Institute for Law, Innovation & Technology (iLIT) at Temple University, Beasley School of Law, submitted expert commentary to the United States White House’s Blueprint for an AI Bill of Rights initiative. 

The White House Office of Science and Technology Policy (OSTP) had embarked on an initiative to design a “Bill of Rights for an AI-Powered World,” and issued a Request for Information on Biometric Identification Technologies. The OSTP asked for input from varied experts to provide information about the scope and extent of the usage of biometric technologies, and to help the OSTP to better understand ‘the stakeholders that are, or may be, impacted by their use or regulation.’ In response to this request, our team submitted a 10-page submission to provide international and comparative information to inform OSTP’s understanding of the social, economic, and political impacts of biometric technologies, in research and regulation. The submission discusses the implications of AI-driven biometric technologies for human rights law, democracy, and the rule of law, and provides information about the ways in which various groups and communities can be negatively impacted by such technologies.

In this submission, we sought especially to draw attention to the importance of learning from other countries’ experiences with biometrics, and to show that the implications of biometric technologies go far beyond the frequently-raised concerns about surveillance and privacy. We therefore provided a range of comparative examples from countries around the world where biometric technologies have been adopted, including within essential services such as social security and housing sectors. We argued that the OSTP, in drafting its upcoming “AI Bill of Rights,” should learn from these comparative examples, to take account of how biometric technologies can affect social rights such as health, social security, education, housing, and employment. The submission also urges the OSTP to place constraints on the actions of the U.S. government and U.S. companies abroad.

This submission fed into the United States White House’s Blueprint for an AI Bill of Rights, released in October 2022. The Blueprint has since laid the groundwork for regulatory efforts to assess, manage, and prevent the risks posed by AI in the United States and abroad, and has been built upon in subsequent policy efforts.

TECHNOLOGY & HUMAN RIGHTS

India’s New National Digital Health Mission: A Trojan Horse for Privatization

Through the national Digital Health ID, India’s Modi government is implementing techno-solutionist and market-based reforms to further entrench the centrality of the private sector in healthcare. This has serious consequences for all Indians, but most of all, for its vulnerable populations.

On August 15, 2021, India’s Prime Minister Narendra Modi launched the National Digital Health Mission (NDHM), under which every Indian citizen is to be provided with a unique digital health ID. This ID will contain patients’ health records—including prescriptions, diagnostic reports, and medical histories—and will enable easy access for both patients and health service providers. The aim of the NDHM is to allow patients to seamlessly switch between health service providers by facilitating their access to patients’ health data and enabling insurance providers to quickly verify and process claims. Accessible registries of health master data will also be created. But this digital health ID program is emblematic of a larger problem in India—the government’s steady withdrawal from healthcare, both as welfare and as a public service.

The digital health ID is a crucial part of Modi’s plans to create a new digital health infrastructure called the National Health Stack. This will form the health component of the existing India Stack, which is defined as “a set of digital public goods” that are intended to make it easy for innovators to introduce digital services in India across different sectors. The India Stack is built on the existing foundational user-base provided by Aadhaar digital ID numbers. A “Unified Health Interface” will be created as a digital platform to manage healthcare-related transactions. It will be administered by the National Health Authority (NHA), which is also responsible for administering the flagship public health insurance scheme, the Ayushman Bharat Pradhan Mantri Jan Arogya Yojana (AB-PMJAY), providing health coverage for around 500 million poor Indians.

The Modi government proclaims that the NDHM and digital health ID will revolutionize the Indian healthcare system through technology-driven solutions. But this glosses over the government’s real motive, which is to incentivize the private sector to participate in and rescue India’s ailing healthcare system. Rather than invest more funds in public health infrastructure, the Indian government has decided to outsource healthcare services to private healthcare providers and insurance companies, using access to vast troves of health data as the proverbial carrot.
Indeed, the benefits of the NDHM for the private healthcare sector are numerous. It will provide valuable, interoperable data in the form of “health registries” which link data silos and act as a “single source of truth” for all healthcare stakeholders. This will enable quicker processing of claims and payments to health service providers. In an op-ed lauding the NDHM, the head of a major Indian hospital chain noted that the NDHM will “reduce administrative burden related to doctor onboarding, regulatory approvals and renewals, and hospital or payer empanelment.”
The government appears to have learned its lessons from the implementation of the AB-PMJAY, which allowed people below the poverty line to purchase healthcare services through state-funded health insurance. Although the scheme included both private and public hospitals, it relied heavily on private hospitals, as public hospitals lacked sufficient facilities. However, not enough private hospitals onboarded because rates were non-competitive as compared to the market, and because the scheme was plagued by long delays in insurance payments and insurance fraud. But, instead of building up public healthcare and reducing dependency on the private sector, the government is eager to fix this problem by providing better incentives to private providers through the NDHM.

Meanwhile, it is unclear what the benefits to the public will be. Digitizing the healthcare system and making it easier for insurance companies to pay private hospitals for services does not solve more urgent and serious problems, such as the lack of healthcare facilities in rural areas. The COVID-19 pandemic saw public hospitals playing a dominant role in treatment and vaccination, while private hospitals took a backseat. Given this, increasing the reliance placed on the private healthcare system through the NDHM is counterintuitive.

This growing reliance on the private sector is also likely to further disadvantage people living in poverty. The lack of suitable government hospitals forces people into private hospitals, and they are often required to pay more than the amount covered by the government-funded AB-PMJAY. Further, India’s National Human Rights Commission has taken the position that denial of care by private service providers is outside its ambit, notwithstanding their enrollment into state-funded insurance schemes like AB-PMJAY. Also, as the digital health ID will enable insurance companies’ access to sensitive health data, they may deny insurance or charge higher premiums to those most in need, thereby further entrenching discrimination and inequalities. Getting coverage with a genetic disorder, for instance, is already extremely difficult in India, something a digital health ID could worsen because insurance companies could access this information, rendering premiums prohibitively expensive for millions who need it. Digitization also renders highly-personal health records susceptible to breaches: such privacy concerns led many persons living with HIV to drop out of treatment programs when antiretroviral therapy centers began collecting Aadhaar details from patients.

Not having a digital health ID could lead to exclusion from vital healthcare. This is not a hypothetical. The government had to issue a clarification that no one should be denied COVID-19 vaccines or oxygen for lack of Aadhaar after numerous concerning reports, including allegations that a patient died after two hospitals demanded Aadhaar details which he did not have.

Nonetheless, plans are speeding ahead as the “usual suspects” of India’s techno-solutionist projects turn their efforts to healthcare. RS Sharma, the ex-Director General of the government agency responsible for Aadhaar, is the current CEO of the NHA. The National Health Stack was reportedly developed in consultation with i-SPIRT, a group of so-called “volunteers” with private sector backgrounds who act as a go-between between the Indian government and the tech sector and played a vital role in embedding Aadhaar in society through private companies. A committee set up to examine the merits of the National Health Stack was headed by another former UIDAI chairman.

Steered by individuals with an endless faith in the power of technology and in the private sector’s entrepreneurial drive to save the Indian government and governance, India is determinedly marching forward with its technology-driven and market-based reforms in public services and welfare. This is all underlined by a heavy tendency towards privatization and is in turn inspired by the private sector. The NDHM, for instance, is guided by the tagline “Think Big, Start Small, Scale Fast,” a business philosophy for start-ups.

Perhaps most concerningly, the neoliberal withdrawal of government from crucial public services to make space for the private sector has resulted in the rationing of those goods and services, with fewer people having access to them. Having a digital health ID is not likely to change this for India’s health sector, and is allowing for this privatization by stealth.

December 14, 2021. Sharngan Aravindakshan, LL.M. program, NYU School of Law; Human Rights Scholar with the Digital Welfare State & Human Rights Project in 2021-22. He previously worked for the Centre for Communication Governance in India.