Centre for Internet and Society

CIS Submission to UN High Level Panel on Digital Co-operation

Aayush Rathi, Ambika Tandon, Arindrajit Basu and Elonnai Hickok — 2019-02-19T01:41:35Z

The High-level Panel on Digital Cooperation was convened by the UN Secretary-General to advance proposals to strengthen cooperation in the digital space among Governments, the private sector, civil society, international organizations, academia, the technical community and other relevant stakeholders. The Panel issued a call for input that called for responses to various questions. CIS responded to the call for inputs.

Download the submission here

For more details visit https://cis-india.org/internet-governance/blog/cis-submission-to-un-high-level-panel-on-digital-co-operation

Participation in the meeting of BIS LITD 17

Admin — 2019-03-03T06:12:01Z

Gurshabad Grover participated in the fifteenth meeting of the Information Systems Security and Biometrics Sectional Committee (LITD 17) of the Bureau of Indian Standards (BIS), which was conducted online on February 26.

Some of the things we discussed included:

Participation of committee members at the ISO level in SC 27 'IT Security Techniques' working groups.
Update from the last SC 27 working group meetings (I updated the committee with some standards I was tracking and my participation as co-rapporteur in the 'Impact of AI on Privacy' study period).
Participation in the next SC 27 working group meetings, which will be held in April (where I will be participating in WG 1 'Information Security Management Systems' and WG 5 'Identity management and privacy technologies' meetings).

For more details visit https://cis-india.org/internet-governance/news/participation-in-the-meeting-of-bis-litd-17

Nullcon Security Conference

Admin — 2019-03-07T14:40:11Z

On March 1 and 2, 2019, Karan Saini attended the Nullcon Security Conference organized by Nullcon at Holiday Inn Resort, Mobor Beach, Cavelossim, Salcette, Goa.

The schedule of the event can be accessed here. Videos of the talks can be accessed here. The event was:attended by:

Security Practitioners (Analysts, Testers, Developers, Cryptographers, Hackers)
Security Executives (CISOs, CXOs)
Business Developers and Venture Capitalists (Presidents, Directors, VPs, Consultants)
Vendor Companies and Sponsors (Hardware, Software, Services)
Career Seekers and Recruiters (Seasoned Veterans, Students, Expanding Companies
Academia (Professors, Students)

The nullcon conference is a unique platform for security companies/evangelists to showcase their research and technology. Nullcon hosts Prototype, Exhibition, Trainings, Free Workshops, null Job Fair at the conference. It is an integrated and structured platform which caters to the needs of IT Security industry at large in a comprehensive way.

For more details visit https://cis-india.org/internet-governance/news/nullcon-security-conference

Seminar on “Evolution of communication: Social Media & Beyond”

Admin — 2019-03-07T14:52:09Z

Sunil Abraham will be a speaker at this event organized by TRAI on March 15 at Hotel Radisson Blu GRT, Near Airport, Chennai. Sunil will be speaking on How should Internet Giants- Social Media, Search engines and ad tech be Regulated.

Click to view the agenda.

For more details visit https://cis-india.org/internet-governance/news/seminar-on-201cevolution-of-communication-social-media-beyond201d

DSCI-Infosys Roundtable

Admin — 2019-04-05T02:06:00Z

Sunil Abraham participated in this meeting organized by Infosys in Bangalore on March 25, 2019 as a speaker.

AGENDA:

10:00-10:15 AM	Opening Remarks: Infosys Context Setting: DSCI and Infosys
10:15- 11:00 AM	Elements shaping Data Economy § Digitization: Personalization, Experience, Productivity & Possibilities § Global Internet Platforms: Transforming B2C and B2B § Phantomization of Technology & Business Models § Changing nature of Deliveries: value driven, subscription based and platform based § Product Economy: Data-centric Designs, Start-ups and Unicorn, § IOT and Industrialisation 4.0: Next generation service & business lines § Data flow and how it’s shaping trade of goods and services § Role of data in delivering the public service and improving public order § Artificial Intelligence: at specific product/service level and its ramification to industrial and national economy § Technology: role of data in developing next generation tech platforms Discussion Facilitation: DSCI and Infosys
11:00- 11:45 AM	Tech’s Dilemmas § Scale and reach of BigTech: Industrial Capitalism versus Internet Capitalism § Competition § Influence on personal, social, transactional, economic and political life § Stressed relations with values of modern value system § Ethical issues: human rights, social harmony, public space decency, health electoral processes, information warfare... § Data Privacy § Tech’s response: Locking down of data, editorial/ censorship controls... § Challenges of law enforcement, fraud management and supervision § Relevance to national security objectives ...... § Principles of Responsible Innovation § Ideas under discussion/ experimentation Discussion Facilitation: DSCI and Infosys
11:45-12:15 AM	Shaping Data Economy § Structures and approaches: state controlled, private sector led, decentralized § Directions: legal/ policy, innovation, investments, architectures (like India Stack), § Searching the role of liberal economic principles § Open architectures and open data ecosystem § Positions, Obligations, Burdens and Liabilities for protecting rights, creating level playing field, ensuring competition... § Regulatory approaches: establishing supervisory controls § National security: Interventions, mandates and cooperation Discussion Facilitation: DSCI and Infosys
12:15 to 12:30 PM	Discussion Summary
12:30 PM onwards	Lunch

For more details visit https://cis-india.org/internet-governance/news/dsci-infosys-roundtable

Data Privacy and Citizen's Rights' Symposium Report

Admin — 2019-04-05T02:24:09Z

The Technology Law Forum at the National Academy of Legal Studies and Research (NALSAR) has published the Report on Data Privacy and Citizen's Rights' Symposium.

This report is a compilation of all the speakers' speeches during the panel discussion. Shweta Mohandas was one of the eight speakers at the panel and the excerpts from her presentation has also been covered in this report. Click to read more

For more details visit https://cis-india.org/internet-governance/news/data-privacy-and-citizens-rights-symposium-report

Debate: Five Aadhaar Myths that Don’t Stand Up to Scrutiny

praskrishna — 2016-04-01T15:48:17Z

We need to reboot the Aadhaar debate by asking why we want to create a centralised biometric database of Indian residents in the first place.

The article by Reetika Khera was published in the Wire on March 23, 2016.

A recent article, ‘Identification simplified, myths busted’, by Piyush Peshwani and Bhuwan Joshi (hereafter, Peshwani & Joshi) makes some questionable claims about the UID project. Peshwani & Joshi’s strategy appears to be to ignore those questions to which they do not have an answer (e.g., that Aadhaar is mostly redundant as far as NREGA, PDS, etc., are concerned). For others, they cherry-pick ‘facts’ without acknowledging the debates surrounding those facts. Here is a selection.

#1: To get Aadhaar, you need a Proof of ID (PoID) and Proof of address (PoA)

Peshwani & Joshi: “For many, Aadhaar is perhaps the first document of their existence – a robust proof of their identity and address that can be verified online. No more closed doors for them!”

Peshwani & Joshi: “The Demographic Data Standards and Verification Procedures committee prescribes a list of valid 18 proof of identity and 33 valid proof of address documents for getting an Aadhaar.”

Fact: In fact, 99.97% of those who have Aadhaar, used PoID and PoA to get it. For those who have neither, there is an “introducer system”, but according to a reply to an RTI request, only 0.03% of those who have the Aadhaar number used this route.

As far as closed doors are concerned, Aadhaar does not guarantee any benefits: work through NREGA, widow or old-age pensions or PDS rations. There are separate eligibility conditions for those programmes which continue to apply.

#2 On costs

Peshwani & Joshi: “Does it justify the cost? Yes, absolutely, according to the World Bank, which said the initiative is estimated to be saving the Indian government about $1 billion annually by thwarting corruption, even as it underlined that digital technologies promote inclusion, efficiency and innovation.”

Fact: Savings due to the use of Aadhaar have been disputed. The government has claimed it has saved Rs. 14,672 crore on LPG subsidies due to Aadhaar while they are likely lower – by a factor of 100 (see Business Standard or Wall Street Journal).

Peshwani & Joshi: “Even before the World Bank’s endorsement of Aadhaar, the Delhi-based National Institute of Public Finance and Policy (NIPFP) conducted a detailed cost-analysis study on Aadhaar in 2012… the study found that the Aadhaar project would yield an internal rate of return in real terms of 52.85% to the government.”

Fact: The NIPFP cost-benefit was based on unrealistic assumptions – e.g., estimates of leakages that Aadhaar could plug were available for only two out of seven schemes; for the rest, they assumed leakage rates which are termed ‘conservative’, but are actually not.

In their response, the NIPFP team admitted that “a full-fledged cost benefit analysis of Aadhaar is difficult” because “many gains from Aadhaar are difficult to quantify because they are intangible” and, “even if in specific schemes there may be tangible benefits, the information available on those schemes does not permit a precise quantification of those benefits.”

They went on to say that “The study has steered away from relying exclusively on analyses of isolated and small sample sets”. What evidence did the NIPFP study rely on? “For ASHAs, Janani Suraksha Yojana and scholarships, no analysis, large or small has been used. For the Indira Awaas Yojana, the three analyses relied on exclusively are a Times of India news report, a press release based on a discussion in Parliament and a “Scheme Brief” by the Institute for Financial Management and Research (IFMR). Interestingly, the corruption estimate in the IFMR brief cross-refers to the Times of India article (apart from a CAG report)!” (Khera, 2013)

#3 De-duplication

Peshwani & Joshi: “Aadhaar means no fake, ghost or duplicate beneficiaries. Double-dipping will become more and more difficult with Aadhaar, a number that is well de-duplicated with the use of biometrics.”

Fact: De-duplication is one possible contribution of Aadhaar – but that needs biometrics, not a centralised biometric database. Local biometrics (used extensively in Andhra Pradesh before UID) mean that biometric data is stored by the concerned government department or on the local e-POS machine’s memory chip. It has the advantage that connectivity is not required (you are authenticated by the machine), errors and corrections can be correctly locally, making it more practical. The distinction between a local and centralised database is important (see #5 below).

Further, no one has a reliable estimate of the duplication problem. Two government estimates of duplicates exist: the Dhande committee for LPG (2%) and in NREGA job cards from the Government of Andhra Pradesh (also 2%).

#4 Exclusion

Peshwani & Joshi: “As far as exclusion in delivery of other services due to biometric authentication accuracy is concerned, it is important to go beyond scratching the surface.”

Fact: When the PDS was integrated with Aadhaar: “The Andhra Pradesh Food and Civil Supplies Corporation found that…nearly one-fifth ration card holders did not buy their ration.” Further, “When the government delved deeper in the issue, it was found that out of the 790 cases interviewed for the study, 400 reported exclusion. Out of the excluded cases, 290 were due to fingerprint mismatch and 93 were because of Aadhaar card mismatch. The remaining 17 cases were due to failure of E-PoS.” More here.

Moreover, Peshwani & Joshi pick one definition of ‘exclusion’ (due to biometric failure) when in fact, exclusion has a broader meaning. For instance, “In Chitradurga (Karnataka), Rs.100-150 million in wages from 2014-15 were held up for a year. When payments were being processed, their job cards could not be traced in NREGAsoft. Upon enquiry, the district administration learnt field staff had deleted them to achieve ‘100% Aadhaar-seeding’.”

#5 Profiling and privacy violations

Peshwani & Joshi: “A prominent criticism of Aadhaar is that it ‘profiles’ people.” …“Most of us have one or more identity/address documents, such as a passport, ration card, PAN card, driving licence, vehicle registration documents or a voter ID card. The government departments managing these already have our data. Aadhaar is no different. We give our data to banks, to insurance companies and to telecom companies for accounts, policies and mobile connections.”

Fact: That’s like saying BJP can be more corrupt because the Congress was corrupt. Instead we need to engage more seriously with the work of Sunil Abraham, Amber Sinha and others at the Centre of Internet and Society. There are crucial differences between Aadhaar and Social Security Number in the US, see this. Malavika Jayaram listed the UID project among a slew of “big brother” projects facilitating mass surveillance in India.

Conclusion

The debate on UID tends to begin with the premise that Aadhaar is necessary for ‘good governance’. Those claims of the UIDAI have long been demolished. In a nutshell, Aadhaar cannot help identify the poor, its possession does not guarantee inclusion into government social welfare (go to #1). It cannot reduce PDS or NREGA corruption as claimed in their early documents. Thankfully, PDS–NREGA corruption has been on the decline without Aadhaar – more needs to be done. (More details? Try this and this.)

Bihar shows how much corruption in the PDS can be reduced without Aadhaar. Credit: Reetika Khera

Aadhaar is not required for portability of benefits or for cash transfers. Cash transfers need bank accounts. To get a bank account, you need a proof of ID and a proof of address (go to #1). Aadhaar can help de-duplicate, but so can local biometrics (go to #3). We need to “reboot” the Aadhaar debate, starting on the right terms – why exactly do we need to create a centralised biometric database of Indian residents?

For more details visit https://cis-india.org/internet-governance/news/the-wire-march-23-2016-reetika-khera-debate-five-aadhaar-myths-that-dont-stand-up-to-scrutiny

Pratap Vikram Singh - Why Aadhaar is Baseless?

praskrishna — 2016-04-02T05:31:30Z

This article by Pratap Vikram Singh, Governance Now, discusses the problems emerging out of the UIDAI project due to its lack of mechanisms for informed and granular consent, and for seeking recourse in the case of denial of service. The article quotes Sumandro Chattapadhyay and mentions Hans Varghese Mathew's work on the biometric basis of UIDAI. It was written before the Aadhaar bill was passed in Lok Sabha.

Cross-posted from Governance Now.

It was no less than a roller-coaster ride for Aadhaar, a programme formulated by the UPA government to assign a 12-digit unique number to every Indian resident. From the time it came into being in 2009, Aadhaar drew a volley of criticism, thanks to the misgivings and apprehensions that various critics and civil society organisations had. It was criticised for lack of a clear purpose, degree of effectiveness and absence of a privacy law and was virtually thrown into the bin by a parliamentary panel headed by BJP’s Yashwant Sinha in December 2011.

When the finance minister Arun Jaitley, in his budget speech, announced that the government would introduce the Aadhaar bill during the budget session, expectations were already set high. The bill, giving statutory backing to the unique identification authority of India (UIDAI), the implementing authority, was passed by the Lok Sabha on March 11. While the privacy and voluntary versus mandatory provisions are under the consideration of the supreme court, the bill makes way for linking Aadhaar with all government subsidies, benefits and services. The law on Aadhaar, former UIIDAI chairman Nandan Nilekani wrote in the Indian Express, will help the government in going paperless, presence-less and cashless. The legislation, however, fails to deliver on several counts.

However, prior to evaluating the bill (yet to be passed by the Rajya Sabha at the time of this writing though it is a money bill), let us take a look at its major aspects. For those, who always wondered whether Aadhaar is mandatory or voluntary, the bill 2016 makes it mandatory to avail subsidy, benefit or a service from the government.

The bill has provisions related to information security and confidentiality (section 28) which not only extend to employees of the UIDAI but also consultants and external agencies working with the authority.

The proposed law restricts information sharing. It bars UIDAI from sharing core biometric information – the bill defines it as fingerprints and iris scan – with “anyone for any reason whatsoever” or “used for any purpose other than generation of Aadhaar numbers and authentication under this Act”. The section 32 of the bill entitles Aadhaar number holders to access her or his authentication record. It also bars the authority from collecting, keeping or maintaining information about the purpose of authentication.

Odd Drives the Bill

While the intent is clear and is aimed at streamlining welfare schemes to ensure it reaches the bottom of the pyramid, cutting through the long chain of pilferage and subversion, the bill, however, has several shortcomings. To begin with, the government should not have taken the money bill route to pass the legislation – tactfully avoiding any conclusive discussion and debate in the Rajya Sabha, where it is in minority.

The bill assumes that the technology and the biometric system used by the UIDAI are flawless and it doesn’t provide any recourse in case of denial of a service. “If your fingerprint is not matching and you lose out on service, then what is the alternative mechanism you have,” asks Sumandro Chattapadhyay, research director, centre for internet and society (CIS). The bill doesn’t provide for recourse. “What if the scanning machine fails? What if the identifiers of two people match?”

Based on experiments conducted in the initial days of the Aadhaar programme, Hans Verghese Mathews, another CIS researcher, did a study on the probability of matching of identifiers of two persons. “For the current population of 1.2 billion the expected proportion of duplicands (users whose identifiers match) is 1/121, a ratio which is far too high,” Mathews wrote in the Economic and Political Weekly in February.

“It is like putting the technology in a black box – which can’t be reviewed,” says Chattapadhyay. The bill doesn’t talk about setting up an independent body to review the logs and keep an eye on wrong and duplicate matches.

Who Defines National Security?

According to public policy experts, it is an attempt to seek “minimal legitimacy” from parliament and further adds to the unbridled power of the executive.

Although the bill restricts information sharing in section 29, sections 33 and 48 provide exemption in cases of national security and public emergency, respectively. The legislation, nevertheless, doesn’t elaborate on what constitutes national security and public emergency, leaving it to the executives. The section 33 reads: “Nothing contained in… shall apply in respect of any disclosure of information, including identity information or authentication records, made in the interest of national security….”

Similarly, section 48 states that if, at any time, the central government is of the opinion that a public emergency exists, “the central government may, by notification, supersede the Authority for such period, not exceeding six months, as may be specified in the notification and appoint a person or persons as the president may direct to exercise powers and discharge functions under this Act”.

Says Jayati Ghosh, professor, centre for economic studies and planning, Jawaharlal Nehru University, “National security is a very opaque term. Who decides what national security is? Today, the whole JNU is being projected as a threat to national security.” Swagato Sarkar, associate professor and executive director, Jindal school of government and public policy, OP Jindal Global University, says, “The bill has provisions for oversight on the use of Aadhaar, but then it suspends those provisions in case of emergency in the later sections, giving the state the power to use biometric information for whatever it deems fit.”

Sarkar adds, “It seems the bill is simply an instrument for seeking minimum legitimacy from parliament. The bill tries to address the concern of privacy minimally and it hardly serves any purpose.” He believes that there is a need to define the broader contours of democratic control of the state and reassess the changing state-citizen relationship, instead of rejecting the whole idea on the basis of surveillance and privacy. In other words, there is a need for strong parliamentary oversight, and that the Aadhaar related matters shouldn’t be completely delegated to the executive.

In its recommendations on formulating Privacy Act, the justice AP Shah committee in 2012 provided for establishing the office of privacy commissioner at the regional and central levels, defining the role of self-regulating organisations and co-regulation, and creating a system of complaints and redressal for aggrieved individuals. Since the country still doesn’t have any legislation on privacy, people are left on their own in case of an infringement or violation of privacy. Moreover, section 47 states, “No court shall take cognizance of any offence punishable under this Act, save on a complaint made by the Authority or any officer or person authorised by it.”

In its report, the parliamentary committee headed by Yashwant Sinha notes that “enactment of national data protection law… is a prerequisite for any law that deals with large scale collection of information from individuals and its linkages across separate databases”. The committee notes that in absence of data protection legislation, it would be difficult to deal with issues of access, misuse of personal information, surveillance, profiling, linking and matching of databases and securing confidentiality of information.

Subsidy-Aadhaar Linkage

The Sinha committee also takes a cautious view of the role of Aadhaar in curbing leakages in subsidy distribution, as beneficiary identification is done by states. It notes, “Even if the Aadhaar number links entitlements to targeted beneficiaries, it may not even ensure that beneficiaries have been correctly identified. Thus, the present problem of proper identification would persist.”

According to Ghosh, the biggest danger in using Aadhaar for social welfare programmes is that the fingerprints of the rural working class is not always in good shape and hence Aadhaar will not be the best way of identification. “If I am misidentified, I can go to so many places for recourse. But what if a labourer in a remote Jharkhand village is misidentified? Where and whether he would go?” the economist asks. Besides, the bill doesn’t limit the use of Aadhaar and defines areas where it can be used. Section 57 says that the law will not prevent the use of Aadhaar number for establishing the identity of an individual for any purpose, “whether by the state or anybody corporate or person, pursuant to any law, for the time being in force or any contract to this effect.”

According to a PRS Legislative review, since the bill also allows private persons to use Aadhaar as a proof of identity for any purpose, the provision will open a floodgate and enable private entities such as airlines, telecom, insurance and real estate companies to mandate Aadhaar as a proof of identity for availing their services.

Since the bill doesn’t restrict its application, people will not have a choice to identify themselves other than using Aadhaar when corporate organisations make it mandatory, says Chattapadhyay of the CIS. Adds Sarkar, “The bill should clearly mention sectors or services where Aadhaar will be potentially used (or made mandatory). Every time a new sector or service is added to the list, it is done after parliamentary approval.”

So far, 98 crore people have been assigned Aadhaar number. So far the project has costed Rs 8,000 crore.

For more details visit https://cis-india.org/internet-governance/news/gov-now-pratap-vikram-singh-17032016-why-aadhaar-is-baseless

Surveillance Project

sunil — 2016-04-05T15:21:27Z

The Aadhaar project’s technological design and architecture is an unmitigated disaster and no amount of legal fixes in the Act will make it any better.

The article will be published in Frontline, April 15, 2016 print edition.

Zero. The probability of some evil actor breaking into the central store of authentication factors (such as keys and passwords) for the Internet. Why? That is because no such store exists. And, what is the probability of someone evil breaking into the Central Identities Data Repository (CIDR) of the Unique Identification Authority of India (UIDAI)? Greater than zero. How do we know this? One, the central store exists and two, the Aadhaar Bill lists breaking into this central store as an offence. Needless to say, it would be redundant to have a law that criminalises a technological impossibility. What is the consequence of someone breaking into the central store? Remember, biometrics is just a fancy word for non-consensual and covert identification technology. High-resolution cameras can capture fingerprints and iris information from a distance.

In other words, on March 16, when Parliament passed the Bill, it was as if Indian lawmakers wrote an open letter to criminals and foreign states saying, “We are going to collect data to non-consensually identify all Indians and we are going to store it in a central repository. Come and get it!” Once again, how do I know that the CIDR will be compromised at some date in the future? How can I make that policy prediction with no evidence to back it up? To quote Sherlock Holmes, “Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.” If a back door to the CIDR exists for the government, then the very same back door can be used by an enemy within or from outside. In other words, the principle of decentralisation in cybersecurity does not require repeated experimental confirmation across markets and technologies.

Zero. The chances that you can fix with the law what you have broken with poor technological choices and architecture. And, to a large extent vice versa. Aadhaar is a surveillance project masquerading as a development intervention because it uses biometrics. There is a big difference between the government identifying you and you identifying yourself to the government. Before UID, it was much more difficult for the government to identify you without your knowledge and conscious cooperation. Tomorrow, using high-resolution cameras and the power of big data, the government will be able to remotely identify those participating in a public protest. There will be no more anonymity in the crowd. I am not saying that law-enforcement agencies and intelligence agencies should not use these powerful technologies to ensure national security, uphold the rule of law and protect individual rights. I am only saying that this type of surveillance technology is inappropriate for everyday interactions between the citizen and the state.

Some software engineers believe that there are technical fixes for these concerns; they point to the consent layer in the India stack developed through a public-private partnership with the UIDAI. But this is exactly what Evgeny Morozov has dubbed “technological solutionism”—fundamental flaws like this cannot be fixed by legal or technical band-aid. If you were to ask the UIDAI how do you ensure that the data do not get stolen between the enrolment machine and the CIDR, the response would be, we use state-of-the-art cryptography. If cryptography is good enough for the UIDAI why is it not good enough for citizens? That is because if citizens use cryptography [on smart cards] to identify themselves to the state, the state will need their conscious cooperation each time. That provides the feature that is required for better governance without the surveillance bonus. If you really must use biometrics, it could be stored on the smart card after being digitally signed by the enrolment officer. If there is ever a doubt whether the person has stolen the smart card, a special machine can be used to read the biometrics off the card and check that against the person. This way the power of biometrics would be leveraged without any of the accompanying harms.

Zero. This time, for the utility of biometrics as a password or authentication factor. There are two principal reasons for which the Act should have prohibited the use of biometrics for authentication. First, biometric authentication factors are irrevocable unlike passwords, PINs, digital signatures, etc. Once a biometric authentication factor has been compromised, there is no way to change it. The security of a system secured by biometrics is permanently compromised. Second, our biometrics is so easy to steal; we leave our fingerprints everywhere.

Also, if I upload my biometric data onto the Internet, I can then plausibly deny all transactions against my name in the CIDR. In order to prevent me from doing that, the government will have to invest in CCTV cameras [with large storage] as they do for passport-control borders and as banks do at ATMs. If you anyway have to invest in CCTV cameras, then you might as well stick with digital signatures on smart cards as the previous National Democratic Alliance (NDA) government proposed the SCOSTA (Smart Card Operating System Standard for Transport Application) standard for the MNIC (Multipurpose National ID Card). Leveraging smart card standards like EMV will ensure harnessing greater network effects thanks to the global financial infrastructure of banks. These network effects will drive down the cost of equipment and afford Indians greater global mobility. And most importantly when a digital signature is compromised the user can be issued a new smart card. As Rufo Guerreschi, executive director of Open Media Cluster, puts it, “World leaders and IT experts should realise that citizen freedoms and states’ ability to pursue suspects are not an ‘either or’ but a ‘both or neither’.”

Near zero. We now move biometrics as the identification factor. The rate of potential duplicates or “False Positive Identification Rate” which according to the UIDAI is only 0.057 per cent. Which according to them will result in only “570 resident enrolments will be falsely identified as duplicate for every one million enrolments.” However, according to an article published in Economic & Political Weekly by my colleague at the Centre for Internet and Society, Hans Verghese Mathews, this will result in one out of every 146 people being rejected during enrolment when total enrolment reaches one billion people. In its rebuttal, the UIDAI disputes the conclusion but offers no alternative extrapolation or mathematical assumptions. “Without getting too deep into the mathematics” it offers an account of “a manual adjudication process to rectify the biometric identification errors”.

This manual adjudication determines whether you exist and has none of the elements of natural justice such as notice to the affected party and opportunity to be heard. Elimination of ghosts is impossible if only machines and unaccountable humans perform this adjudication. This is because there is zero skin in the game. There are free tools available on the Internet such as SFinGe (Synthetic Fingerprint Generator) which allow you to create fake biometrics. The USB cables on the UIDAI-approved enrolment setup can be intercepted using generic hardware that can be bought online. With a little bit of clever programming, countless number of ghosts can be created which will easily clear the manual adjudication process that the UIDAI claims will ensure that “no one is denied an Aadhaar number because of a biometric false positive”.

Near zero. This time for surveillance, which I believe should be used like salt in cooking. Essential in small quantities but counterproductive even if slightly in excess. There is a popular misconception that privacy researchers such as myself are opposed to surveillance. In reality, I am all for surveillance. I am totally convinced that surveillance is good anti-corruption technology.

But I also want good returns on investment for my surveillance tax rupee. According to Julian Assange, transparency requirements should be directly proportionate to power; in other words, the powerful should be subject to more surveillance. And conversely, I add, privacy protections must be inversely proportionate to power—or again, in other words, the poor should be spared from intrusions that do not serve the public interest. The UIDAI makes the exact opposite design assumption; it assumes that the poor are responsible for corruption and that technology will eliminate small-ticket or retail corruption. But we all know that politicians and bureaucrats are responsible for most of large-ticket corruption.

Why does not the UIDAI first assign UID numbers to all politicians and bureaucrats? Then using digital signatures why do not we ensure that we have a public non-repudiable audit trail wherein everyone can track the flow of benefits, subsidies and services from New Delhi to the panchayat office or local corporation office? That will eliminate big-ticket or wholesale corruption. In other words, since most of Aadhaar’s surveillance is targeted at the bottom of the pyramid, there will be limited bang for the buck. Surveillance is the need of the hour; we need more CCTVs with microphones turned on in government offices than biometric devices in slums.

Instantiation technology

One. And zero. In the contemporary binary and digital age, we have lost faith in the old gods. Science and its instantiation technology have become the new gods. The cult of technology is intolerant to blasphemy. For example, Shekhar Gupta recently tweeted saying that part of the opposition to Aadhaar was because “left-libs detest science/tech”. Technology as ideology is based on some fundamental articles of faith: one, new technology is better than old technology; two, expensive technology is better than cheap technology; three, complex technology is better than simple technology; and four, all technology is empowering or at the very least neutral. Unfortunately, there is no basis in science for any of these articles of faith.

Let me use a simple story to illustrate this. I was fortunate to serve as a member of a committee that the Department of Biotechnology established to finalise the Human DNA Profiling Bill, 2015, which was to be introduced in Parliament in the last monsoon session. Aside: the language of the Act also has room for the database to expand into a national DNA database circumventing 10 years of debate around the controversial DNA Profiling Bill, 2015. The first version of this Bill that I read in January 2013 said that DNA profiling was a “powerful technology that makes it possible to determine whether the source of origin of one body substance is identical to that of another … without any doubt”. In other words, to quote K.P.C. Gandhi, a scientist from Truth Labs, “I can vouch for the scientific infallibility of using DNA profiling for carrying out justice.”

Unfortunately, though, the infallible science is conducted by fallible humans. During one of the meetings, a scientist described the process of generating a biometric profile. The first step after the laboratory technician generated the profile was to compare the generated profile with her or his own profile because during the process of loading the machine with the DNA sample, some of the laboratory technician’s DNA could have contaminated the sample. This error would not be a possibility in much older, cheaper and rudimentary biometric technology for example, photography. A photographer developing a photograph in a darkroom does not have to ensure that his or her own image has not accidentally ended up on the negative. But the UIDAI is filled with die-hard techno-utopians; if you tell them that fingerprints will not work for those who are engaged in manual labour, they will say then we will use iris-based biometrics. But again, complex technologies are more fragile and often come with increased risks. They may provide greater performance and features, but sometimes they are easier to circumvent. A gummy finger to fool a biometric scanner can be produced using glue and a candle, but to fake a passport takes a lot of sophisticated technology. Therefore, it is important for us as a nation to give up our unquestioning faith in technology and start to debate the exact technological configurations of surveillance technology for different contexts and purposes.

One. This time representing a monopoly. Prior to the UID project, nobody got paid when citizens identified themselves to the state. While the Act says that the UIDAI will get paid, it does not specify how much. Sooner or later, this cost of identification will be passed on to the citizens and residents. There will be a consumer-service provider relationship established between the citizen and the state when it comes to identification. The UIDAI will become the monopoly provider of identification and authentication services in India which is trusted by the government. That sounds like a centrally planned communist state to me. Should not the right-wing oppose the Act because it prevents the free market from working? Should not the free market pick the best technology and business model for identification and authentication? Will not that drive the cost of identification and authentication down and ensure higher quality of service for citizens and residents?

Competing providers

Competing providers can also publish transparency reports regarding their compliance with data requests from law-enforcement and intelligence agencies, and if this is important to consumers they will be punished by the market. The government can use mechanisms such as permanent and temporary bans and price regulation as disincentives for the creation of ghosts. There will be a clear financial incentive to keep the database clean. Just like the government established a regulatory framework for digital certificates in the Information Technology Act allowing for e-commerce and e-governance. Ideally, the Aadhaar Bill should have done something similar and established an ecosystem for multiple actors to provide services in this two-sided market. For it is impossible for a “small government” to have the expertise and experience to run one of the world’s largest database of biometric and transaction records securely for perpetuity.

To conclude, I support the use of biometrics. I support government use of identification and authentication technology. I support the use of ID numbers in government databases. I support targeted surveillance to reduce corruption and protect national security. But I believe all these must be put in place with care and thought so that we do not end up sacrificing our constitutional rights or compromising the security of our nation state. Unfortunately, the Aadhaar project’s technological design and architecture is an unmitigated disaster and no amount of legal fixes in the Act will make it any better. Our children will pay a heavy price for our folly in the years to come. To quote the security guru Bruce Schneier, “Data is a toxic asset. We need to start thinking about it as such, and treat it as we would any other source of toxicity. To do anything else is to risk our security and privacy.”

For more details visit https://cis-india.org/internet-governance/blog/frontline-april-15-2016-sunil-abraham-surveillance-project

Aadhaar: Still Too Many Problems

pranesh — 2016-04-06T15:31:32Z

While one wishes to welcome govt’s attempt to bring Aadhaar within a legislative framework, the fact is there are too many problems that still remain unaddressed for one to be optimistic.

The article was published by Livemint on March 7, 2016.

The Aadhaar Bill has been introduced as a money bill, even though it doesn’t qualify as such under Article 110 of the Constitution. If the Speaker agrees to this, it will render the Rajya Sabha toothless in this matter, and will weaken our democracy. The government should reintroduce it as an ordinary legislative bill, which is what it is.

While the government has in the past argued before the Supreme Court that Aadhaar is voluntary, Section 7 of the bill allows the government to mandate an Aadhaar number (or application for an Aadhaar number) as a prerequisite for obtaining some subsidies, benefits, services, etc. This undermines its arguments before the Supreme Court, which led the court to pass orders holding that Aadhaar should not be made mandatory. This move to make it mandatory will now need the government to argue that rather than contravene the apex court order, it has instead removed the rationale for it.

Interestingly, the Bharatiya Janata Party (BJP)-led National Democratic Alliance (NDA) government seems to have done a U-turn on the issue of the unique identification number not being proof of citizenship or domicile. The previous Congress-led United Progressive Alliance (UPA) government never meant the Aadhaar number to be proof of citizenship or domicile. This was attacked by the Yashwant Sinha-chaired standing committee on finance, which feared that illegal immigrants would get Aadhaar numbers. Now, the BJP and the NDA seem to be in agreement with the original UPA vision of Aadhaar.

Importantly, there is very strong language when it comes to the issue of privacy and confidentiality of the information that is held by the Unique Identification Authority of India (UIDAI). Section 29 (1), for instance, says that no biometric information will be shared for any reason whatsoever, or used for any purpose other than Aadhaar number generation and authentication. However, that provision is undermined wholly by Section 33, which says that “in the interest of national security”, the biometric info may be accessed if authorized by a joint secretary. This will only fan the fears of those who have argued that the real rationale for Aadhaar was not, in fact, delivery of services, but to create a national database of biometric data available to government snoops.

Also Read

Pros and cons of Aadhaar bill

Further, there are no remedies available for governmental abuse of this provision.

Lastly, in terms of privacy, the concern of those people who have been opposing Aadhaar is not just that the biometric and other identity information may be leaked to private parties, but also that having a unique Aadhaar number helps private parties to combine and use other databases that are linked with Aadhaar numbers in a manner that is not within the subject’s control. This is not at all addressed in this bill, and we need a robust data protection law in order to do that.

There are some other crucial details that the law doesn’t address: Is user consent, to be taken by third parties that use the UID database for authentication, needed for each instance of authentication, or would a general consent hold forever? How can consent be revoked?

There were many other objections that were raised against the Aadhaar scheme that have not been addressed by the government. For instance, in a recent article in the Economic and Political Weekly, Hans Varghese Mathews points out that going by the test data UIDAI made available in 2012, for a population of 1.3 billion people, the incidence of false positives—the probability of the identities of two people matching—is 1/112.

This is far too high a ratio to be acceptable.

Actual data from the field in Andhra Pradesh—of people who were unable to claim rations under the public distribution system (PDS)—paints a worse picture. A survey commissioned by the Andhra Pradesh government said 48% of respondents pointed to Aadhaar-related failures as the cause of their inability to claim rations.

So, even if the Aadhaar numbers were no longer issued to Lord Hanuman (Rajasthan), to dogs (e.g., Tommy Singh, a mutt in Madhya Pradesh), and with photos of a tree (New Delhi), it might not prove to be usable in a country of India’s size, given the capabilities of the fingerprint machines. As my colleague Sunil Abraham notes, the law cannot fix technological flaws.

So, while one wishes one could welcome the government’s attempt to bring Aadhaar within a legislative framework, the fact is there are too many problems that still remain unaddressed for one to be optimistic.

Pranesh Prakash is policy director at the Centre for Internet and Society, a think tank.

For more details visit https://cis-india.org/internet-governance/blog/livemint-march-7-2016-pranesh-prakash-aadhaar-still-too-many-problems

GNI-Industry Dialogue Learning Session: Human Rights Impact Assessments and Due Diligence in the ICT sector

elonnai — 2016-04-06T15:42:41Z

Elonnai Hickok attended the meeting organized by Global Network Initiative on March 11, 2016 in Washington D.C.

The GNI welcomed its new observers from the Telecommunications Industry Dialogue by holding a learning session in conjunction with the GNI Board Meeting on March 10. This learning session aimed to increase understanding between the GNI and the ID by examining some of the common challenges that face ICT companies in the area of human rights due diligence and highlighting good practices. A second objective was to help the GNI develop a learning program and materials that will be useful for its members and draw on their expertise. Finally, this learning session informed the review of the GNI Implementation Guidelines that will take place during 2016.

The session took place according to the Chatham House Rule. Each short presentation was followed by a space for questions and answers.

Human Rights Impact Assessments in the ICT sector – Michael Samway
The Human Rights Due Diligence Process at Nokia – Laura Okkonen
Yahoo’s approach to Human Rights Impact Assessments– Nicole Karlebach and Katie Shay
Orange’s challenges and approach to doing business in Africa – Yves Nissim
Microsoft’s human rights impacts and the warrant case – Steve Crown and Bernard Shen
TeliaSonera’s approach to withdrawing from Eurasia – Patrik Hiselius
Considerations for company due diligence on the ground – Kathleen Reen and Babette Ngene, Internews

For discussion:

What are some of the common challenges facing current GNI member companies and ID member companies?
What do we consider to be good practices that are applicable to all?
What lessons can be applied to the review of the GNI Implementation Guidelines that will take place during 2016?

For more details visit https://cis-india.org/internet-governance/news/gni-industry-dialogue-learning-session-human-rights-impact-assessments-and-due-diligence-in-the-ict-sector

Infographic: The Impending Right to Privacy Judgment

Amber Sinha and Pooja Saxena — 2017-08-22T23:50:44Z

The ruling will be important not just for the immediate Aadhaar case but also numerous other matters to do with state intrusions, decisional autonomy and informational privacy.

The article was published in the Wire on August 17, 2017.

Over the last month, a nine-judge constitutional bench of the Supreme Court has heard arguments on the existence of a fundamental right to privacy in India. Media coverage of judicial hearings in the apex court is often ripe with inaccuracies, thanks in no small measure to the court’s own restrictive policies, which, for instance, prevent video recordings. In this case, the arguments – which were heard over the course of three weeks – were widely reported in much greater detail and with fidelity, thanks largely to the live tweets by Gautam Bhatia and Prasanna S. (the entire collection of tweets is available here).

The availability of the entire set of written arguments made available by LiveLaw was another rich source for anyone following this matter in detail. The ruling by the bench will be of extreme importance not just for the immediate Aadhaar case, which has witnessed gross delays, but also numerous other matters in the future to do with state intrusions, decisional autonomy and informational privacy.

The questions before this bench are two fold – do the judgments in M.P. Sharma and Others vs Satish Chandra (decided by an eight-judge bench in 1954) and Kharak Singh vs State of UP and Others (decided by a six-judge bench in 1962) lead to the conclusion that there is no fundamental right to privacy, and whether the decisions in the later cases upholding a right to privacy were correct.

This infographic tries to unpack the hearings in the court into distinct issues, and the key arguments advanced by both the sides on them. The arguments from both sides on a particular question have been presented side by side for better appreciation, even though they were not argued together

Given the nature of the exercise, some of the arguments made in the infographic are bound to be a simplification of the actual issue. But it is hoped that this will provide a good overview of the issues argued.

Research and writing by Amber Sinha. Design by Pooja Saxena. Amber Sinha is a lawyer and works at the Centre for Internet and Society. Pooja Saxena is a typeface and graphic designer, specialising in Indic scripts.

For more details visit https://cis-india.org/internet-governance/blog/the-wire-amber-sinha-and-pooja-saxena-august-17-2017-infographic-the-impending-right-to-privacy-judgment

MediaNama - #NAMAprivacy: The Future of User Data (Delhi, Sep 6)

sumandro — 2017-09-05T10:22:12Z

MediaNama is hosting a full day conference on "the future of user data in India", on the 6th of September 2017, which is particularly significant given the recent Supreme Court ruling on the fundamental right to privacy, and two government consultations: one at the TRAI, and another at MEITY. This discussion is supported by Facebook, Google, and Microsoft. Sumandro Chattapadhyay, Research Director, will participate as a speaker in the session titled "regulating storage, sharing and transfer of data."

Details

Time: September 6th 2017, 9 am to 4:30 pm

Venue: Gulmohar Hall, India Habitat Centre, Lodhi Road (please enter from Gate #3)

Agenda: https://www.medianama.com/2017/08/223-agenda-namaprivacy-future-of-user-data/

Announced Speakers

Chinmayi Arun, Centre for Communication Governance at NLU Delhi
Malavika Raghavan, IFMR Finance Foundation
Renuka Sane, NIPFP
Smitha Krishna Prasad, Centre for Communication Governance at NLU Delhi
Ananth Padmanabhan, Carnegie India
Avinash Ramachandra, Amazon
Hitesh Oberoi, Naukri
Jochai Ben-Avie, Mozilla
Mrinal Sinha, Mobikwik
Murari Sreedharan, Bankbazaar
Sumandro Chattapadhyay, Centre for Internet and Society

Facilitators

Saikat Datta, Asia Times Online
Shashidar KJ, MediaNama
Nikhil Pahwa, MediaNama

Attendees

We have confirmed 140+ attendees from: Adobe, Amber Health, Amazon, APCO Worldwide, Bank Bazaar, Bloomberg-Quint, Blume Ventures, Broadband India Forum, Business Standard, BuzzFeed News, CCOAI, CEIP, Change Alliance, Chase India, CIS, CNN News18, DEF, Deloitte, DNA, DSCI, E2E Networks, British High Commission, Eurus Network Services, FICCI, Firefly Networks, Flipkart, Forrester Research, Fortumo, DoT, MEITY, IAMAI, IBM, ICRIER, IFMR Finance Foundation, IIMC, Indian Law Institute, Indic Project, Info Edge, ISPAI, IT for Change, ITU-APT, Jamia Millia Islamia, Jindal Global Law School, Mimir Technologies, Mozilla, Newslaundry, NIPFP, Nishith Desai Associates, NIXI, NLU-Delhi, ORF, Paytm, PLR Chambers, PRS Legislative Research, Publicis Groupe, Quartz India, Reliance Jio, Reuters, Saikrishna & Associates, Scroll.in, SFLC.in, Spectranet, The Economics Times, The Indian Express, The Times of India, The Wire, Times Internet, Twitter, and more.

For more details visit https://cis-india.org/internet-governance/news/medianama-namaprivacy-the-future-of-user-data-delhi-sep-6

Artificial Intelligence in Governance: A Report of the Roundtable held in New Delhi

Saman Goudarzi and Natallia Khaniejo — 2018-05-03T15:49:40Z

This Report provides an overview of the proceedings of the Roundtable on Artificial Intelligence (AI) in Governance, conducted at the Indian Islamic Cultural Centre, in New Delhi on March 16, 2018. The main purpose of the Roundtable was to discuss the deployment and implementation of AI in various aspects of governance within the Indian context. This report summarises the discussions on the development and implementation of AI in various aspects of governance in India. The event was attended by participants from academia, civil society, the legal sector, the finance sector, and the government.

Event Report: Download (PDF)

This report provides a summary of the proceedings of the Roundtable on Artificial Intelligence (AI) in Governance (hereinafter referred to as ‘the Roundtable’). The Roundtable took place at the India Islamic Cultural Centre in New Delhi on March 16, 2018 and included participation from academia, civil society, law, finance, and government. The main purpose of the Roundtable was to discuss the deployment and implementation of AI in various aspects of governance within the Indian context.

The Roundtable began with a presentation by Amber Sinha (Centre for Internet and Society - CIS) providing an overview of the CIS’s research objectives and findings thus far. During this presentation, he defined both AI and the scope of CIS’s research, outlining the areas of law enforcement, defense, education, judicial decision making, and the discharging of administrative functions as the main areas of concerns for the study. The presentation then outlined the key AI deployments and implementations that have been identified by the research in each of these areas. Lastly, the presentation raised some of the ethical and legal concerns related to this phenomenon.

The presentation was followed by the Roundtable discussion that saw various topics in regards to the usages, challenges, ethical considerations and implications of AI in the sector being discussed. This report has identified a number of key themes of importance evident throughout these discussions.These themes include: (1) the meaning and scope of AI, (2) AI’s sectoral applications, (3) human involvement with automated decision making, (4) social and power relations surrounding AI, (5) regulatory approaches to AI and, (6) challenges to adopting AI. These themes in relation to the Roundtable are explored further below.

Meaning and Scope of AI

One of the first tasks recommended by the group of participants was to define the meaning and scope of AI and the way those terms are used and adopted today. These concerns included the need to establish a distinction between the use of algorithms, machine learning, automation and artificial intelligence. Several participants believed that establishing consensus around these terms was essential before proceeding towards a stage of developing regulatory frameworks around them.

The general fact agreed to was that AI as we understand it does not necessarily extend to complete independence in terms of automated decision making but it refers instead to the varying levels of machine learning (ML), and the automation of certain processes that has already been achieved. Several concerns that emerged during the course of the discussion centred around the question of autonomy and transparency in the process of ML and algorithmic processing. Stakeholders recommended that over and above the debates of humans in the loop [1] on the loop [2] and out of the loop, [3] there were several other gaps with respect to AI and its usage in the industry today which also need to be considered before building a roadmap for future usage. Key issues like information asymmetries, communication lags, a lack of transparency, the increased mystification of the coding process and the centralization of power all needed to be examined and analysed under the rubric of developing regulatory frameworks.

Takeaway Point: The group brought out the need for standardization of terminology as well as the establishment of globally replicable standards surrounding the usage, control and proliferation of AI. The discussion also brought up the problems with universal applicability of norms. One of the participants brought up an issue regarding the lack of normative frameworks around the usage and proliferation of AI. Another participant responded to the concern by alluding to the Asilomar AI principles.[4] The Asilomar AI principles are a set of 23 principles aimed at directing and shaping AI research in the future. The discussion brought out further issues regarding the enforceability as well universal applicability of the principles and their global relevance as well. Participants recommended the development of a shorter, more universally applicable regulatory framework that could address various contextual limitations as well.

AI Sectoral Applications

Participants mentioned a number of both current and potential applications of AI technologies, referencing the defence sector, the financial sector, and the agriculture sector. There are several developments taking place on the Indian military front with the Committee on AI and National Security being established by the Ministry of Defence. Through the course of the discussion it was also stated that the Indian Armed Forces were very interested in the possibilities of using AI for their own strategic and tactical purposes. From a technological standpoint, however, there has been limited progress in India in researching and developing AI.

While India does deploy some Unmanned Aerial Vehicles (UAVs), they are mostly bought from Israel, and often are not autonomous. It was also pointed out that contrary to reportage in the media, the defence establishment in India is extremely cautious about the adoption of autonomous weapons systems, and that the autonomous technology being rolled out by the CAIR is not yet considered trustworthy enough for deployment.

Discussions further revealed that the few technologies that have a relative degree of autonomy are primarily loitering ammunitions and are used to target radar insulations for reconnaissance purposes. One participant mentioned that while most militaries are interested in deploying AI, it is primarily from an Intelligence, Surveillance and Reconnaissance (ISR) perspective. The only exception to this generalization is China where the military ethos and command structure would work better with increased reliance on independent AI systems. One major AI system rolled out by the US is Project Maven which is primarily an ISR system. The aim of using these systems is to improve decision making and enhance data analysis particularly since battlefields generate a lot of data that isn’t used anywhere.

Another sector discussed was the securities market where algorithms were used from an analytical and data collection perspective. A participant referred to the fact that machine learning was being used for processes like credit and trade scoring -- all with humans on the loop. The participant further suggested that while trade scoring was increasingly automated, the overall predictive nature of such technologies remained within a self limiting capacity wherein statistical models, collected data and pattern analysis were used to predict future trends. The participant questioned whether these algorithms could be considered as AI in the truest sense of the term since they primarily performed statistical functions and data analysis.

One participant also recommended the application of AI to sectors like agriculture with the intention of gradually acclimatizing users to the technology itself. Respondents also stated that while AI technologies were being used in the agricultural space it was primarily from the standpoint of data collection and analysis as opposed to predictive methods. It was mentioned that a challenge to the broad adoption of AI in this sector is the core problem of adopting AI as a methodology – namely information asymmetries, excessive data collection, limited control/centralization and the obfuscatory nature of code – would not be addressed/modified. Lastly, participants also suggested that within the Indian framework not much was being done aside from addressing farmers’ queries and analysing the data from those concerns.

Takeaway Point: The discussion drew attention to the various sectors where AI was currently being used -- such as the military space, agricultural development and the securities market -- as well as potential spaces of application -- such as healthcare and manual scavenging. The key challenges that emerged were information asymmetries with respect to the usage of these technologies as well as limited capacity in terms of technological advancement.

Human Involvement with Automated Decision Making

Large parts of discussions throughout the Roundtable event were preoccupied with automated decision making and specifically, the involvement of humans (human on and in the loop) or lack thereof (human out of the loop) in this process. These discussions often took place with considerations of AI for prescriptive and descriptive uses.

Participants expressed that human involvement was not needed when AI was being used for descriptive uses, such as determining relationships between various variables in large data sets. Many agreed to the superior ability of ML and similar AI technologies in describing large and unorganized datasets. It was the prescriptive uses of AI where participants saw the need for human involvement, with many questioning the technology making more important decisions by itself.

The need for human involvement in automated decision making was further justified by references to various instances of algorithmic bias in the American context. One participant, for example, brought up the use of algorithmic decision making by a school board in the United States for human resource practices (hirings, firing, etc.) based on the standardized test scores of students. In this instance, such practices resulted in the termination of teachers primarily from low income neighbourhoods.[5] The main challenge participants identified in regards to human on the loop automated decision making is the issue of capacity, as significant training would have to be achieved for sectors to have employees actively involved in the automated decision making workflow.

An example in the context of the healthcare field was brought up by one participant arguing for human in the loop in regards to prescriptive scenarios. The participant suggested that AI technology, when given x-ray or MRI data for example, should only be limited to pointing out the correlations of diseases with patients’ scans/x-rays. Analysis of such correlations should be reserved for the medical expertise of doctors who would then determine if any instances of causality can be identified from this data and if it’s appropriate for diagnosing patients.

It was emphasized that, despite a preference for human on/in the loop in regards to automated decision making, there is a need to be cognisant of techno-solutionism due to the human tendency of over reliance on technology when making decisions. A need for command and control structures and protocols was emphasized for various governance sectors in order to avoid potentially disastrous results through a checks and balances system. It was noted that the defense sector has already developed such protocols, having established a chain of command due to its long history of algorithmic decision making (e.g. the Aegis Combat System being used by the US Navy in the 1980s).

One key reason why militaries prefer human in and on the loop systems as opposed to out of the loop systems is because of the protocol associated with human action on the battlefield. International Humanitarian Law has clear indicators of what constitutes a war crime and who is to be held responsible in the scenario but developing such a framework with AI systems would be challenging as it would be difficult to determine which party ought to be held accountable in the case of a transgression or a mistake.

Takeaway Point: It was reiterated by many participants that neither AI technology or India’s regulatory framework is at a point where AI can be trusted to make significant decisions alone -- especially when such decisions are evaluating humans directly. It was recommended that human out of the loop decision making should be reserved for descriptive practices whereas human on and in the loop decision making should be used for prescriptive practices. Lastly, it was also suggested that appropriate protocols be put in place to direct those involved in the automated decision making workflow. Particularly when the process involves judgements and complex decision making in sectors such as jurisprudence and the military.

The Social and Power Relations Surrounding AI

Some participants emphasized the need to contextualize discussions of AI and governance within larger themes of poverty, global capital and power/social relations. Their concerns were that the use of AI technologies would only create and reinforce existing power structures and should instead be utilized towards ameliorating such issues. Manual scavenging, for example, was identified as an area where AI could be used to good effect if coupled with larger socio-political policy changes. There are several hierarchies that could potentially be reinforced through this process and all these failings needed to be examined thoroughly before such a system was adopted and incorporated within the real world.

Furthermore the discussion also revealed that the objectivity attributed to AI and ML tends to gloss over the fact that there are nonetheless implicit biases that exist in the minds of the creators that might work themselves into the code. Fears regarding technology recreating a more exclusionary system were not entirely unfounded as participants pointed out the fact that the knowledge base of the user would determine whether technology was used as a tool of centralization or democratization.

One participant also questioned the concept of governance itself, contrasting the Indian government’s usage of the term in the 1950s (as it appears in the Directive Principle) with that of the World Bank in the 1990s.

Takeaway Point: Discussions of the implementation and deployment of AI within the governance landscape should attempt to take into consideration larger power relations and concepts of equity.

Regulatory Approaches to AI

Many recognized the need for AI-specific regulations across Indian sectors, including governance. These regulations, participants stated, should draw from notions of accountability, algorithmic transparency and efficiency. Furthermore, it was also stated that such regulations should consider the variations across the different legs of the governance sector, especially in regards to defence. One participant, pointing to the larger trends towards automation, recommended the establishment of certain fundamental guidelines aimed at directing the applicability of AI in general. The participant drew attention to the need for a robust evaluation system for various sectors (the criminal justice system, the securities market, etc.) as a way of providing checks on algorithmic biases. Another emphasized for the need of regulations for better quality data as to ensure machine readability and processiblity for various AI systems.

Another key point that emerged was the importance of examining how specific algorithms performed processes like identification or detection. A participant recommended the need to examine the ways in which machines identify humans and what categories/biases could infiltrate machine-judgement. They reiterated that if a new element was introduced in the system, the pre-existing variables would be impacted as well. The participant further recommended that it would be useful to look at these systems in terms of the couplings that get created in order to determine what kinds of relations are fostered within that system.

The roundtable saw some debate regarding the most appropriate approach to developing such regulations. Some participants argued for a harms-based approach, particularly in regards to determining if regulations are needed all together for specific sectors (as opposed to guidelines, best practices, etc.). The need to be cognisant of both individual and structural harms was emphasized, mindful of the possibility of algorithmic biases affecting traditionally marginalized groups.

Others only saw value in a harms based approach insomuch that it could help outline the appropriate penalties in an event of regulations being violated, arguing instead for a rights-based approach as it enabled greater room for technological changes. An approach that kept in mind emerging AI technologies was reiterated by a number of participants as being crucial to any regulatory framework. The need for a regulatory space that allowed for technological experimentation without the fear of constitutional violation was also communicated.

Takeaway Point: The need for a AI-specific regulatory framework cognisant of differentiations across sectors in India was emphasized. There is some debate about the most appropriate approach for such a framework, a harms-based approach being identified by many as providing the best perspective on regulatory need and penalties. Some identified the rights-based approach as providing the most flexibility for an rapidly evolving technological landscape.

Challenges to Adopting AI

Out of all the concerns regarding the adoption of algorithms, ML and AI, the two key points of resistance that emerged, centred around issues of accountability and transparency. Participants suggested that within an AI system, predictability would be a key concern, and in the absence of predictable outcomes, establishing redressal mechanisms would pose key challenges as well.

A discussion was also initiated regarding the problems involved in attributing responsibility within the AI chain as well as the need to demystify the process of using AI in daily life. While reiterating the current landscape, participants spoke about how the usage of AI is currently limited to the automation of certain tasks and processes in certain sectors where algorithmic processing is primarily used as a tool of data collection and analysis as opposed to an independent decision making tool.

One of the suggestions and thought points that emerged during the discussion was whether a gradual adoption of AI on a sectoral basis might be more beneficial as it would provide breathing room in the middle to test the system and establish trust between the developers, providers, and consumers. This prompted a debate about the controllers and the consumers of AI and how the gap between the two would need to be negotiated. The debate also brought up larger concerns regarding the mystification of AI as a process itself and the complications of translating the code into communicable points of intervention.

Another major issue that emerged was the question of attribution of responsibility in the case of mistakes. In the legal process as it currently exists, human imperfections notwithstanding, it would be possible to attribute the blame for decisions taken to certain actants undertaking the action. Similarly in the defence sector, it would be possible to trace the chain of command and identify key points of failure, but in the case of AI based judgements, it would be difficult to place responsibility or blame. This observation led to a debate regarding accountability in the AI chain. It was inconclusive whether the error should be attributed to the developer, the distributor or the consumer.

A suggestion that was offered in order to counter the information asymmetry as well as reduce the mystification of computational method was to make the algorithm and its processes transparent. This sparked a debate, however, as participants stated that while such a state of transparency ought to be sought after and aspired towards, it would be accompanied by certain threats to the system. A key challenge that was pointed out was the fact that if the algorithm was made transparent, and its details were shared, there would be several ways to manipulate it, translate it and misuse it.

Another question that emerged was the distribution of AI technologies and the centralization of the proliferation process particularly in terms of service provision. One participant suggested that given the limited nature of research being undertaken and the paucity of resources, a limited number of companies would end up holding the best tech, the best resources and the best people. They further suggested that these technologies might end up being rolled out as a service on a contractual basis. In which case it would be important to track how the service was being controlled and delivered. Models of transference would become central points of negotiation with alternations between procurement based, lease based, and ownership based models of service delivery. Participants suggested that this was going to be a key factor in determining how to approach these issues from a legal and policy standpoint.

Takeaway Point: The two key points of resistance that emerged during the course of discussion were accountability and transparency. Participants pointed out the various challenges involved in attributing blame within the AI chain and they also spoke about the complexities of opening up AI code, thereby leaving it vulnerable to manipulation. Certain other challenges that were briefly touched upon were the information asymmetry, excessive data collection, centralization of power in the hands of the controllers and complicated service distribution models.

Conclusion

The Roundtable provided some insight into larger debates regarding the deployment and applications of AI in the governance sector of India. The need for a regulatory framework as well as globally replicable standards surrounding AI was emphasized, particularly one mindful of the particular needs of differing fields of the governance sector (especially defence). Furthermore, a need for human on/in the loop practices with regards to automated decision making was highlighted for prescriptive instances, particularly when such decisions are responsible for directly evaluating humans. Contextualising AI within its sociopolitical parameters was another key recommendation as it would help filter out the biases that might work themselves into the code and affect the performance of the algorithm. Further, it is necessary to see the involvement and influence of the private sector in the deployment of AI for governance, it often translating into the delivery of technological services from private actors to public bodies towards discharge of public functions. This has clear implications for requirements of transparency and procedural fairness even in private sector delivery of these services. Defining the meaning and scope of AI while working to demystify algorithms themselves would serve to strengthen regulatory frameworks as well as make AI more accessible for the user / consumer.

[1]. Automated decision making model where final decisions are made by a human operator

[2]. Automated decision making model where decisions can be made without human involvement but a human can override the system.

[3]. A completely autonomous decision making model requiring no human involvement

[4]. https://futureoflife.org/ai-principles/

[5]. The participant was drawing this example from Cathy O’Neil’s Weapons of Math Destruction, (Penguin,2016), at 4-13.

For more details visit https://cis-india.org/internet-governance/blog/artificial-intelligence-in-governance-a-report-of-the-roundtable-held-in-new-delhi

What’s up with WhatsApp?

Aayush Rathi and Sunil Abraham — 2018-04-23T16:45:51Z

In 2016, WhatsApp Inc announced it was rolling out end-to-end encryption, but is the company doing what it claims to be doing?

The article by Aayush Rathi and Sunil Abraham was published in Asia Times on April 20, 2018.

Back in April 2016, when WhatsApp Inc announced it was rolling out end-to-end encryption (E2EE) for its billion-plus strong user base as a default setting, the messaging behemoth signaled to its users it was at the forefront of providing technological solutions to protect privacy.

Emphasized in the security white paper explaining the implementation of the technology is the encryption of both forms of communication – one-to-one and group and also of all types of messages shared within such communications – text as well as media.

Simply put, all communication taking place over WhatsApp would be decipherable only to the sender and recipient – it would be virtual gibberish even to WhatsApp.

This announcement came in the backdrop of Apple locking horns with the FBI after being asked to provide a backdoor to unlock the San Bernardino mass shooter’s iPhone. This further reinforced WhatsApp Inc’s stand on the ensuing debate between the interplay of privacy and security in the digital age.

Kudos to WhatsApp, for there is growing discussion around how encryption and anonymity is central to enabling secure online communication which in turn is integral to essential human rights such as those of freedom of opinion and expression.

WhatsApp may have taken encryption to the masses, but here we outline why WhatsApp’s provisioning of privacy and security measures needs a more granular analysis – is the company doing what it claims to be doing? Security issues with WhatsApp’s messaging protocol certainly are not new.

Man-in-the-middle attacks

A study published by a group of German researchers from Ruhr University highlighted issues with WhatsApp’s implementation of its E2EE protocol to group communications. Another paper points out how WhatsApp’s session establishment strategy itself could be problematic and potentially be targeted for what are called man-in-the-middle (MITM) attacks.

An MITM attack takes the form of a malicious actor, as the term suggests, placing itself between the communicating parties to eavesdrop or impersonate. The Electronic Frontier Foundation also highlighted other security vulnerabilities, or trade-offs, depending upon ideological inclinations, with respect to WhatsApp allowing for storage of unencrypted backups, issues with WhatsApp’s web client and also with its approach to cryptographic key change notifications.

Much has been written questioning WhatsApp’s shifting approach to ensuring privacy too. Quoting straight from WhatsApp’s Privacy Policy: “We joined the Facebook family of companies in 2014. As part of the Facebook family of companies, WhatsApp receives information from, and shares information with, this family of companies.” Speaking of Facebook …

Culling out larger issues with WhatsApp’s privacy policies is not the intention here. What we specifically seek to explore is right at the nexus of WhatsApp’s security and privacy provisioning clashing with its marketing strategy: the storage of data on WhatsApp’s servers, or ‘blobs,’ as they are referred to in the technical paper. Facebook’s rather. In WhatsApp’s words: “Once your messages (including your chats, photos, videos, voice messages, files and share location information) are delivered, they are deleted from our servers. Your messages are stored on your own device.”

In fact, this non-storage of data on their ‘blobs’ is emphasizes at several other points on the official website. Let us call this the deletion-upon-delivery model.

A simple experiment

While drawing up a rigorous proof of concept, made near-impossible thanks to WhatsApp being a closed source messaging protocol, a simple experiment is enough to raise some very pertinent questions about WhatsApp’s outlined deletion-upon-delivery model. It should, however, be mentioned that the Signal Protocol developed by Open Whisper Systems and pivotal in WhatsApp’s rolling out of E2EE is open source. Here is how the experiment proceeds:

Rick sends Morty an attachment.

Morty then switches off the data on her mobile device.

Rick downloads the attachment, an image.

Subsequently, Rick deletes the image from his mobile device’s internal storage.

Rick then logs into a WhatsApp’s web client on his browser. (Prior to this experiment, both Rick and Morty had logged out from all instances of the web client)

Upon a fresh log-in to the web client and opening the chat with Morty, the option to download the image is available to Rick.

The experiment concludes with bewilderment at WhatsApp’s claim of deletion-upon-delivery as outlined earlier. The only place from which Morty could have downloaded the image would be from Facebook’s ‘blobs.’ The attachment could not have been retrieved from Morty’s mobile device as it had no way of sending data and neither from Rick’s mobile device as it no longer existed in the device’s storage.

As per the Privacy Policy, the data is stored on the ‘blobs’ for a period of 30 days after transmission of a message only when it can’t be delivered to the recipient. Upon delivery, the deletion-upon-delivery model is supposed to kick in.

Another straightforward experiment that leads to a similar conclusion is seeing the difference in time taken for a large attachment to be forwarded as opposed to when the same large attachment is uploaded. Forwarding is palpably quicker than uploading afresh: non-storage of attachments on the ‘blob’ would entail that the same amount should be taken for both.

The plot thickens. WhatsApp’s Privacy Policy goes on to state: “To improve performance and deliver media messages more efficiently, such as when many people are sharing a popular photo or video, we may retain that content on our servers for a longer period of time.” The technical paper offers no help in understanding how WhatsApp systems assess frequently shared encrypted media messages without decrypting it at its end.

A possible explanation could be the usage of metadata by WhatsApp, which it discloses in its Privacy Policy while simultaneously being sufficiently vague about the specifics of it. That WhatsApp may be capable of reading encrypted communication through the inclusion of a backdoor bodes well for law enforcement, but not so much for unsuspecting users.

The weakest link in the chain

Concerns about backdoors in WhatsApp’s product have led the French government to start developing their own encrypted messaging service. This will be built using Matrix – an open protocol designed for real-time communication. Indeed, the Privacy Policy lays out that the company “may collect, use, preserve, and share your information if we have a good-faith belief that it is reasonably necessary to respond pursuant to applicable law or regulations, to legal process, or to government requests.”

The Signal Protocol is the undisputed gold standard of E2EE implementations. It is the integration with the surrounding functionality that WhatsApp offers which leads to vulnerabilities. After all, a chain is only as strong as its weakest link. Assuming that the attachments stored on the ‘blobs’ are in encrypted form, indecipherable to all but the intended recipients, this does not pose a privacy risk for the users from a technological point of view.

However, it is easy lose sight of the fact that the Privacy Policy is a legally binding document and it specifically states that messages are not stored on the ‘blobs’ as a matter of routine. As a side note, WhatsApp’s Privacy Policy and Terms of Service are refreshing in their readability and lack of legalese.

As we were putting the final touches to this piece, news from WABetaInfo, a well-reputed source of information on WhatsApp features, has broken that newer updates of WhatsApp for Android are permitting users to re-download media deleted up to three months back. WhatsApp cannot possibly achieve this without storing the media in the ‘blobs,’ or in other words, in violation of its Privacy Policy.

As the aphorism goes: “When the service is free, you are the product.”

For more details visit https://cis-india.org/internet-governance/blog/asia-times-april-20-2018-aayush-rathi-sunil-abraham-what-s-up-with-whatsapp